2023-11-28T10:55:17.185 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T10:55:17.185 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} --name neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi --owner scheduled_neesingh@teuthology --archive /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956 --verbose -- /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956/orig.config.yaml 2023-11-28T10:55:17.240 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T10:55:17.324 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956 branch: wip-neesingh-testing-231123 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7469956' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-neesingh-testing-231123 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: issue_54557 sha1: e67b67438c1e5e13734a7203ff9e57e30b8e4d32 owner: scheduled_neesingh@teuthology priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9437 sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 sleep_before_teardown: 0 suite: fs suite_branch: issue_54557 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: e67b67438c1e5e13734a7203ff9e57e30b8e4d32 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 5ae09a5cc7064ca2bf994df19f3006b79794789a timestamp: 2023-11-28_10:48:03 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T10:55:17.325 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa; will attempt to use it 2023-11-28T10:55:17.326 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks 2023-11-28T10:55:17.326 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T10:55:17.328 INFO:teuthology.task.internal:Checking packages... 2023-11-28T10:55:17.352 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '2f5f6028b893ad41357cd06e827373f0fe2a1232' 2023-11-28T10:55:17.352 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T10:55:17.352 INFO:teuthology.packaging:ref: None 2023-11-28T10:55:17.352 INFO:teuthology.packaging:tag: None 2023-11-28T10:55:17.353 INFO:teuthology.packaging:branch: wip-neesingh-testing-231123 2023-11-28T10:55:17.353 INFO:teuthology.packaging:sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T10:55:17.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-neesingh-testing-231123 2023-11-28T10:55:17.568 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2697.g2f5f6028 2023-11-28T10:55:17.568 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T10:55:17.574 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T10:55:17.575 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T10:55:17.587 INFO:teuthology.task.internal:Saving configuration 2023-11-28T10:55:17.603 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T10:55:17.609 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T10:55:17.635 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:49:56.503330', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T10:55:17.659 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:49:56.501948', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T10:55:17.660 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T10:55:17.665 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-28T10:55:17.666 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-28T10:55:17.666 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T10:55:17.735 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7eff8e78aa60>, signals=[15]) 2023-11-28T10:55:17.735 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T10:55:17.741 INFO:teuthology.task.internal:Opening connections... 2023-11-28T10:55:17.741 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T10:55:17.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:55:17.810 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T10:55:17.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:55:17.877 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T10:55:17.884 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2023-11-28T10:55:17.903 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2023-11-28T10:55:17.904 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2023-11-28T10:55:17.962 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T10:55:17.962 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T10:55:17.962 INFO:teuthology.orchestra.run.smithi156.stdout:ID="rhel" 2023-11-28T10:55:17.962 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="fedora" 2023-11-28T10:55:17.963 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="8.6" 2023-11-28T10:55:17.963 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el8" 2023-11-28T10:55:17.963 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T10:55:17.963 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2023-11-28T10:55:17.963 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T10:55:17.963 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T10:55:17.963 INFO:teuthology.orchestra.run.smithi156.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T10:55:17.964 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T10:55:17.964 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T10:55:17.964 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T10:55:17.964 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T10:55:17.964 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T10:55:17.964 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T10:55:17.965 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-11-28T10:55:17.985 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2023-11-28T10:55:18.007 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2023-11-28T10:55:18.007 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2023-11-28T10:55:18.066 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T10:55:18.066 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T10:55:18.066 INFO:teuthology.orchestra.run.smithi186.stdout:ID="rhel" 2023-11-28T10:55:18.066 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="fedora" 2023-11-28T10:55:18.066 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="8.6" 2023-11-28T10:55:18.066 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el8" 2023-11-28T10:55:18.066 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T10:55:18.067 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2023-11-28T10:55:18.067 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T10:55:18.067 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T10:55:18.067 INFO:teuthology.orchestra.run.smithi186.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T10:55:18.067 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T10:55:18.067 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T10:55:18.067 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T10:55:18.067 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T10:55:18.067 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T10:55:18.067 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T10:55:18.068 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2023-11-28T10:55:18.084 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T10:55:18.094 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T10:55:18.103 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T10:55:18.103 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2023-11-28T10:55:18.105 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2023-11-28T10:55:18.124 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T10:55:18.130 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T10:55:18.131 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2023-11-28T10:55:18.164 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2023-11-28T10:55:18.185 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T10:55:18.239 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T10:55:18.252 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-11-28T10:55:18.252 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-11-28T10:55:18.252 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-11-28T10:55:18.253 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T10:55:18.253 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T10:55:18.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:18.271 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2023-11-28T10:55:18.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:18.272 DEBUG:teuthology.orchestra.run.smithi186:> uname -r 2023-11-28T10:55:18.330 INFO:teuthology.orchestra.run.smithi156.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:18.330 INFO:teuthology.task.kernel:Running kernel on smithi156: 4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:18.330 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum install -y kernel 2023-11-28T10:55:18.331 INFO:teuthology.orchestra.run.smithi186.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:18.331 INFO:teuthology.task.kernel:Running kernel on smithi186: 4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:18.331 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum install -y kernel 2023-11-28T10:55:19.497 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:19.497 INFO:teuthology.orchestra.run.smithi186.stdout:Unable to read consumer identity 2023-11-28T10:55:19.536 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:19.536 INFO:teuthology.orchestra.run.smithi156.stdout:Unable to read consumer identity 2023-11-28T10:55:20.910 INFO:teuthology.orchestra.run.smithi186.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-11-28T10:55:20.951 INFO:teuthology.orchestra.run.smithi156.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-11-28T10:55:22.019 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:01 2023-11-28T10:55:22.235 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-11-28T10:55:25.594 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 184 kB/s | 24 kB 00:00 2023-11-28T10:55:25.786 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 455 kB/s | 24 kB 00:00 2023-11-28T10:55:26.288 INFO:teuthology.orchestra.run.smithi186.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-28T10:55:26.346 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-11-28T10:55:26.347 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2023-11-28T10:55:26.347 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-11-28T10:55:26.445 DEBUG:teuthology.orchestra.run.smithi186:> echo no | sudo yum reinstall kernel || true 2023-11-28T10:55:26.488 INFO:teuthology.orchestra.run.smithi156.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-28T10:55:26.535 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-11-28T10:55:26.536 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2023-11-28T10:55:26.536 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-11-28T10:55:26.672 DEBUG:teuthology.orchestra.run.smithi156:> echo no | sudo yum reinstall kernel || true 2023-11-28T10:55:26.778 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:26.778 INFO:teuthology.orchestra.run.smithi186.stdout:Unable to read consumer identity 2023-11-28T10:55:27.042 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 10:55:25 AM UTC. 2023-11-28T10:55:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:Unable to read consumer identity 2023-11-28T10:55:27.089 INFO:teuthology.orchestra.run.smithi186.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T10:55:27.130 INFO:teuthology.orchestra.run.smithi186.stderr:Error: No packages marked for reinstall. 2023-11-28T10:55:27.176 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum reinstall -y kernel || true 2023-11-28T10:55:27.305 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 10:55:25 AM UTC. 2023-11-28T10:55:27.346 INFO:teuthology.orchestra.run.smithi156.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T10:55:27.388 INFO:teuthology.orchestra.run.smithi156.stderr:Error: No packages marked for reinstall. 2023-11-28T10:55:27.434 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum reinstall -y kernel || true 2023-11-28T10:55:27.508 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:27.508 INFO:teuthology.orchestra.run.smithi186.stdout:Unable to read consumer identity 2023-11-28T10:55:27.762 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 10:55:25 AM UTC. 2023-11-28T10:55:27.764 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:27.764 INFO:teuthology.orchestra.run.smithi156.stdout:Unable to read consumer identity 2023-11-28T10:55:27.802 INFO:teuthology.orchestra.run.smithi186.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T10:55:27.833 INFO:teuthology.orchestra.run.smithi186.stderr:Error: No packages marked for reinstall. 2023-11-28T10:55:27.883 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T10:55:27.964 INFO:teuthology.orchestra.run.smithi186.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:27.964 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:27.964 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-28T10:55:27.964 DEBUG:teuthology.parallel:result is None 2023-11-28T10:55:28.022 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:00:03 ago on Tue 28 Nov 2023 10:55:25 AM UTC. 2023-11-28T10:55:28.065 INFO:teuthology.orchestra.run.smithi156.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T10:55:28.092 INFO:teuthology.orchestra.run.smithi156.stderr:Error: No packages marked for reinstall. 2023-11-28T10:55:28.138 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T10:55:28.218 INFO:teuthology.orchestra.run.smithi156.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:28.219 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:28.219 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-28T10:55:28.219 DEBUG:teuthology.parallel:result is None 2023-11-28T10:55:28.219 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T10:55:28.226 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T10:55:28.226 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T10:55:28.229 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T10:55:28.246 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T10:55:28.253 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T10:55:28.261 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T10:55:28.261 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T10:55:28.285 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T10:55:28.315 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T10:55:28.322 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T10:55:28.322 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T10:55:28.343 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T10:55:28.386 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T10:55:28.389 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T10:55:28.401 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T10:55:28.403 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T10:55:28.405 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T10:55:28.412 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T10:55:28.412 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T10:55:28.414 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T10:55:28.469 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T10:55:28.477 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T10:55:28.478 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T10:55:28.481 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T10:55:28.526 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T10:55:28.577 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T10:55:28.630 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T10:55:28.716 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T10:55:28.758 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T10:55:28.758 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T10:55:28.824 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T10:55:28.868 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T10:55:28.918 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T10:55:29.001 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T10:55:29.043 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T10:55:29.044 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T10:55:29.110 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2023-11-28T10:55:29.113 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2023-11-28T10:55:29.171 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T10:55:29.203 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T10:55:29.595 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T10:55:29.603 INFO:teuthology.task.internal:Starting timer... 2023-11-28T10:55:29.603 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T10:55:29.623 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T10:55:29.632 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-11-28T10:55:29.633 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2023-11-28T10:55:29.683 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2023-11-28T10:55:29.685 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2023-11-28T10:55:29.742 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2023-11-28T10:55:29.743 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T10:55:29.744 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2023-11-28T10:55:29.766 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2023-11-28T10:55:29.766 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2023-11-28T10:55:29.791 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2023-11-28T10:55:29.791 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi156.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2023-11-28T10:55:29.791 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T10:55:29.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:29.834 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T10:55:29.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:29.862 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T10:55:29.863 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2023-11-28T10:55:29.902 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2023-11-28T10:55:29.929 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T10:55:29.939 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-28T10:55:29.944 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-28T10:55:30.056 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T10:55:30.070 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T10:55:30.072 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi156.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T11:04:51.955 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi156.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2023-11-28T11:04:51.956 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-28T11:04:51.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:04:52.028 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-11-28T11:04:52.096 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-28T11:04:52.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-11-28T11:04:52.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:04:52.169 DEBUG:teuthology.orchestra.run.smithi186:> true 2023-11-28T11:04:52.238 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-11-28T11:04:52.239 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T11:04:52.256 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T11:04:52.257 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:04:52.257 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:04:52.265 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:04:52.265 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:04:52.292 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:04:52.306 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:04:52.312 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:04:52.327 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:04:52.363 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2023-11-28T11:04:52.375 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2023-11-28T11:04:52.389 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:04:52.399 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2023-11-28T11:04:52.403 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:04:52.411 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2023-11-28T11:04:52.426 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:04:52.440 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:04:52.470 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-11-28T11:04:52.474 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:04:52.474 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-11-28T11:04:52.474 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:04:52.474 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:04:52.474 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:04:52.474 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:04:52.510 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-11-28T11:04:52.514 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:04:52.514 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-11-28T11:04:52.514 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:04:52.515 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:04:52.515 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:04:52.515 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:04:52.515 INFO:teuthology.run_tasks:Running task install... 2023-11-28T11:04:52.527 DEBUG:teuthology.task.install:project ceph 2023-11-28T11:04:52.527 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}} 2023-11-28T11:04:52.527 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'} 2023-11-28T11:04:52.527 INFO:teuthology.task.install:Using flavor: default 2023-11-28T11:04:52.533 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T11:04:52.533 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T11:04:52.534 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:04:52.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:04:52.537 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:04:52.538 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:04:52.683 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:04:52.683 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:04:52.750 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:04:52.750 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:04:52.866 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:04:52.867 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:04:52.867 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:04:52.878 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:04:52.878 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:04:52.878 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:04:52.895 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:04:52.896 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:04:52.912 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:04:52.912 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:04:52.964 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:04:52.981 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:04:53.041 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:04:53.058 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:04:53.107 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2023-11-28T11:04:53.108 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2023-11-28T11:04:53.126 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2023-11-28T11:04:53.127 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2023-11-28T11:04:53.478 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-11-28T11:04:53.503 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-11-28T11:04:54.527 INFO:teuthology.orchestra.run.smithi156.stdout:44 files removed 2023-11-28T11:04:54.579 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:04:54.655 INFO:teuthology.orchestra.run.smithi186.stdout:44 files removed 2023-11-28T11:04:54.709 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:04:54.915 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-11-28T11:04:55.043 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-11-28T11:04:56.489 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 244 kB/s | 80 kB 00:00 2023-11-28T11:04:56.490 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 226 kB/s | 80 kB 00:00 2023-11-28T11:04:56.833 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-11-28T11:04:56.881 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 26 kB/s | 9.4 kB 00:00 2023-11-28T11:04:57.092 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 6.3 kB/s | 1.5 kB 00:00 2023-11-28T11:04:57.182 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 5.3 kB/s | 1.5 kB 00:00 2023-11-28T11:04:57.251 INFO:teuthology.orchestra.run.smithi186.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-11-28T11:04:57.324 INFO:teuthology.orchestra.run.smithi156.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-11-28T11:04:58.896 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 16 MB 00:01 2023-11-28T11:05:01.844 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 16 MB 00:04 2023-11-28T11:05:02.402 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 509 kB/s | 24 kB 00:00 2023-11-28T11:05:03.545 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-11-28T11:05:05.375 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 465 kB/s | 24 kB 00:00 2023-11-28T11:05:06.552 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 47 MB 00:01 2023-11-28T11:05:12.128 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 MB/s | 47 MB 00:01 2023-11-28T11:05:14.711 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-11-28T11:05:23.144 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:05:23.145 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:05:23.159 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T11:05:23.160 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T11:05:23.161 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T11:05:23.163 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:05:23.165 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:05:23.168 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:05:23.169 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:05:23.301 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-11-28T11:05:23.314 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-11-28T11:05:23.314 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-11-28T11:05:23.314 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-11-28T11:05:23.314 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-11-28T11:05:23.314 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T11:05:23.314 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T11:05:23.315 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:05:23.315 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:05:23.315 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:05:23.315 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:05:23.315 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:05:23.315 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:05:23.315 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:05:23.315 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:05:23.315 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:05:23.315 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:05:23.315 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:05:23.316 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:05:23.316 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:05:23.316 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:05:23.316 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:05:23.316 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T11:05:23.316 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T11:05:23.316 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T11:05:23.316 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:05:23.316 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:05:23.316 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:05:23.316 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T11:05:23.316 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T11:05:23.317 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T11:05:23.317 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T11:05:23.317 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T11:05:23.317 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T11:05:23.317 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:05:23.317 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:05:23.317 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:05:23.317 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:05:23.317 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:05:23.317 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:05:23.317 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:05:23.317 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T11:05:23.318 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2023-11-28T11:05:23.318 INFO:teuthology.orchestra.run.smithi186.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T11:05:23.318 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:05:23.318 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:05:23.318 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-11-28T11:05:23.318 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T11:05:23.318 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:05:23.318 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:05:23.318 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:05:23.318 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:05:23.318 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:05:23.319 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:05:23.319 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:05:23.319 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:05:23.319 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T11:05:23.319 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:05:23.319 INFO:teuthology.orchestra.run.smithi186.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T11:05:23.319 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T11:05:23.319 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T11:05:23.319 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:05:23.319 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T11:05:23.319 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T11:05:23.319 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:05:23.320 INFO:teuthology.orchestra.run.smithi186.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T11:05:23.320 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T11:05:23.320 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T11:05:23.320 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:05:23.320 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T11:05:23.320 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:05:23.320 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T11:05:23.320 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T11:05:23.320 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:05:23.320 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T11:05:23.320 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T11:05:23.321 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T11:05:23.321 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T11:05:23.321 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T11:05:23.321 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T11:05:23.321 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T11:05:23.321 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T11:05:23.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:05:23.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T11:05:23.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:05:23.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:05:23.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:05:23.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:05:23.322 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:05:23.322 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:05:23.322 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T11:05:23.322 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:05:23.322 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:05:23.322 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T11:05:23.322 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:05:23.322 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:05:23.322 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:05:23.322 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T11:05:23.322 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T11:05:23.322 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T11:05:23.323 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:05:23.323 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:05:23.323 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:05:23.323 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T11:05:23.323 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T11:05:23.323 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:05:23.323 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:05:23.323 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T11:05:23.323 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T11:05:23.323 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T11:05:23.323 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:05:23.323 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:05:23.324 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T11:05:23.324 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T11:05:23.324 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T11:05:23.324 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T11:05:23.324 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T11:05:23.324 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T11:05:23.324 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:05:23.324 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:05:23.324 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:05:23.324 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:05:23.324 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T11:05:23.325 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:05:23.325 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:05:23.325 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:05:23.325 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:05:23.325 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:05:23.325 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:05:23.325 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:05:23.325 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:05:23.325 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T11:05:23.325 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:05:23.325 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T11:05:23.325 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:05:23.326 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T11:05:23.326 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T11:05:23.326 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T11:05:23.326 INFO:teuthology.orchestra.run.smithi186.stdout:Enabling module streams: 2023-11-28T11:05:23.326 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql 10 2023-11-28T11:05:23.326 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:05:23.326 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-11-28T11:05:23.326 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-11-28T11:05:23.326 INFO:teuthology.orchestra.run.smithi186.stdout:Install 128 Packages 2023-11-28T11:05:23.326 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 3 Packages 2023-11-28T11:05:23.327 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:05:23.327 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 203 M 2023-11-28T11:05:23.327 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-11-28T11:05:23.612 INFO:teuthology.orchestra.run.smithi186.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 29 kB/s | 6.4 kB 00:00 2023-11-28T11:05:24.996 INFO:teuthology.orchestra.run.smithi186.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 611 kB/s | 845 kB 00:01 2023-11-28T11:05:25.221 INFO:teuthology.orchestra.run.smithi186.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 736 kB/s | 165 kB 00:00 2023-11-28T11:05:25.874 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:05:25.875 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:05:25.889 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T11:05:25.890 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T11:05:25.891 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T11:05:25.892 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:05:25.894 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:05:25.897 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:05:25.898 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:05:26.028 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-11-28T11:05:26.040 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-11-28T11:05:26.040 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-11-28T11:05:26.040 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-11-28T11:05:26.040 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2023-11-28T11:05:26.040 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T11:05:26.041 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T11:05:26.041 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:05:26.041 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:05:26.041 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:05:26.041 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:05:26.041 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:05:26.041 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:05:26.041 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:05:26.041 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:05:26.041 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:05:26.041 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:05:26.041 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T11:05:26.043 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T11:05:26.043 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T11:05:26.043 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T11:05:26.043 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T11:05:26.043 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:05:26.043 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:05:26.043 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:05:26.043 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:05:26.043 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:05:26.043 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:05:26.044 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:05:26.044 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T11:05:26.044 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2023-11-28T11:05:26.044 INFO:teuthology.orchestra.run.smithi156.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T11:05:26.044 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:05:26.044 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:05:26.044 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-11-28T11:05:26.044 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T11:05:26.044 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:05:26.045 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:05:26.045 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:05:26.045 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:05:26.045 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:05:26.045 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:05:26.045 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:05:26.045 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:05:26.045 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T11:05:26.045 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:05:26.045 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T11:05:26.045 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T11:05:26.046 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T11:05:26.046 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:05:26.046 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T11:05:26.046 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T11:05:26.046 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:05:26.046 INFO:teuthology.orchestra.run.smithi156.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T11:05:26.046 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T11:05:26.046 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T11:05:26.046 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:05:26.046 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T11:05:26.046 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:05:26.047 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T11:05:26.047 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T11:05:26.047 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:05:26.047 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T11:05:26.047 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T11:05:26.047 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T11:05:26.047 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T11:05:26.047 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T11:05:26.047 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T11:05:26.047 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T11:05:26.047 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T11:05:26.047 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:05:26.048 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T11:05:26.049 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T11:05:26.050 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:05:26.050 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:05:26.050 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T11:05:26.050 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T11:05:26.050 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T11:05:26.050 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T11:05:26.050 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T11:05:26.050 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T11:05:26.050 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:05:26.050 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:05:26.051 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:05:26.051 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:05:26.051 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T11:05:26.051 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:05:26.051 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:05:26.051 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:05:26.051 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:05:26.051 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:05:26.051 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:05:26.051 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:05:26.051 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:05:26.051 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T11:05:26.052 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:05:26.052 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T11:05:26.052 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:05:26.052 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T11:05:26.052 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T11:05:26.052 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T11:05:26.052 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-11-28T11:05:26.052 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql 10 2023-11-28T11:05:26.052 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:05:26.052 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-11-28T11:05:26.052 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-11-28T11:05:26.052 INFO:teuthology.orchestra.run.smithi156.stdout:Install 128 Packages 2023-11-28T11:05:26.053 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 3 Packages 2023-11-28T11:05:26.053 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:05:26.053 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 203 M 2023-11-28T11:05:26.053 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-11-28T11:05:26.163 INFO:teuthology.orchestra.run.smithi186.stdout:(4/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 1.9 MB/s | 5.4 MB 00:02 2023-11-28T11:05:26.255 INFO:teuthology.orchestra.run.smithi186.stdout:(5/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 2.2 MB/s | 2.3 MB 00:01 2023-11-28T11:05:26.439 INFO:teuthology.orchestra.run.smithi186.stdout:(6/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 5.6 MB/s | 1.5 MB 00:00 2023-11-28T11:05:26.552 INFO:teuthology.orchestra.run.smithi156.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 41 kB/s | 6.4 kB 00:00 2023-11-28T11:05:27.528 INFO:teuthology.orchestra.run.smithi156.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 867 kB/s | 845 kB 00:00 2023-11-28T11:05:27.620 INFO:teuthology.orchestra.run.smithi156.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 1.8 MB/s | 165 kB 00:00 2023-11-28T11:05:27.639 INFO:teuthology.orchestra.run.smithi186.stdout:(7/131): ceph-common-18.0.0-2697.g2f5f6028.el8. 5.1 MB/s | 22 MB 00:04 2023-11-28T11:05:27.706 INFO:teuthology.orchestra.run.smithi186.stdout:(8/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 3.2 MB/s | 4.6 MB 00:01 2023-11-28T11:05:27.823 INFO:teuthology.orchestra.run.smithi186.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 208 kB/s | 24 kB 00:00 2023-11-28T11:05:28.923 INFO:teuthology.orchestra.run.smithi186.stdout:(10/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 7.1 MB/s | 18 MB 00:02 2023-11-28T11:05:28.986 INFO:teuthology.orchestra.run.smithi156.stdout:(4/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 1.7 MB/s | 2.3 MB 00:01 2023-11-28T11:05:29.057 INFO:teuthology.orchestra.run.smithi186.stdout:(11/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el 9.2 MB/s | 13 MB 00:01 2023-11-28T11:05:29.090 INFO:teuthology.orchestra.run.smithi186.stdout:(12/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 1.5 MB/s | 257 kB 00:00 2023-11-28T11:05:29.128 INFO:teuthology.orchestra.run.smithi156.stdout:(5/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 2.0 MB/s | 5.4 MB 00:02 2023-11-28T11:05:29.141 INFO:teuthology.orchestra.run.smithi186.stdout:(13/131): libcephfs-devel-18.0.0-2697.g2f5f6028 365 kB/s | 30 kB 00:00 2023-11-28T11:05:29.241 INFO:teuthology.orchestra.run.smithi186.stdout:(14/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 5.0 MB/s | 767 kB 00:00 2023-11-28T11:05:29.266 INFO:teuthology.orchestra.run.smithi186.stdout:(15/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 1.1 MB/s | 137 kB 00:00 2023-11-28T11:05:30.233 INFO:teuthology.orchestra.run.smithi186.stdout:(16/131): librados-devel-18.0.0-2697.g2f5f6028. 140 kB/s | 139 kB 00:00 2023-11-28T11:05:30.316 INFO:teuthology.orchestra.run.smithi186.stdout:(17/131): libradosstriper1-18.0.0-2697.g2f5f602 451 kB/s | 474 kB 00:01 2023-11-28T11:05:30.417 INFO:teuthology.orchestra.run.smithi186.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 446 kB/s | 45 kB 00:00 2023-11-28T11:05:30.542 INFO:teuthology.orchestra.run.smithi186.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 938 kB/s | 117 kB 00:00 2023-11-28T11:05:30.628 INFO:teuthology.orchestra.run.smithi156.stdout:(6/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 963 kB/s | 1.5 MB 00:01 2023-11-28T11:05:31.667 INFO:teuthology.orchestra.run.smithi186.stdout:(20/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 4.5 MB/s | 6.5 MB 00:01 2023-11-28T11:05:32.469 INFO:teuthology.orchestra.run.smithi186.stdout:(21/131): python3-rados-18.0.0-2697.g2f5f6028.e 464 kB/s | 371 kB 00:00 2023-11-28T11:05:32.493 INFO:teuthology.orchestra.run.smithi186.stdout:(22/131): python3-cephfs-18.0.0-2697.g2f5f6028. 97 kB/s | 190 kB 00:01 2023-11-28T11:05:32.593 INFO:teuthology.orchestra.run.smithi186.stdout:(23/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 2.8 MB/s | 364 kB 00:00 2023-11-28T11:05:32.618 INFO:teuthology.orchestra.run.smithi186.stdout:(24/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 851 kB/s | 107 kB 00:00 2023-11-28T11:05:34.177 INFO:teuthology.orchestra.run.smithi186.stdout:(25/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 60 kB/s | 94 kB 00:01 2023-11-28T11:05:34.427 INFO:teuthology.orchestra.run.smithi186.stdout:(26/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 721 kB/s | 180 kB 00:00 2023-11-28T11:05:34.487 INFO:teuthology.orchestra.run.smithi156.stdout:(7/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 878 kB/s | 4.6 MB 00:05 2023-11-28T11:05:34.836 INFO:teuthology.orchestra.run.smithi186.stdout:(27/131): ceph-test-18.0.0-2697.g2f5f6028.el8.x 6.6 MB/s | 47 MB 00:07 2023-11-28T11:05:34.853 INFO:teuthology.orchestra.run.smithi186.stdout:(28/131): ceph-grafana-dashboards-18.0.0-2697.g 54 kB/s | 23 kB 00:00 2023-11-28T11:05:34.928 INFO:teuthology.orchestra.run.smithi186.stdout:(29/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 1.5 MB/s | 3.4 MB 00:02 2023-11-28T11:05:34.945 INFO:teuthology.orchestra.run.smithi186.stdout:(30/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.1 MB/s | 123 kB 00:00 2023-11-28T11:05:35.012 INFO:teuthology.orchestra.run.smithi156.stdout:(8/131): ceph-common-18.0.0-2697.g2f5f6028.el8. 2.5 MB/s | 22 MB 00:08 2023-11-28T11:05:35.153 INFO:teuthology.orchestra.run.smithi186.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2697.g2f 1.1 MB/s | 236 kB 00:00 2023-11-28T11:05:35.162 INFO:teuthology.orchestra.run.smithi156.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 162 kB/s | 24 kB 00:00 2023-11-28T11:05:35.187 INFO:teuthology.orchestra.run.smithi186.stdout:(32/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 5.2 MB/s | 1.7 MB 00:00 2023-11-28T11:05:35.229 INFO:teuthology.orchestra.run.smithi186.stdout:(33/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 631 kB/s | 47 kB 00:00 2023-11-28T11:05:35.262 INFO:teuthology.orchestra.run.smithi186.stdout:(34/131): ceph-prometheus-alerts-18.0.0-2697.g2 194 kB/s | 15 kB 00:00 2023-11-28T11:05:35.346 INFO:teuthology.orchestra.run.smithi186.stdout:(35/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 2.2 MB/s | 263 kB 00:00 2023-11-28T11:05:35.396 INFO:teuthology.orchestra.run.smithi186.stdout:(36/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 619 kB/s | 83 kB 00:00 2023-11-28T11:05:35.438 INFO:teuthology.orchestra.run.smithi186.stdout:(37/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 196 kB/s | 18 kB 00:00 2023-11-28T11:05:35.589 INFO:teuthology.orchestra.run.smithi186.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-11-28T11:05:35.756 INFO:teuthology.orchestra.run.smithi186.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 185 kB/s | 59 kB 00:00 2023-11-28T11:05:35.781 INFO:teuthology.orchestra.run.smithi186.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 797 kB/s | 306 kB 00:00 2023-11-28T11:05:35.856 INFO:teuthology.orchestra.run.smithi186.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 581 kB/s | 44 kB 00:00 2023-11-28T11:05:35.882 INFO:teuthology.orchestra.run.smithi186.stdout:(42/131): libunwind-1.3.1-3.el8.x86_64.rpm 257 kB/s | 75 kB 00:00 2023-11-28T11:05:35.973 INFO:teuthology.orchestra.run.smithi186.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 361 kB/s | 33 kB 00:00 2023-11-28T11:05:35.999 INFO:teuthology.orchestra.run.smithi186.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-11-28T11:05:36.024 INFO:teuthology.orchestra.run.smithi186.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-11-28T11:05:36.058 INFO:teuthology.orchestra.run.smithi186.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 194 kB/s | 16 kB 00:00 2023-11-28T11:05:36.141 INFO:teuthology.orchestra.run.smithi186.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2023-11-28T11:05:36.175 INFO:teuthology.orchestra.run.smithi186.stdout:(48/131): python3-cheroot-8.5.2-1.el8.noarch.rp 982 kB/s | 173 kB 00:00 2023-11-28T11:05:36.200 INFO:teuthology.orchestra.run.smithi186.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 571 kB/s | 81 kB 00:00 2023-11-28T11:05:36.225 INFO:teuthology.orchestra.run.smithi186.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 132 kB/s | 11 kB 00:00 2023-11-28T11:05:36.259 INFO:teuthology.orchestra.run.smithi186.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2023-11-28T11:05:36.309 INFO:teuthology.orchestra.run.smithi186.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 583 kB/s | 49 kB 00:00 2023-11-28T11:05:36.351 INFO:teuthology.orchestra.run.smithi186.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 638 kB/s | 59 kB 00:00 2023-11-28T11:05:36.393 INFO:teuthology.orchestra.run.smithi186.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 723 kB/s | 60 kB 00:00 2023-11-28T11:05:36.460 INFO:teuthology.orchestra.run.smithi186.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 4.5 MB/s | 1.2 MB 00:00 2023-11-28T11:05:36.485 INFO:teuthology.orchestra.run.smithi186.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 169 kB/s | 16 kB 00:00 2023-11-28T11:05:36.510 INFO:teuthology.orchestra.run.smithi186.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2023-11-28T11:05:36.544 INFO:teuthology.orchestra.run.smithi186.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 401 kB/s | 34 kB 00:00 2023-11-28T11:05:36.578 INFO:teuthology.orchestra.run.smithi186.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-11-28T11:05:36.595 INFO:teuthology.orchestra.run.smithi186.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 765 kB/s | 64 kB 00:00 2023-11-28T11:05:36.628 INFO:teuthology.orchestra.run.smithi186.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 223 kB/s | 19 kB 00:00 2023-11-28T11:05:36.653 INFO:teuthology.orchestra.run.smithi186.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2023-11-28T11:05:36.679 INFO:teuthology.orchestra.run.smithi186.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 349 kB/s | 29 kB 00:00 2023-11-28T11:05:36.704 INFO:teuthology.orchestra.run.smithi186.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2023-11-28T11:05:36.746 INFO:teuthology.orchestra.run.smithi186.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-11-28T11:05:36.771 INFO:teuthology.orchestra.run.smithi186.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-11-28T11:05:36.796 INFO:teuthology.orchestra.run.smithi186.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 665 kB/s | 61 kB 00:00 2023-11-28T11:05:36.822 INFO:teuthology.orchestra.run.smithi186.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-11-28T11:05:36.839 INFO:teuthology.orchestra.run.smithi186.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 340 kB/s | 23 kB 00:00 2023-11-28T11:05:36.864 INFO:teuthology.orchestra.run.smithi186.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-11-28T11:05:36.956 INFO:teuthology.orchestra.run.smithi186.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2023-11-28T11:05:37.240 INFO:teuthology.orchestra.run.smithi186.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 117 kB/s | 47 kB 00:00 2023-11-28T11:05:37.257 INFO:teuthology.orchestra.run.smithi186.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 142 kB/s | 43 kB 00:00 2023-11-28T11:05:37.290 INFO:teuthology.orchestra.run.smithi186.stdout:(74/131): libselinux-devel-2.9-5.el8.x86_64.rpm 476 kB/s | 200 kB 00:00 2023-11-28T11:05:37.457 INFO:teuthology.orchestra.run.smithi186.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 240 kB/s | 48 kB 00:00 2023-11-28T11:05:37.499 INFO:teuthology.orchestra.run.smithi186.stdout:(76/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 851 kB/s | 220 kB 00:00 2023-11-28T11:05:37.524 INFO:teuthology.orchestra.run.smithi186.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 372 kB/s | 87 kB 00:00 2023-11-28T11:05:37.625 INFO:teuthology.orchestra.run.smithi186.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 255 kB/s | 43 kB 00:00 2023-11-28T11:05:37.658 INFO:teuthology.orchestra.run.smithi186.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 687 kB/s | 109 kB 00:00 2023-11-28T11:05:37.696 INFO:teuthology.orchestra.run.smithi156.stdout:(10/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el 4.1 MB/s | 13 MB 00:03 2023-11-28T11:05:37.758 INFO:teuthology.orchestra.run.smithi186.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 980 kB/s | 229 kB 00:00 2023-11-28T11:05:37.837 INFO:teuthology.orchestra.run.smithi156.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 1.8 MB/s | 257 kB 00:00 2023-11-28T11:05:37.884 INFO:teuthology.orchestra.run.smithi186.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.1 MB/s | 560 kB 00:00 2023-11-28T11:05:37.909 INFO:teuthology.orchestra.run.smithi186.stdout:(82/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 75 kB/s | 19 kB 00:00 2023-11-28T11:05:37.938 INFO:teuthology.orchestra.run.smithi156.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 305 kB/s | 30 kB 00:00 2023-11-28T11:05:37.959 INFO:teuthology.orchestra.run.smithi186.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 772 kB/s | 155 kB 00:00 2023-11-28T11:05:38.051 INFO:teuthology.orchestra.run.smithi186.stdout:(84/131): python3-cryptography-3.2.1-5.el8.x86_ 3.3 MB/s | 559 kB 00:00 2023-11-28T11:05:38.102 INFO:teuthology.orchestra.run.smithi186.stdout:(85/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 93 kB/s | 18 kB 00:00 2023-11-28T11:05:38.135 INFO:teuthology.orchestra.run.smithi186.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-11-28T11:05:38.310 INFO:teuthology.orchestra.run.smithi186.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 1.7 MB/s | 605 kB 00:00 2023-11-28T11:05:38.344 INFO:teuthology.orchestra.run.smithi186.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 347 kB/s | 72 kB 00:00 2023-11-28T11:05:38.427 INFO:teuthology.orchestra.run.smithi186.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 708 kB/s | 83 kB 00:00 2023-11-28T11:05:38.513 INFO:teuthology.orchestra.run.smithi156.stdout:(13/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 1.3 MB/s | 767 kB 00:00 2023-11-28T11:05:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:(90/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 5.6 MB/s | 2.3 MB 00:00 2023-11-28T11:05:38.605 INFO:teuthology.orchestra.run.smithi156.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 1.5 MB/s | 137 kB 00:00 2023-11-28T11:05:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 211 kB/s | 60 kB 00:00 2023-11-28T11:05:38.697 INFO:teuthology.orchestra.run.smithi156.stdout:(15/131): librados-devel-18.0.0-2697.g2f5f6028. 1.5 MB/s | 139 kB 00:00 2023-11-28T11:05:38.745 INFO:teuthology.orchestra.run.smithi186.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 309 kB/s | 98 kB 00:00 2023-11-28T11:05:38.829 INFO:teuthology.orchestra.run.smithi186.stdout:(93/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 187 kB/s | 58 kB 00:00 2023-11-28T11:05:38.854 INFO:teuthology.orchestra.run.smithi186.stdout:(94/131): libcom_err-devel-1.45.6-4.el8.x86_64. 171 kB/s | 39 kB 00:00 2023-11-28T11:05:38.895 INFO:teuthology.orchestra.run.smithi186.stdout:(95/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.2 MB/s | 644 kB 00:00 2023-11-28T11:05:38.980 INFO:teuthology.orchestra.run.smithi156.stdout:(16/131): libradosstriper1-18.0.0-2697.g2f5f602 1.6 MB/s | 474 kB 00:00 2023-11-28T11:05:39.060 INFO:teuthology.orchestra.run.smithi186.stdout:(96/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 759 kB/s | 171 kB 00:00 2023-11-28T11:05:39.071 INFO:teuthology.orchestra.run.smithi186.stdout:(97/131): NetworkManager-initscripts-updown-1.3 634 kB/s | 138 kB 00:00 2023-11-28T11:05:39.138 INFO:teuthology.orchestra.run.smithi186.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-11-28T11:05:39.438 INFO:teuthology.orchestra.run.smithi186.stdout:(99/131): python3-scipy-1.0.0-21.module+el8.5.0 25 MB/s | 14 MB 00:00 2023-11-28T11:05:39.464 INFO:teuthology.orchestra.run.smithi186.stdout:(100/131): python3-pytz-2017.2-9.el8.noarch.rpm 138 kB/s | 54 kB 00:00 2023-11-28T11:05:39.564 INFO:teuthology.orchestra.run.smithi186.stdout:(101/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 608 kB/s | 259 kB 00:00 2023-11-28T11:05:39.656 INFO:teuthology.orchestra.run.smithi186.stdout:(102/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2023-11-28T11:05:39.689 INFO:teuthology.orchestra.run.smithi186.stdout:(103/131): python3-markupsafe-0.23-19.el8.x86_6 172 kB/s | 39 kB 00:00 2023-11-28T11:05:39.840 INFO:teuthology.orchestra.run.smithi186.stdout:(104/131): bison-3.0.4-10.el8.x86_64.rpm 2.4 MB/s | 688 kB 00:00 2023-11-28T11:05:39.899 INFO:teuthology.orchestra.run.smithi186.stdout:(105/131): python3-jinja2-2.10.1-3.el8.noarch.r 2.2 MB/s | 538 kB 00:00 2023-11-28T11:05:39.924 INFO:teuthology.orchestra.run.smithi186.stdout:(106/131): python3-mako-1.0.6-13.el8.noarch.rpm 671 kB/s | 157 kB 00:00 2023-11-28T11:05:40.000 INFO:teuthology.orchestra.run.smithi186.stdout:(107/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2023-11-28T11:05:40.033 INFO:teuthology.orchestra.run.smithi186.stdout:(108/131): automake-1.16.1-7.el8.noarch.rpm 5.2 MB/s | 713 kB 00:00 2023-11-28T11:05:40.149 INFO:teuthology.orchestra.run.smithi186.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-11-28T11:05:40.167 INFO:teuthology.orchestra.run.smithi186.stdout:(110/131): python3-jsonpatch-1.21-2.el8.noarch. 159 kB/s | 27 kB 00:00 2023-11-28T11:05:40.267 INFO:teuthology.orchestra.run.smithi186.stdout:(111/131): flex-2.6.1-9.el8.x86_64.rpm 1.3 MB/s | 320 kB 00:00 2023-11-28T11:05:40.322 INFO:teuthology.orchestra.run.smithi156.stdout:(17/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 1.8 MB/s | 18 MB 00:09 2023-11-28T11:05:40.333 INFO:teuthology.orchestra.run.smithi186.stdout:(112/131): python3-prettytable-0.7.2-14.el8.noa 239 kB/s | 44 kB 00:00 2023-11-28T11:05:40.406 INFO:teuthology.orchestra.run.smithi156.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 538 kB/s | 45 kB 00:00 2023-11-28T11:05:40.481 INFO:teuthology.orchestra.run.smithi156.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 1.5 MB/s | 117 kB 00:00 2023-11-28T11:05:40.484 INFO:teuthology.orchestra.run.smithi186.stdout:(113/131): python3-werkzeug-0.12.2-4.el8.noarch 2.1 MB/s | 457 kB 00:00 2023-11-28T11:05:40.567 INFO:teuthology.orchestra.run.smithi186.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 9.3 MB/s | 3.7 MB 00:00 2023-11-28T11:05:40.592 INFO:teuthology.orchestra.run.smithi186.stdout:(115/131): python3-numpy-f2py-1.14.3-10.el8.x86 870 kB/s | 225 kB 00:00 2023-11-28T11:05:40.648 INFO:teuthology.orchestra.run.smithi156.stdout:(20/131): python3-cephfs-18.0.0-2697.g2f5f6028. 1.1 MB/s | 190 kB 00:00 2023-11-28T11:05:40.693 INFO:teuthology.orchestra.run.smithi186.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 307 kB/s | 64 kB 00:00 2023-11-28T11:05:40.890 INFO:teuthology.orchestra.run.smithi156.stdout:(21/131): python3-rados-18.0.0-2697.g2f5f6028.e 1.5 MB/s | 371 kB 00:00 2023-11-28T11:05:40.893 INFO:teuthology.orchestra.run.smithi186.stdout:(117/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 517 kB/s | 103 kB 00:00 2023-11-28T11:05:41.001 INFO:teuthology.orchestra.run.smithi186.stdout:(118/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 291 kB/s | 126 kB 00:00 2023-11-28T11:05:41.027 INFO:teuthology.orchestra.run.smithi186.stdout:(119/131): python3-jsonpointer-1.10-11.el8.noar 46 kB/s | 20 kB 00:00 2023-11-28T11:05:41.060 INFO:teuthology.orchestra.run.smithi186.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 4.1 MB/s | 709 kB 00:00 2023-11-28T11:05:41.107 INFO:teuthology.orchestra.run.smithi156.stdout:(22/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 1.6 MB/s | 364 kB 00:00 2023-11-28T11:05:41.183 INFO:teuthology.orchestra.run.smithi156.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.4 MB/s | 107 kB 00:00 2023-11-28T11:05:41.261 INFO:teuthology.orchestra.run.smithi186.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 847 kB/s | 198 kB 00:00 2023-11-28T11:05:41.283 INFO:teuthology.orchestra.run.smithi156.stdout:(24/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 947 kB/s | 94 kB 00:00 2023-11-28T11:05:41.386 INFO:teuthology.orchestra.run.smithi186.stdout:(122/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-11-28T11:05:41.470 INFO:teuthology.orchestra.run.smithi186.stdout:(123/131): openblas-threads-0.3.15-3.el8.x86_64 10 MB/s | 4.9 MB 00:00 2023-11-28T11:05:41.497 INFO:teuthology.orchestra.run.smithi186.stdout:(124/131): jq-1.6-3.el8.x86_64.rpm 865 kB/s | 202 kB 00:00 2023-11-28T11:05:41.578 INFO:teuthology.orchestra.run.smithi186.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.8 MB/s | 354 kB 00:00 2023-11-28T11:05:41.742 INFO:teuthology.orchestra.run.smithi156.stdout:(25/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 2.4 MB/s | 6.5 MB 00:02 2023-11-28T11:05:41.820 INFO:teuthology.orchestra.run.smithi186.stdout:(126/131): postgresql-contrib-10.21-2.module+el 2.4 MB/s | 808 kB 00:00 2023-11-28T11:05:41.904 INFO:teuthology.orchestra.run.smithi186.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 4.6 MB/s | 1.5 MB 00:00 2023-11-28T11:05:41.917 INFO:teuthology.orchestra.run.smithi156.stdout:(26/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 1.0 MB/s | 180 kB 00:00 2023-11-28T11:05:42.029 INFO:teuthology.orchestra.run.smithi186.stdout:(128/131): postgresql-server-10.21-2.module+el8 9.1 MB/s | 5.1 MB 00:00 2023-11-28T11:05:42.171 INFO:teuthology.orchestra.run.smithi186.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.5 MB/s | 798 kB 00:00 2023-11-28T11:05:42.375 INFO:teuthology.orchestra.run.smithi156.stdout:(27/131): ceph-grafana-dashboards-18.0.0-2697.g 50 kB/s | 23 kB 00:00 2023-11-28T11:05:42.630 INFO:teuthology.orchestra.run.smithi186.stdout:(130/131): librados2-18.0.0-2697.g2f5f6028.el8. 4.5 MB/s | 3.7 MB 00:00 2023-11-28T11:05:42.861 INFO:teuthology.orchestra.run.smithi156.stdout:(28/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 253 kB/s | 123 kB 00:00 2023-11-28T11:05:43.122 INFO:teuthology.orchestra.run.smithi186.stdout:(131/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 2.9 MB/s | 3.6 MB 00:01 2023-11-28T11:05:43.125 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:05:43.125 INFO:teuthology.orchestra.run.smithi186.stdout:Total 10 MB/s | 203 MB 00:19 2023-11-28T11:05:43.484 INFO:teuthology.orchestra.run.smithi156.stdout:(29/131): ceph-test-18.0.0-2697.g2f5f6028.el8.x 5.6 MB/s | 47 MB 00:08 2023-11-28T11:05:43.684 INFO:teuthology.orchestra.run.smithi156.stdout:(30/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 1.4 MB/s | 3.4 MB 00:02 2023-11-28T11:05:43.843 INFO:teuthology.orchestra.run.smithi156.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2697.g2f 1.5 MB/s | 236 kB 00:00 2023-11-28T11:05:43.935 INFO:teuthology.orchestra.run.smithi156.stdout:(32/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 518 kB/s | 47 kB 00:00 2023-11-28T11:05:43.963 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-11-28T11:05:44.035 INFO:teuthology.orchestra.run.smithi156.stdout:(33/131): ceph-prometheus-alerts-18.0.0-2697.g2 145 kB/s | 15 kB 00:00 2023-11-28T11:05:44.085 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-11-28T11:05:44.085 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-11-28T11:05:44.252 INFO:teuthology.orchestra.run.smithi156.stdout:(34/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 1.2 MB/s | 263 kB 00:00 2023-11-28T11:05:44.344 INFO:teuthology.orchestra.run.smithi156.stdout:(35/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 901 kB/s | 83 kB 00:00 2023-11-28T11:05:44.386 INFO:teuthology.orchestra.run.smithi156.stdout:(36/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 1.1 MB/s | 1.7 MB 00:01 2023-11-28T11:05:44.411 INFO:teuthology.orchestra.run.smithi156.stdout:(37/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 270 kB/s | 18 kB 00:00 2023-11-28T11:05:44.604 INFO:teuthology.orchestra.run.smithi156.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 305 kB/s | 59 kB 00:00 2023-11-28T11:05:44.687 INFO:teuthology.orchestra.run.smithi156.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 911 kB/s | 75 kB 00:00 2023-11-28T11:05:44.712 INFO:teuthology.orchestra.run.smithi156.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 940 kB/s | 306 kB 00:00 2023-11-28T11:05:44.771 INFO:teuthology.orchestra.run.smithi156.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 748 kB/s | 44 kB 00:00 2023-11-28T11:05:44.796 INFO:teuthology.orchestra.run.smithi156.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2023-11-28T11:05:44.846 INFO:teuthology.orchestra.run.smithi156.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 661 kB/s | 33 kB 00:00 2023-11-28T11:05:44.872 INFO:teuthology.orchestra.run.smithi156.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-11-28T11:05:44.897 INFO:teuthology.orchestra.run.smithi156.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 324 kB/s | 16 kB 00:00 2023-11-28T11:05:44.941 INFO:teuthology.orchestra.run.smithi156.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2023-11-28T11:05:44.973 INFO:teuthology.orchestra.run.smithi156.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 5.0 MB/s | 384 kB 00:00 2023-11-28T11:05:44.998 INFO:teuthology.orchestra.run.smithi156.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-11-28T11:05:45.023 INFO:teuthology.orchestra.run.smithi156.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 220 kB/s | 11 kB 00:00 2023-11-28T11:05:45.049 INFO:teuthology.orchestra.run.smithi156.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 348 kB/s | 18 kB 00:00 2023-11-28T11:05:45.107 INFO:teuthology.orchestra.run.smithi156.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 839 kB/s | 49 kB 00:00 2023-11-28T11:05:45.141 INFO:teuthology.orchestra.run.smithi156.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 10 MB/s | 1.2 MB 00:00 2023-11-28T11:05:45.191 INFO:teuthology.orchestra.run.smithi156.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 701 kB/s | 59 kB 00:00 2023-11-28T11:05:45.194 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-11-28T11:05:45.199 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-11-28T11:05:45.217 INFO:teuthology.orchestra.run.smithi156.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 800 kB/s | 60 kB 00:00 2023-11-28T11:05:45.258 INFO:teuthology.orchestra.run.smithi156.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-11-28T11:05:45.284 INFO:teuthology.orchestra.run.smithi156.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2023-11-28T11:05:45.309 INFO:teuthology.orchestra.run.smithi156.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 670 kB/s | 34 kB 00:00 2023-11-28T11:05:45.352 INFO:teuthology.orchestra.run.smithi156.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-11-28T11:05:45.368 INFO:teuthology.orchestra.run.smithi156.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-11-28T11:05:45.402 INFO:teuthology.orchestra.run.smithi156.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 374 kB/s | 19 kB 00:00 2023-11-28T11:05:45.427 INFO:teuthology.orchestra.run.smithi156.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 409 kB/s | 24 kB 00:00 2023-11-28T11:05:45.461 INFO:teuthology.orchestra.run.smithi156.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 499 kB/s | 29 kB 00:00 2023-11-28T11:05:45.494 INFO:teuthology.orchestra.run.smithi156.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 410 kB/s | 27 kB 00:00 2023-11-28T11:05:45.528 INFO:teuthology.orchestra.run.smithi156.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2023-11-28T11:05:45.587 INFO:teuthology.orchestra.run.smithi156.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-11-28T11:05:45.612 INFO:teuthology.orchestra.run.smithi156.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 733 kB/s | 61 kB 00:00 2023-11-28T11:05:45.645 INFO:teuthology.orchestra.run.smithi156.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-11-28T11:05:45.671 INFO:teuthology.orchestra.run.smithi156.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 392 kB/s | 23 kB 00:00 2023-11-28T11:05:45.696 INFO:teuthology.orchestra.run.smithi156.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-28T11:05:45.772 INFO:teuthology.orchestra.run.smithi156.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2023-11-28T11:05:45.905 INFO:teuthology.orchestra.run.smithi156.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 863 kB/s | 110 kB 00:00 2023-11-28T11:05:46.030 INFO:teuthology.orchestra.run.smithi156.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 434 kB/s | 54 kB 00:00 2023-11-28T11:05:46.189 INFO:teuthology.orchestra.run.smithi156.stdout:(73/131): ceph-mgr-diskprediction-local-18.0.0- 2.7 MB/s | 7.4 MB 00:02 2023-11-28T11:05:46.222 INFO:teuthology.orchestra.run.smithi156.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2023-11-28T11:05:46.415 INFO:teuthology.orchestra.run.smithi156.stdout:(75/131): python3-scipy-1.0.0-21.module+el8.5.0 19 MB/s | 14 MB 00:00 2023-11-28T11:05:46.431 INFO:teuthology.orchestra.run.smithi156.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 191 kB/s | 39 kB 00:00 2023-11-28T11:05:46.457 INFO:teuthology.orchestra.run.smithi156.stdout:(77/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.2 MB/s | 323 kB 00:00 2023-11-28T11:05:46.490 INFO:teuthology.orchestra.run.smithi156.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 9.0 MB/s | 688 kB 00:00 2023-11-28T11:05:46.574 INFO:teuthology.orchestra.run.smithi156.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2023-11-28T11:05:46.599 INFO:teuthology.orchestra.run.smithi156.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2023-11-28T11:05:46.624 INFO:teuthology.orchestra.run.smithi156.stdout:(81/131): python3-jinja2-2.10.1-3.el8.noarch.rp 2.7 MB/s | 538 kB 00:00 2023-11-28T11:05:46.658 INFO:teuthology.orchestra.run.smithi156.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 8.3 MB/s | 713 kB 00:00 2023-11-28T11:05:46.683 INFO:teuthology.orchestra.run.smithi156.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 453 kB/s | 27 kB 00:00 2023-11-28T11:05:46.717 INFO:teuthology.orchestra.run.smithi156.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2023-11-28T11:05:46.742 INFO:teuthology.orchestra.run.smithi156.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 752 kB/s | 44 kB 00:00 2023-11-28T11:05:46.792 INFO:teuthology.orchestra.run.smithi156.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-11-28T11:05:46.885 INFO:teuthology.orchestra.run.smithi156.stdout:(87/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-11-28T11:05:46.910 INFO:teuthology.orchestra.run.smithi156.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.9 MB/s | 225 kB 00:00 2023-11-28T11:05:46.993 INFO:teuthology.orchestra.run.smithi156.stdout:(89/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-11-28T11:05:47.018 INFO:teuthology.orchestra.run.smithi156.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 479 kB/s | 64 kB 00:00 2023-11-28T11:05:47.044 INFO:teuthology.orchestra.run.smithi156.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 399 kB/s | 20 kB 00:00 2023-11-28T11:05:47.069 INFO:teuthology.orchestra.run.smithi156.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 793 kB/s | 126 kB 00:00 2023-11-28T11:05:47.095 INFO:teuthology.orchestra.run.smithi156.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.3 MB/s | 103 kB 00:00 2023-11-28T11:05:47.128 INFO:teuthology.orchestra.run.smithi156.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 8.2 MB/s | 709 kB 00:00 2023-11-28T11:05:47.162 INFO:teuthology.orchestra.run.smithi156.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-11-28T11:05:47.221 INFO:teuthology.orchestra.run.smithi156.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-11-28T11:05:47.271 INFO:teuthology.orchestra.run.smithi156.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-11-28T11:05:47.338 INFO:teuthology.orchestra.run.smithi156.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 18 MB/s | 4.9 MB 00:00 2023-11-28T11:05:47.396 INFO:teuthology.orchestra.run.smithi156.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-11-28T11:05:47.438 INFO:teuthology.orchestra.run.smithi156.stdout:(100/131): postgresql-contrib-10.21-2.module+el 7.9 MB/s | 808 kB 00:00 2023-11-28T11:05:47.498 INFO:teuthology.orchestra.run.smithi156.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-11-28T11:05:47.546 INFO:teuthology.orchestra.run.smithi156.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 436 kB/s | 47 kB 00:00 2023-11-28T11:05:47.581 INFO:teuthology.orchestra.run.smithi156.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.3 MB/s | 200 kB 00:00 2023-11-28T11:05:47.606 INFO:teuthology.orchestra.run.smithi156.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 711 kB/s | 43 kB 00:00 2023-11-28T11:05:47.631 INFO:teuthology.orchestra.run.smithi156.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-28T11:05:47.657 INFO:teuthology.orchestra.run.smithi156.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 957 kB/s | 48 kB 00:00 2023-11-28T11:05:47.682 INFO:teuthology.orchestra.run.smithi156.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-28T11:05:47.707 INFO:teuthology.orchestra.run.smithi156.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 847 kB/s | 43 kB 00:00 2023-11-28T11:05:47.733 INFO:teuthology.orchestra.run.smithi156.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-11-28T11:05:47.758 INFO:teuthology.orchestra.run.smithi156.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-11-28T11:05:47.792 INFO:teuthology.orchestra.run.smithi156.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-11-28T11:05:47.817 INFO:teuthology.orchestra.run.smithi156.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 319 kB/s | 19 kB 00:00 2023-11-28T11:05:47.842 INFO:teuthology.orchestra.run.smithi156.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-11-28T11:05:47.876 INFO:teuthology.orchestra.run.smithi156.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-11-28T11:05:47.901 INFO:teuthology.orchestra.run.smithi156.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 305 kB/s | 18 kB 00:00 2023-11-28T11:05:47.968 INFO:teuthology.orchestra.run.smithi156.stdout:(116/131): postgresql-server-10.21-2.module+el8 7.3 MB/s | 5.1 MB 00:00 2023-11-28T11:05:48.002 INFO:teuthology.orchestra.run.smithi156.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-11-28T11:05:48.027 INFO:teuthology.orchestra.run.smithi156.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.8 MB/s | 238 kB 00:00 2023-11-28T11:05:48.044 INFO:teuthology.orchestra.run.smithi156.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-11-28T11:05:48.078 INFO:teuthology.orchestra.run.smithi156.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-28T11:05:48.124 INFO:teuthology.orchestra.run.smithi156.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 746 kB/s | 60 kB 00:00 2023-11-28T11:05:48.220 INFO:teuthology.orchestra.run.smithi156.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 692 kB/s | 98 kB 00:00 2023-11-28T11:05:48.270 INFO:teuthology.orchestra.run.smithi156.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 396 kB/s | 58 kB 00:00 2023-11-28T11:05:48.287 INFO:teuthology.orchestra.run.smithi156.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 575 kB/s | 39 kB 00:00 2023-11-28T11:05:48.321 INFO:teuthology.orchestra.run.smithi156.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 13 MB/s | 644 kB 00:00 2023-11-28T11:05:48.338 INFO:teuthology.orchestra.run.smithi156.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 3.3 MB/s | 171 kB 00:00 2023-11-28T11:05:48.363 INFO:teuthology.orchestra.run.smithi156.stdout:(127/131): NetworkManager-initscripts-updown-1. 3.2 MB/s | 138 kB 00:00 2023-11-28T11:05:48.414 INFO:teuthology.orchestra.run.smithi156.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 5.2 MB/s | 2.3 MB 00:00 2023-11-28T11:05:48.465 INFO:teuthology.orchestra.run.smithi156.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-28T11:05:50.188 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-11-28T11:05:50.343 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T11:05:50.518 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T11:05:50.670 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T11:05:50.793 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T11:05:50.905 INFO:teuthology.orchestra.run.smithi156.stdout:(130/131): librados2-18.0.0-2697.g2f5f6028.el8. 1.4 MB/s | 3.7 MB 00:02 2023-11-28T11:05:51.004 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T11:05:51.055 INFO:teuthology.orchestra.run.smithi156.stdout:(131/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 1.3 MB/s | 3.6 MB 00:02 2023-11-28T11:05:51.058 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:05:51.059 INFO:teuthology.orchestra.run.smithi156.stdout:Total 8.1 MB/s | 203 MB 00:25 2023-11-28T11:05:51.408 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T11:05:51.620 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T11:05:51.882 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-11-28T11:05:52.001 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-11-28T11:05:52.002 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-11-28T11:05:52.012 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T11:05:52.228 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:05:52.358 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:05:52.545 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:05:52.642 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:05:52.825 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:05:53.002 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-28T11:05:53.115 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-11-28T11:05:53.119 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-11-28T11:05:53.223 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T11:05:53.431 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T11:05:53.919 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T11:05:54.165 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T11:05:54.336 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:05:54.424 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:05:54.594 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:05:54.727 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:05:54.890 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-28T11:05:55.050 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-28T11:05:55.211 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-28T11:05:55.344 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T11:05:55.489 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T11:05:55.637 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T11:05:56.248 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T11:05:56.428 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T11:05:57.024 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T11:05:57.186 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T11:05:57.900 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T11:05:58.210 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-28T11:05:58.237 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-11-28T11:05:58.392 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-28T11:05:58.501 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T11:05:58.603 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T11:05:58.750 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T11:05:58.775 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T11:05:58.896 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T11:05:58.974 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-28T11:05:59.133 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T11:05:59.169 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T11:05:59.279 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T11:05:59.327 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T11:05:59.460 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T11:05:59.842 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T11:06:00.043 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:06:00.422 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:06:00.601 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:06:00.734 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:06:00.943 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:06:01.044 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:06:01.076 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T11:06:01.236 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:06:01.505 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T11:06:01.696 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T11:06:02.090 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-28T11:06:02.095 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T11:06:02.338 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T11:06:02.398 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-28T11:06:02.543 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:06:02.651 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:06:02.661 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-28T11:06:02.843 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:06:02.843 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-28T11:06:02.919 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:06:03.034 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T11:06:03.038 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T11:06:03.203 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T11:06:03.205 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T11:06:03.351 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T11:06:03.387 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T11:06:03.566 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T11:06:03.700 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-28T11:06:03.852 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-28T11:06:03.966 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T11:06:04.018 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T11:06:04.073 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T11:06:04.142 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T11:06:04.269 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-28T11:06:04.488 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-28T11:06:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T11:06:04.719 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T11:06:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T11:06:04.947 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-28T11:06:04.959 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T11:06:05.113 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-28T11:06:05.267 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-28T11:06:05.447 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-28T11:06:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-28T11:06:05.666 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T11:06:05.779 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-28T11:06:05.907 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-28T11:06:05.963 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-28T11:06:06.145 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-28T11:06:06.159 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-28T11:06:06.371 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-28T11:06:06.508 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-28T11:06:06.565 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-28T11:06:06.773 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-28T11:06:06.794 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-28T11:06:06.993 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T11:06:07.012 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-28T11:06:07.182 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T11:06:07.238 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-28T11:06:07.310 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T11:06:07.439 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-28T11:06:07.741 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-28T11:06:07.959 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-28T11:06:08.127 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-28T11:06:08.202 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T11:06:08.323 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-28T11:06:08.464 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-28T11:06:08.540 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-28T11:06:08.620 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-28T11:06:08.777 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-28T11:06:08.930 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-28T11:06:09.075 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-28T11:06:09.213 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-28T11:06:09.267 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T11:06:09.455 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-28T11:06:09.669 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-28T11:06:09.825 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T11:06:10.003 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T11:06:10.259 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T11:06:10.457 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T11:06:10.604 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T11:06:10.744 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T11:06:10.940 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T11:06:11.025 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T11:06:11.136 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T11:06:11.326 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T11:06:11.425 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-28T11:06:11.478 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T11:06:11.597 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T11:06:11.610 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T11:06:11.832 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T11:06:11.834 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T11:06:11.997 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T11:06:12.036 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T11:06:12.145 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T11:06:12.205 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T11:06:12.341 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-28T11:06:12.457 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T11:06:12.542 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-28T11:06:12.699 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T11:06:12.710 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-28T11:06:12.869 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-28T11:06:12.872 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T11:06:13.055 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T11:06:13.086 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-28T11:06:13.293 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-28T11:06:13.656 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-28T11:06:13.894 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-28T11:06:14.101 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-28T11:06:14.116 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T11:06:14.302 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-28T11:06:14.474 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T11:06:14.528 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-28T11:06:14.685 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T11:06:14.864 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-28T11:06:14.959 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T11:06:15.107 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-28T11:06:15.234 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T11:06:15.317 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-28T11:06:15.419 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T11:06:15.530 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-28T11:06:15.687 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-28T11:06:15.860 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-28T11:06:16.179 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-28T11:06:16.183 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T11:06:16.358 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:06:16.430 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:06:16.470 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-28T11:06:16.609 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T11:06:16.695 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-28T11:06:16.955 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-28T11:06:17.123 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-28T11:06:17.266 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-28T11:06:17.432 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T11:06:17.525 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T11:06:17.754 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-28T11:06:17.952 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-28T11:06:18.132 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T11:06:18.195 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T11:06:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-28T11:06:18.583 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-28T11:06:18.773 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T11:06:18.960 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T11:06:19.236 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T11:06:19.447 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T11:06:19.594 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T11:06:19.725 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T11:06:19.933 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T11:06:20.119 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T11:06:20.226 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:20.294 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T11:06:20.425 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:20.477 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T11:06:20.613 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T11:06:20.829 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T11:06:21.004 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:21.052 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T11:06:21.237 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T11:06:21.291 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:06:21.377 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:06:21.487 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T11:06:21.515 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:06:21.731 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T11:06:21.881 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T11:06:22.063 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T11:06:23.156 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T11:06:23.409 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T11:06:23.622 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T11:06:23.897 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T11:06:24.228 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T11:06:24.406 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T11:06:25.180 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T11:06:25.330 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:06:25.406 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:06:25.640 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T11:06:29.304 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:29.731 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:30.309 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:30.593 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:06:30.675 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:06:30.846 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:06:40.293 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:06:40.293 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2023-11-28T11:06:40.293 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2023-11-28T11:06:40.293 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-11-28T11:06:40.293 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /run 2023-11-28T11:06:40.294 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2023-11-28T11:06:40.294 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2023-11-28T11:06:40.294 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2023-11-28T11:06:40.294 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2023-11-28T11:06:40.294 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-11-28T11:06:40.294 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:06:40.630 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:06:42.207 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:06:42.371 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:06:42.588 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:06:42.730 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:06:43.183 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:06:43.371 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:06:43.388 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:06:45.063 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:06:45.231 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:06:45.343 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T11:06:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T11:06:45.678 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:06:45.895 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:06:46.208 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:06:47.171 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:06:47.320 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:06:47.409 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:06:47.781 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T11:06:47.953 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:06:47.989 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:06:48.137 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:06:48.148 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:06:49.384 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T11:06:49.600 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:06:49.633 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:06:49.778 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:06:49.820 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:06:49.947 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:06:49.948 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2023-11-28T11:06:49.948 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2023-11-28T11:06:49.948 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-11-28T11:06:49.948 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /run 2023-11-28T11:06:49.948 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2023-11-28T11:06:49.948 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2023-11-28T11:06:49.948 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2023-11-28T11:06:49.948 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2023-11-28T11:06:49.949 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-11-28T11:06:49.949 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:06:49.954 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:06:50.294 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:06:50.300 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:06:50.445 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:06:51.872 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:06:52.073 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:06:52.291 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:06:52.418 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:06:52.876 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:06:53.077 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:06:53.093 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:06:54.810 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:06:54.878 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:06:55.016 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:06:55.132 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T11:06:55.290 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T11:06:55.509 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:06:55.728 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:06:55.740 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T11:06:56.043 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-28T11:06:56.055 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:06:56.222 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 117/134 2023-11-28T11:06:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-28T11:06:56.602 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T11:06:56.792 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T11:06:57.017 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:06:57.040 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-28T11:06:57.198 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T11:06:57.217 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:06:57.308 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:06:57.329 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:06:57.658 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T11:06:57.855 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:06:57.890 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:06:58.059 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:06:58.070 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:06:59.328 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T11:06:59.558 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:06:59.562 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:06:59.595 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:06:59.734 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:06:59.769 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:06:59.775 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:06:59.790 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:06:59.917 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:06:59.945 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T11:07:00.210 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T11:07:00.261 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:07:00.417 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:07:00.500 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T11:07:00.855 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T11:07:01.035 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T11:07:01.120 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T11:07:01.335 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T11:07:01.347 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T11:07:01.493 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T11:07:01.501 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T11:07:01.637 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-28T11:07:01.637 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:01.829 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:01.830 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:07:02.057 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:07:02.225 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:02.716 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:02.716 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T11:07:02.716 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T11:07:02.717 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T11:07:02.717 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T11:07:02.717 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T11:07:02.717 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:07:02.717 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:07:02.717 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:07:02.717 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:07:02.717 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T11:07:02.717 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T11:07:02.717 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T11:07:02.717 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T11:07:02.718 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T11:07:02.719 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T11:07:02.728 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T11:07:02.728 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T11:07:02.728 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T11:07:02.728 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T11:07:02.728 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T11:07:02.729 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T11:07:02.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T11:07:02.731 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-28T11:07:02.732 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-28T11:07:02.733 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-28T11:07:02.733 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-28T11:07:02.733 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-28T11:07:02.733 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-28T11:07:02.733 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-28T11:07:02.733 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-28T11:07:02.733 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-28T11:07:02.733 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-28T11:07:02.733 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-28T11:07:02.733 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-28T11:07:02.733 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-28T11:07:02.733 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-28T11:07:02.734 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-28T11:07:02.734 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-28T11:07:02.734 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-28T11:07:02.734 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-28T11:07:02.734 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-28T11:07:02.734 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-28T11:07:02.734 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-28T11:07:02.734 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-28T11:07:02.734 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-28T11:07:02.735 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-28T11:07:02.735 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-28T11:07:02.735 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-28T11:07:02.735 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-28T11:07:02.735 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-28T11:07:02.735 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-28T11:07:02.735 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-28T11:07:02.735 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-28T11:07:02.736 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-28T11:07:02.736 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-28T11:07:02.736 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-28T11:07:02.736 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-28T11:07:02.736 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-28T11:07:02.736 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-28T11:07:02.736 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-28T11:07:02.736 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-28T11:07:02.737 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-28T11:07:02.737 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-28T11:07:02.737 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-28T11:07:02.737 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-28T11:07:02.737 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-28T11:07:02.737 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T11:07:02.737 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T11:07:02.737 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T11:07:02.737 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:02.737 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T11:07:04.830 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:07:05.106 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T11:07:05.291 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T11:07:05.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:05.428 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:54 ago on Tue 28 Nov 2023 11:05:11 AM UTC. 2023-11-28T11:07:05.483 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T11:07:06.363 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-28T11:07:06.673 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-28T11:07:06.830 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 119/134 2023-11-28T11:07:07.036 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T11:07:07.195 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T11:07:07.334 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:07:08.164 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-11-28T11:07:08.399 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:08.400 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2023-11-28T11:07:08.400 INFO:teuthology.orchestra.run.smithi186.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T11:07:08.400 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.400 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.400 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-11-28T11:07:08.400 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T11:07:08.400 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T11:07:08.400 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T11:07:08.400 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.400 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.400 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.400 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.401 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:08.401 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.401 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.401 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.401 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:08.401 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:08.401 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:08.401 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:08.401 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:08.401 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.401 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.401 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T11:07:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T11:07:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T11:07:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T11:07:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: jq-1.6-3.el8.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T11:07:08.403 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.404 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.404 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T11:07:08.404 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.404 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T11:07:08.404 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T11:07:08.404 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T11:07:08.404 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T11:07:08.404 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T11:07:08.404 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T11:07:08.404 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T11:07:08.404 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T11:07:08.404 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T11:07:08.405 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T11:07:08.406 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.406 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.406 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.406 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T11:07:08.406 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T11:07:08.406 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T11:07:08.406 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T11:07:08.406 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T11:07:08.406 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T11:07:08.406 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T11:07:08.406 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T11:07:08.406 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T11:07:08.407 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T11:07:08.407 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T11:07:08.407 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T11:07:08.407 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T11:07:08.407 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T11:07:08.407 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T11:07:08.407 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T11:07:08.407 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T11:07:08.407 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T11:07:08.407 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T11:07:08.407 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T11:07:08.407 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T11:07:08.408 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T11:07:08.408 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T11:07:08.408 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T11:07:08.408 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T11:07:08.408 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T11:07:08.408 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T11:07:08.408 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T11:07:08.408 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T11:07:08.408 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.408 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.408 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T11:07:08.409 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T11:07:08.410 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T11:07:08.410 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T11:07:08.410 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.410 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.410 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:08.410 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T11:07:08.410 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T11:07:08.410 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T11:07:08.410 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T11:07:08.410 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T11:07:08.411 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T11:07:08.411 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:08.411 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-11-28T11:07:08.587 DEBUG:teuthology.parallel:result is None 2023-11-28T11:07:09.550 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:09.953 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T11:07:10.240 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T11:07:10.530 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T11:07:10.792 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T11:07:10.932 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:07:10.940 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:07:11.138 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-28T11:07:11.306 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T11:07:11.388 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T11:07:11.623 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T11:07:11.636 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T11:07:11.636 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:11.835 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:11.835 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:07:12.022 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:07:12.205 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:12.724 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:12.724 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T11:07:12.724 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T11:07:12.724 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T11:07:12.724 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T11:07:12.724 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T11:07:12.725 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:07:12.725 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:07:12.725 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:07:12.725 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:07:12.725 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T11:07:12.725 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T11:07:12.725 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T11:07:12.725 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T11:07:12.725 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T11:07:12.725 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T11:07:12.725 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T11:07:12.725 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T11:07:12.726 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T11:07:12.726 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T11:07:12.726 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T11:07:12.726 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T11:07:12.726 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T11:07:12.726 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T11:07:12.726 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T11:07:12.726 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T11:07:12.726 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T11:07:12.726 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T11:07:12.726 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T11:07:12.726 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T11:07:12.727 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T11:07:12.727 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T11:07:12.727 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T11:07:12.727 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T11:07:12.727 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T11:07:12.727 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T11:07:12.727 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T11:07:12.727 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T11:07:12.727 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T11:07:12.727 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T11:07:12.727 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T11:07:12.727 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T11:07:12.728 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T11:07:12.729 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T11:07:12.729 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T11:07:12.729 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T11:07:12.729 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T11:07:12.729 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T11:07:12.729 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T11:07:12.729 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T11:07:12.729 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T11:07:12.729 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T11:07:12.729 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T11:07:12.729 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T11:07:12.729 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T11:07:12.730 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T11:07:12.730 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T11:07:12.730 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T11:07:12.730 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T11:07:12.730 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T11:07:12.730 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-28T11:07:12.730 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-28T11:07:12.730 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-28T11:07:12.730 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-28T11:07:12.730 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-28T11:07:12.730 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-28T11:07:12.730 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-28T11:07:12.731 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-28T11:07:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-28T11:07:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-28T11:07:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-28T11:07:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-28T11:07:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-28T11:07:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-28T11:07:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-28T11:07:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-28T11:07:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-28T11:07:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-28T11:07:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-28T11:07:12.732 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-28T11:07:12.733 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T11:07:12.734 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T11:07:12.735 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T11:07:12.735 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:12.735 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T11:07:15.486 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:15.486 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:02:01 ago on Tue 28 Nov 2023 11:05:14 AM UTC. 2023-11-28T11:07:18.173 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-11-28T11:07:18.403 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:18.404 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2023-11-28T11:07:18.404 INFO:teuthology.orchestra.run.smithi156.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T11:07:18.404 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.404 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.404 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-11-28T11:07:18.404 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T11:07:18.404 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T11:07:18.404 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T11:07:18.404 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.404 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.404 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.405 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.405 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:18.405 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.405 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.405 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.405 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:18.405 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:18.405 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:18.405 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:18.405 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:18.405 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.405 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.406 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:18.406 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.406 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.406 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.406 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:18.406 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:18.406 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.406 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:18.406 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T11:07:18.406 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T11:07:18.406 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T11:07:18.407 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T11:07:18.407 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-3.el8.x86_64 2023-11-28T11:07:18.407 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T11:07:18.407 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T11:07:18.407 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T11:07:18.407 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.407 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.407 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.407 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T11:07:18.407 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:07:18.407 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T11:07:18.407 INFO:teuthology.orchestra.run.smithi156.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T11:07:18.408 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:07:18.408 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T11:07:18.408 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.408 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.408 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T11:07:18.408 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.408 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T11:07:18.408 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T11:07:18.408 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T11:07:18.408 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T11:07:18.409 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T11:07:18.409 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T11:07:18.409 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T11:07:18.409 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T11:07:18.409 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T11:07:18.409 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T11:07:18.409 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T11:07:18.410 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T11:07:18.410 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T11:07:18.410 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T11:07:18.410 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:18.410 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:18.410 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:18.410 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T11:07:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T11:07:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T11:07:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T11:07:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T11:07:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T11:07:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T11:07:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T11:07:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T11:07:18.411 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T11:07:18.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T11:07:18.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T11:07:18.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T11:07:18.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T11:07:18.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T11:07:18.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T11:07:18.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T11:07:18.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T11:07:18.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T11:07:18.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T11:07:18.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T11:07:18.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T11:07:18.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T11:07:18.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T11:07:18.415 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T11:07:18.415 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T11:07:18.415 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T11:07:18.415 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T11:07:18.415 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.415 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.415 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:18.415 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T11:07:18.415 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T11:07:18.415 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T11:07:18.415 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T11:07:18.415 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T11:07:18.416 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T11:07:18.416 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:18.416 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-11-28T11:07:18.611 DEBUG:teuthology.parallel:result is None 2023-11-28T11:07:18.611 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:07:18.828 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T11:07:18.851 INFO:teuthology.orchestra.run.smithi156.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T11:07:18.853 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T11:07:18.853 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T11:07:18.853 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:07:19.002 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T11:07:19.032 INFO:teuthology.orchestra.run.smithi186.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T11:07:19.032 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T11:07:19.032 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T11:07:19.032 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T11:07:19.033 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:19.033 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T11:07:19.075 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:19.075 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T11:07:19.122 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T11:07:19.122 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:19.122 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T11:07:19.150 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T11:07:19.218 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:19.218 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T11:07:19.248 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T11:07:19.319 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T11:07:19.319 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:19.319 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T11:07:19.356 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T11:07:19.426 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:19.426 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T11:07:19.455 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T11:07:19.524 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T11:07:19.524 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:19.524 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T11:07:19.550 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T11:07:19.616 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:19.616 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T11:07:19.644 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T11:07:19.712 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T11:07:19.773 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T11:07:19.773 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /var/log/ceph 2023-11-28T11:07:19.775 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /var/log/ceph 2023-11-28T11:07:19.802 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T11:07:19.802 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T11:07:19.843 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T11:07:19.870 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T11:07:19.870 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T11:07:19.909 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T11:07:19.943 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T11:07:19.943 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T11:07:19.944 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T11:07:19.944 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:19.944 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T11:07:19.990 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:20.121 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:20.149 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:20.149 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T11:07:20.166 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:20.301 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:20.333 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T11:07:20.333 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T11:07:20.333 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956', 'branch': 'wip-neesingh-testing-231123', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7469956', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-231123'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'issue_54557', 'sha1': 'e67b67438c1e5e13734a7203ff9e57e30b8e4d32'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9437, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'issue_54557', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'e67b67438c1e5e13734a7203ff9e57e30b8e4d32', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi156.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi186.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '5ae09a5cc7064ca2bf994df19f3006b79794789a', 'timestamp': '2023-11-28_10:48:03', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T11:07:20.334 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T11:07:20.336 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T11:07:20.385 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T11:07:20.393 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T11:07:20.451 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:20.452 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T11:07:20.473 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T11:07:20.473 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2023-11-28T11:07:20.531 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T11:07:20.531 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:20.531 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 379754 Links: 1 2023-11-28T11:07:20.531 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:20.531 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:20.531 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-11-28 11:07:12.414179624 +0000 2023-11-28T11:07:20.531 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-11-28 11:04:07.033261933 +0000 2023-11-28T11:07:20.531 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-11-28 11:04:07.033261933 +0000 2023-11-28T11:07:20.531 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-11-28T11:07:20.532 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T11:07:20.596 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-11-28T11:07:20.596 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-11-28T11:07:20.596 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.00012485 s, 4.1 MB/s 2023-11-28T11:07:20.597 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T11:07:20.655 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2023-11-28T11:07:20.712 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T11:07:20.713 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:20.713 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 392240 Links: 1 2023-11-28T11:07:20.713 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:20.713 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:20.713 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-11-28 11:07:12.414179624 +0000 2023-11-28T11:07:20.713 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-11-28 11:04:07.579254020 +0000 2023-11-28T11:07:20.714 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-11-28 11:04:07.579254020 +0000 2023-11-28T11:07:20.714 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-11-28T11:07:20.714 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T11:07:20.778 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-11-28T11:07:20.778 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-11-28T11:07:20.778 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 8.5703e-05 s, 6.0 MB/s 2023-11-28T11:07:20.779 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T11:07:20.836 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2023-11-28T11:07:20.894 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T11:07:20.894 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:20.894 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 383886 Links: 1 2023-11-28T11:07:20.894 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:20.894 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:20.894 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-11-28 11:07:12.415179610 +0000 2023-11-28T11:07:20.895 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-11-28 11:04:08.188245195 +0000 2023-11-28T11:07:20.895 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-11-28 11:04:08.188245195 +0000 2023-11-28T11:07:20.895 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-11-28T11:07:20.895 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T11:07:20.958 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-11-28T11:07:20.959 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-11-28T11:07:20.959 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000120094 s, 4.3 MB/s 2023-11-28T11:07:20.960 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T11:07:21.017 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2023-11-28T11:07:21.075 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T11:07:21.075 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:21.076 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 383927 Links: 1 2023-11-28T11:07:21.076 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:21.076 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:21.076 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-11-28 11:07:12.415179610 +0000 2023-11-28T11:07:21.077 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-11-28 11:04:08.754236992 +0000 2023-11-28T11:07:21.077 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-11-28 11:04:08.754236992 +0000 2023-11-28T11:07:21.077 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-11-28T11:07:21.077 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T11:07:21.140 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-11-28T11:07:21.141 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-11-28T11:07:21.141 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000116669 s, 4.4 MB/s 2023-11-28T11:07:21.142 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T11:07:21.199 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:21.199 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:21.200 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T11:07:21.219 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T11:07:21.219 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2023-11-28T11:07:21.278 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T11:07:21.279 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:21.279 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 391312 Links: 1 2023-11-28T11:07:21.279 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:21.279 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:21.279 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-11-28 11:07:02.424832074 +0000 2023-11-28T11:07:21.279 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-11-28 11:04:21.994762918 +0000 2023-11-28T11:07:21.279 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-11-28 11:04:21.994762918 +0000 2023-11-28T11:07:21.279 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-11-28T11:07:21.279 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T11:07:21.348 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-11-28T11:07:21.348 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-11-28T11:07:21.348 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000277236 s, 1.8 MB/s 2023-11-28T11:07:21.349 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T11:07:21.409 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2023-11-28T11:07:21.468 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T11:07:21.468 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:21.468 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 385964 Links: 1 2023-11-28T11:07:21.468 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:21.468 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:21.468 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-11-28 11:07:02.425832056 +0000 2023-11-28T11:07:21.468 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-11-28 11:04:22.520753397 +0000 2023-11-28T11:07:21.468 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-11-28 11:04:22.520753397 +0000 2023-11-28T11:07:21.469 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-11-28T11:07:21.469 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T11:07:21.535 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-11-28T11:07:21.536 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-11-28T11:07:21.536 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000252245 s, 2.0 MB/s 2023-11-28T11:07:21.537 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T11:07:21.596 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2023-11-28T11:07:21.654 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T11:07:21.654 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:21.654 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 384940 Links: 1 2023-11-28T11:07:21.654 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:21.655 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:21.655 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-11-28 11:07:02.425832056 +0000 2023-11-28T11:07:21.655 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-11-28 11:04:23.027744219 +0000 2023-11-28T11:07:21.655 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-11-28 11:04:23.027744219 +0000 2023-11-28T11:07:21.655 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-11-28T11:07:21.655 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T11:07:21.721 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-11-28T11:07:21.721 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-11-28T11:07:21.721 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000247085 s, 2.1 MB/s 2023-11-28T11:07:21.722 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T11:07:21.781 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2023-11-28T11:07:21.839 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T11:07:21.839 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:21.839 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 390843 Links: 1 2023-11-28T11:07:21.839 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:21.839 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:21.839 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-11-28 11:07:02.425832056 +0000 2023-11-28T11:07:21.839 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-11-28 11:04:23.566734462 +0000 2023-11-28T11:07:21.839 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-11-28 11:04:23.566734462 +0000 2023-11-28T11:07:21.840 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-11-28T11:07:21.840 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T11:07:21.905 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-11-28T11:07:21.905 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-11-28T11:07:21.905 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000197726 s, 2.6 MB/s 2023-11-28T11:07:21.906 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T11:07:21.964 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:21.964 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi156.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi186.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T11:07:21.964 INFO:tasks.ceph:Generating config... 2023-11-28T11:07:21.966 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T11:07:21.966 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T11:07:21.966 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T11:07:21.966 INFO:tasks.ceph:[client] pid file = 2023-11-28T11:07:21.966 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T11:07:21.966 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T11:07:21.966 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-28T11:07:21.966 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T11:07:21.967 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T11:07:21.967 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T11:07:21.967 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T11:07:21.967 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T11:07:21.967 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T11:07:21.967 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T11:07:21.967 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T11:07:21.967 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T11:07:21.967 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T11:07:21.967 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T11:07:21.968 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T11:07:21.968 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T11:07:21.968 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T11:07:21.968 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T11:07:21.968 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T11:07:21.968 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-28T11:07:21.968 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-28T11:07:21.968 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-28T11:07:21.968 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T11:07:21.968 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T11:07:21.969 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-28T11:07:21.969 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-28T11:07:21.969 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T11:07:21.969 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T11:07:21.969 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-28T11:07:21.969 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T11:07:21.969 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T11:07:21.969 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T11:07:21.969 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T11:07:21.969 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T11:07:21.969 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T11:07:21.970 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T11:07:21.970 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T11:07:22.054 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T11:07:22.057 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T11:07:22.106 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:07:22.134 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.156'), ('mon.b', '172.21.15.186'), ('mon.c', '[v2:172.21.15.186:3301,v1:172.21.15.186:6790]')] 2023-11-28T11:07:22.135 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.156,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T11:07:22.136 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:22.136 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T11:07:22.192 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.156 --add b 172.21.15.186 --addv c '[v2:172.21.15.186:3301,v1:172.21.15.186:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:22.281 INFO:teuthology.orchestra.run.smithi156.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:22.281 INFO:teuthology.orchestra.run.smithi156.stdout:monmaptool: generated fsid 351da896-8dbf-473b-9472-c3454d9368bf 2023-11-28T11:07:22.282 INFO:teuthology.orchestra.run.smithi156.stdout:setting min_mon_release = pacific 2023-11-28T11:07:22.282 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 0 2023-11-28T11:07:22.282 INFO:teuthology.orchestra.run.smithi156.stdout:fsid 351da896-8dbf-473b-9472-c3454d9368bf 2023-11-28T11:07:22.282 INFO:teuthology.orchestra.run.smithi156.stdout:last_changed 2023-11-28T11:07:22.281452+0000 2023-11-28T11:07:22.282 INFO:teuthology.orchestra.run.smithi156.stdout:created 2023-11-28T11:07:22.281452+0000 2023-11-28T11:07:22.282 INFO:teuthology.orchestra.run.smithi156.stdout:min_mon_release 16 (pacific) 2023-11-28T11:07:22.282 INFO:teuthology.orchestra.run.smithi156.stdout:election_strategy: 1 2023-11-28T11:07:22.282 INFO:teuthology.orchestra.run.smithi156.stdout:0: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.a 2023-11-28T11:07:22.282 INFO:teuthology.orchestra.run.smithi156.stdout:1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.b 2023-11-28T11:07:22.283 INFO:teuthology.orchestra.run.smithi156.stdout:2: [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] mon.c 2023-11-28T11:07:22.283 INFO:teuthology.orchestra.run.smithi156.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T11:07:22.284 DEBUG:teuthology.orchestra.run.smithi156:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T11:07:22.302 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 351da896-8dbf-473b-9472-c3454d9368bf... 2023-11-28T11:07:22.303 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T11:07:22.344 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T11:07:22.386 INFO:teuthology.orchestra.run.smithi156.stdout:[global] 2023-11-28T11:07:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: chdir = "" 2023-11-28T11:07:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T11:07:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: auth supported = cephx 2023-11-28T11:07:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: filestore xattr use omap = true 2023-11-28T11:07:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: mon clock drift allowed = 1.000 2023-11-28T11:07:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: osd crush chooseleaf type = 0 2023-11-28T11:07:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: auth debug = true 2023-11-28T11:07:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on old message = true 2023-11-28T11:07:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on bug = true 2023-11-28T11:07:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T11:07:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: mon pg warn max object skew = 0 2023-11-28T11:07:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T11:07:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T11:07:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default size = 2 2023-11-28T11:07:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow primary affinity = true 2023-11-28T11:07:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow pg remap = true 2023-11-28T11:07:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on legacy crush tunables = false 2023-11-28T11:07:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on crush straw calc version zero = false 2023-11-28T11:07:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on no sortbitwise = false 2023-11-28T11:07:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on osd down out interval zero = false 2023-11-28T11:07:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on too few osds = false 2023-11-28T11:07:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T11:07:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T11:07:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: mon_allow_pool_size_one = true 2023-11-28T11:07:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T11:07:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: osd default data pool replay window = 5 2023-11-28T11:07:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: mon allow pool delete = true 2023-11-28T11:07:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: mon cluster log file level = debug 2023-11-28T11:07:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: debug asserts on shutdown = true 2023-11-28T11:07:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: mon health detail to clog = false 2023-11-28T11:07:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: mon host = "172.21.15.156,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2023-11-28T11:07:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on skipped message = False 2023-11-28T11:07:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: fsid = 351da896-8dbf-473b-9472-c3454d9368bf 2023-11-28T11:07:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.393 INFO:teuthology.orchestra.run.smithi156.stdout:[osd] 2023-11-28T11:07:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: osd journal size = 100 2023-11-28T11:07:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub load threshold = 5.0 2023-11-28T11:07:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub max interval = 600 2023-11-28T11:07:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: osd recover clone overlap = true 2023-11-28T11:07:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: osd recovery max chunk = 1048576 2023-11-28T11:07:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug shutdown = true 2023-11-28T11:07:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug op order = true 2023-11-28T11:07:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug verify stray on activate = true 2023-11-28T11:07:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: osd open classes on start = true 2023-11-28T11:07:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug pg log writeout = true 2023-11-28T11:07:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: osd deep scrub update digest min age = 30 2023-11-28T11:07:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: osd map max advance = 10 2023-11-28T11:07:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: journal zero on create = true 2023-11-28T11:07:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: filestore ondisk finisher threads = 3 2023-11-28T11:07:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: filestore apply finisher threads = 3 2023-11-28T11:07:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: bdev debug aio = true 2023-11-28T11:07:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug misdirected ops = true 2023-11-28T11:07:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: bdev async discard = True 2023-11-28T11:07:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: bdev enable discard = True 2023-11-28T11:07:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore allocator = bitmap 2023-11-28T11:07:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore block size = 96636764160 2023-11-28T11:07:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore fsck on mount = True 2023-11-28T11:07:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluefs = 1/20 2023-11-28T11:07:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluestore = 1/20 2023-11-28T11:07:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-28T11:07:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: debug osd = 20 2023-11-28T11:07:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: debug rocksdb = 4/10 2023-11-28T11:07:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T11:07:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd full ratio = 0.9 2023-11-28T11:07:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd nearfull ratio = 0.8 2023-11-28T11:07:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: osd failsafe full ratio = 0.95 2023-11-28T11:07:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: osd objectstore = bluestore 2023-11-28T11:07:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2023-11-28T11:07:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.398 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr] 2023-11-28T11:07:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-28T11:07:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgr = 20 2023-11-28T11:07:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2023-11-28T11:07:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2023-11-28T11:07:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:07:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:07:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: mgr/telemetry/nag = false 2023-11-28T11:07:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.399 INFO:teuthology.orchestra.run.smithi156.stdout:[mon] 2023-11-28T11:07:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-28T11:07:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2023-11-28T11:07:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: debug paxos = 20 2023-11-28T11:07:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2023-11-28T11:07:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: mon data avail warn = 5 2023-11-28T11:07:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: mon mgr mkfs grace = 240 2023-11-28T11:07:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:07:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd reporter subtree level = osd 2023-11-28T11:07:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd prime pg temp = true 2023-11-28T11:07:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:07:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T11:07:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T11:07:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T11:07:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T11:07:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T11:07:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T11:07:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: # 1m isn't quite enough 2023-11-28T11:07:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: mon_down_mkfs_grace = 2m 2023-11-28T11:07:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_filestore_osds = false 2023-11-28T11:07:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: mon op complaint time = 120 2023-11-28T11:07:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:22.403 INFO:teuthology.orchestra.run.smithi156.stdout:[client] 2023-11-28T11:07:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: rgw cache enabled = true 2023-11-28T11:07:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable ops log = true 2023-11-28T11:07:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable usage log = true 2023-11-28T11:07:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T11:07:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T11:07:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: client mount timeout = 600 2023-11-28T11:07:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: debug client = 20 2023-11-28T11:07:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-28T11:07:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = "" 2023-11-28T11:07:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2023-11-28T11:07:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2023-11-28T11:07:22.405 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.a] 2023-11-28T11:07:22.405 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.b] 2023-11-28T11:07:22.405 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.c] 2023-11-28T11:07:22.405 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2023-11-28T11:07:22.405 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2023-11-28T11:07:22.405 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2023-11-28T11:07:22.405 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2023-11-28T11:07:22.405 INFO:teuthology.orchestra.run.smithi156.stdout:[mds] 2023-11-28T11:07:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds = 20 2023-11-28T11:07:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds balancer = 20 2023-11-28T11:07:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-28T11:07:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug frag = True 2023-11-28T11:07:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug scatterstat = True 2023-11-28T11:07:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: mds op complaint time = 180 2023-11-28T11:07:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: mds verify scatter = True 2023-11-28T11:07:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2023-11-28T11:07:22.407 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2023-11-28T11:07:22.407 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2023-11-28T11:07:22.408 INFO:teuthology.orchestra.run.smithi186.stdout:[global] 2023-11-28T11:07:22.408 INFO:teuthology.orchestra.run.smithi186.stdout: chdir = "" 2023-11-28T11:07:22.409 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T11:07:22.409 INFO:teuthology.orchestra.run.smithi186.stdout: auth supported = cephx 2023-11-28T11:07:22.409 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.409 INFO:teuthology.orchestra.run.smithi186.stdout: filestore xattr use omap = true 2023-11-28T11:07:22.409 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.409 INFO:teuthology.orchestra.run.smithi186.stdout: mon clock drift allowed = 1.000 2023-11-28T11:07:22.409 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.410 INFO:teuthology.orchestra.run.smithi186.stdout: osd crush chooseleaf type = 0 2023-11-28T11:07:22.410 INFO:teuthology.orchestra.run.smithi186.stdout: auth debug = true 2023-11-28T11:07:22.410 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.410 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on old message = true 2023-11-28T11:07:22.410 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on bug = true 2023-11-28T11:07:22.410 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.410 INFO:teuthology.orchestra.run.smithi186.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T11:07:22.410 INFO:teuthology.orchestra.run.smithi186.stdout: mon pg warn max object skew = 0 2023-11-28T11:07:22.411 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.411 INFO:teuthology.orchestra.run.smithi186.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T11:07:22.411 INFO:teuthology.orchestra.run.smithi186.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T11:07:22.411 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.411 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default size = 2 2023-11-28T11:07:22.411 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.411 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow primary affinity = true 2023-11-28T11:07:22.412 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow pg remap = true 2023-11-28T11:07:22.412 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on legacy crush tunables = false 2023-11-28T11:07:22.412 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on crush straw calc version zero = false 2023-11-28T11:07:22.412 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on no sortbitwise = false 2023-11-28T11:07:22.412 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on osd down out interval zero = false 2023-11-28T11:07:22.412 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on too few osds = false 2023-11-28T11:07:22.412 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T11:07:22.412 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T11:07:22.413 INFO:teuthology.orchestra.run.smithi186.stdout: mon_allow_pool_size_one = true 2023-11-28T11:07:22.413 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.413 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T11:07:22.413 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.413 INFO:teuthology.orchestra.run.smithi186.stdout: osd default data pool replay window = 5 2023-11-28T11:07:22.413 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.413 INFO:teuthology.orchestra.run.smithi186.stdout: mon allow pool delete = true 2023-11-28T11:07:22.413 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.414 INFO:teuthology.orchestra.run.smithi186.stdout: mon cluster log file level = debug 2023-11-28T11:07:22.414 INFO:teuthology.orchestra.run.smithi186.stdout: debug asserts on shutdown = true 2023-11-28T11:07:22.414 INFO:teuthology.orchestra.run.smithi186.stdout: mon health detail to clog = false 2023-11-28T11:07:22.414 INFO:teuthology.orchestra.run.smithi186.stdout: mon host = "172.21.15.156,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2023-11-28T11:07:22.414 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on skipped message = False 2023-11-28T11:07:22.414 INFO:teuthology.orchestra.run.smithi186.stdout: fsid = 351da896-8dbf-473b-9472-c3454d9368bf 2023-11-28T11:07:22.414 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.414 INFO:teuthology.orchestra.run.smithi186.stdout:[osd] 2023-11-28T11:07:22.415 INFO:teuthology.orchestra.run.smithi186.stdout: osd journal size = 100 2023-11-28T11:07:22.415 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.415 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub load threshold = 5.0 2023-11-28T11:07:22.415 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub max interval = 600 2023-11-28T11:07:22.415 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.415 INFO:teuthology.orchestra.run.smithi186.stdout: osd recover clone overlap = true 2023-11-28T11:07:22.415 INFO:teuthology.orchestra.run.smithi186.stdout: osd recovery max chunk = 1048576 2023-11-28T11:07:22.415 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.416 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug shutdown = true 2023-11-28T11:07:22.416 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug op order = true 2023-11-28T11:07:22.416 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug verify stray on activate = true 2023-11-28T11:07:22.416 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.416 INFO:teuthology.orchestra.run.smithi186.stdout: osd open classes on start = true 2023-11-28T11:07:22.416 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug pg log writeout = true 2023-11-28T11:07:22.416 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.417 INFO:teuthology.orchestra.run.smithi186.stdout: osd deep scrub update digest min age = 30 2023-11-28T11:07:22.417 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.417 INFO:teuthology.orchestra.run.smithi186.stdout: osd map max advance = 10 2023-11-28T11:07:22.417 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.417 INFO:teuthology.orchestra.run.smithi186.stdout: journal zero on create = true 2023-11-28T11:07:22.417 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.417 INFO:teuthology.orchestra.run.smithi186.stdout: filestore ondisk finisher threads = 3 2023-11-28T11:07:22.417 INFO:teuthology.orchestra.run.smithi186.stdout: filestore apply finisher threads = 3 2023-11-28T11:07:22.417 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.418 INFO:teuthology.orchestra.run.smithi186.stdout: bdev debug aio = true 2023-11-28T11:07:22.418 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug misdirected ops = true 2023-11-28T11:07:22.418 INFO:teuthology.orchestra.run.smithi186.stdout: bdev async discard = True 2023-11-28T11:07:22.418 INFO:teuthology.orchestra.run.smithi186.stdout: bdev enable discard = True 2023-11-28T11:07:22.418 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore allocator = bitmap 2023-11-28T11:07:22.418 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore block size = 96636764160 2023-11-28T11:07:22.418 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore fsck on mount = True 2023-11-28T11:07:22.419 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluefs = 1/20 2023-11-28T11:07:22.419 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluestore = 1/20 2023-11-28T11:07:22.419 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-11-28T11:07:22.419 INFO:teuthology.orchestra.run.smithi186.stdout: debug osd = 20 2023-11-28T11:07:22.419 INFO:teuthology.orchestra.run.smithi186.stdout: debug rocksdb = 4/10 2023-11-28T11:07:22.419 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T11:07:22.419 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd full ratio = 0.9 2023-11-28T11:07:22.419 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd nearfull ratio = 0.8 2023-11-28T11:07:22.419 INFO:teuthology.orchestra.run.smithi186.stdout: osd failsafe full ratio = 0.95 2023-11-28T11:07:22.419 INFO:teuthology.orchestra.run.smithi186.stdout: osd objectstore = bluestore 2023-11-28T11:07:22.419 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2023-11-28T11:07:22.420 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.420 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr] 2023-11-28T11:07:22.420 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-11-28T11:07:22.420 INFO:teuthology.orchestra.run.smithi186.stdout: debug mgr = 20 2023-11-28T11:07:22.420 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2023-11-28T11:07:22.420 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2023-11-28T11:07:22.420 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:07:22.420 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:07:22.420 INFO:teuthology.orchestra.run.smithi186.stdout: mgr/telemetry/nag = false 2023-11-28T11:07:22.420 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.420 INFO:teuthology.orchestra.run.smithi186.stdout:[mon] 2023-11-28T11:07:22.420 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-11-28T11:07:22.421 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2023-11-28T11:07:22.421 INFO:teuthology.orchestra.run.smithi186.stdout: debug paxos = 20 2023-11-28T11:07:22.421 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2023-11-28T11:07:22.421 INFO:teuthology.orchestra.run.smithi186.stdout: mon data avail warn = 5 2023-11-28T11:07:22.421 INFO:teuthology.orchestra.run.smithi186.stdout: mon mgr mkfs grace = 240 2023-11-28T11:07:22.421 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:07:22.421 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd reporter subtree level = osd 2023-11-28T11:07:22.421 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd prime pg temp = true 2023-11-28T11:07:22.421 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:07:22.421 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.421 INFO:teuthology.orchestra.run.smithi186.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T11:07:22.421 INFO:teuthology.orchestra.run.smithi186.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T11:07:22.422 INFO:teuthology.orchestra.run.smithi186.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T11:07:22.422 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.422 INFO:teuthology.orchestra.run.smithi186.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T11:07:22.422 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T11:07:22.422 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T11:07:22.422 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.422 INFO:teuthology.orchestra.run.smithi186.stdout: # 1m isn't quite enough 2023-11-28T11:07:22.422 INFO:teuthology.orchestra.run.smithi186.stdout: mon_down_mkfs_grace = 2m 2023-11-28T11:07:22.422 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.422 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_filestore_osds = false 2023-11-28T11:07:22.422 INFO:teuthology.orchestra.run.smithi186.stdout: mon op complaint time = 120 2023-11-28T11:07:22.423 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T11:07:22.423 INFO:teuthology.orchestra.run.smithi186.stdout:[client] 2023-11-28T11:07:22.423 INFO:teuthology.orchestra.run.smithi186.stdout: rgw cache enabled = true 2023-11-28T11:07:22.423 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable ops log = true 2023-11-28T11:07:22.423 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable usage log = true 2023-11-28T11:07:22.423 INFO:teuthology.orchestra.run.smithi186.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T11:07:22.423 INFO:teuthology.orchestra.run.smithi186.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T11:07:22.423 INFO:teuthology.orchestra.run.smithi186.stdout: client mount timeout = 600 2023-11-28T11:07:22.423 INFO:teuthology.orchestra.run.smithi186.stdout: debug client = 20 2023-11-28T11:07:22.423 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-11-28T11:07:22.423 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = "" 2023-11-28T11:07:22.423 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2023-11-28T11:07:22.424 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2023-11-28T11:07:22.424 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.a] 2023-11-28T11:07:22.424 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.b] 2023-11-28T11:07:22.424 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.c] 2023-11-28T11:07:22.424 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.a] 2023-11-28T11:07:22.424 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.c] 2023-11-28T11:07:22.424 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.b] 2023-11-28T11:07:22.424 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.d] 2023-11-28T11:07:22.424 INFO:teuthology.orchestra.run.smithi186.stdout:[mds] 2023-11-28T11:07:22.424 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds = 20 2023-11-28T11:07:22.424 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds balancer = 20 2023-11-28T11:07:22.424 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-11-28T11:07:22.425 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug frag = True 2023-11-28T11:07:22.425 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug scatterstat = True 2023-11-28T11:07:22.425 INFO:teuthology.orchestra.run.smithi186.stdout: mds op complaint time = 180 2023-11-28T11:07:22.425 INFO:teuthology.orchestra.run.smithi186.stdout: mds verify scatter = True 2023-11-28T11:07:22.425 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2023-11-28T11:07:22.425 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2023-11-28T11:07:22.425 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2023-11-28T11:07:22.425 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T11:07:22.425 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T11:07:22.505 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T11:07:22.505 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:22.505 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T11:07:22.522 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:22.522 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T11:07:22.578 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:07:22.579 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:22.579 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T11:07:22.579 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:07:22.655 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:22.656 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:22.712 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T11:07:22.713 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:22.713 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T11:07:22.713 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:07:22.757 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:22.758 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:22.815 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T11:07:22.815 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T11:07:22.816 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T11:07:22.869 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T11:07:22.872 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T11:07:22.985 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T11:07:22.988 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T11:07:22.988 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T11:07:23.042 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T11:07:23.044 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T11:07:23.072 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T11:07:23.165 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T11:07:23.168 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T11:07:23.195 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T11:07:23.258 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T11:07:23.261 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T11:07:23.289 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T11:07:23.386 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T11:07:23.388 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T11:07:23.415 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T11:07:23.416 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T11:07:23.460 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T11:07:23.473 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-28T11:07:23.516 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-28T11:07:23.529 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-28T11:07:23.574 INFO:teuthology.orchestra.run.smithi186.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-28T11:07:23.586 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-28T11:07:23.629 INFO:teuthology.orchestra.run.smithi186.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-28T11:07:23.642 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T11:07:23.642 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi156.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi186.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T11:07:23.643 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T11:07:23.670 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:23.671 INFO:tasks.ceph:role: osd.0 2023-11-28T11:07:23.671 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:07:23.671 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T11:07:23.738 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:23.738 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:23.738 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:23.738 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:23.738 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:23.738 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:23.739 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:23.739 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:23.739 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:23.739 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:23.760 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-11-28T11:07:23.762 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-11-28T11:07:23.762 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T11:07:23.843 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T11:07:23.914 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T11:07:23.980 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:23.981 INFO:tasks.ceph:role: osd.1 2023-11-28T11:07:23.981 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:07:23.981 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T11:07:24.047 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:24.047 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:24.047 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:24.047 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:24.048 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:24.048 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:24.048 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:24.048 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:24.048 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:24.048 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:24.070 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-11-28T11:07:24.071 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-11-28T11:07:24.071 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T11:07:24.151 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T11:07:24.222 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T11:07:24.290 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:24.290 INFO:tasks.ceph:role: osd.2 2023-11-28T11:07:24.290 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:07:24.290 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T11:07:24.356 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:24.356 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:24.356 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:24.357 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:24.357 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:24.357 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:24.357 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:24.357 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:24.357 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:24.357 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:24.379 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-11-28T11:07:24.380 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-11-28T11:07:24.381 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T11:07:24.464 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T11:07:24.536 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T11:07:24.606 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:24.606 INFO:tasks.ceph:role: osd.3 2023-11-28T11:07:24.606 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:07:24.607 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T11:07:24.675 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:24.675 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:24.675 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:24.675 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:24.676 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:24.676 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:24.677 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:24.677 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:24.677 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:24.677 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:24.698 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-11-28T11:07:24.699 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-11-28T11:07:24.700 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T11:07:24.779 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T11:07:24.850 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:24.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:24.966+0000 7fc4971e4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T11:07:24.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:24.966+0000 7fc4971e4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T11:07:24.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:24.966+0000 7fc4971e4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:07:24.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:24.966+0000 7fc4971e4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:07:24.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:24.966+0000 7fc4971e4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:07:24.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:24.967+0000 7fc4971e4640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T11:07:27.830 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T11:07:27.862 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:27.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:27.951+0000 7f462b82e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T11:07:27.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:27.951+0000 7f462b82e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T11:07:27.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:27.951+0000 7f462b82e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:07:27.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:27.951+0000 7f462b82e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:07:27.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:27.951+0000 7f462b82e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:07:27.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:27.951+0000 7f462b82e640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T11:07:30.825 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T11:07:30.856 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:30.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:30.943+0000 7f8802112640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T11:07:30.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:30.943+0000 7f8802112640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T11:07:30.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:30.943+0000 7f8802112640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:07:30.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:30.943+0000 7f8802112640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:07:30.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:30.943+0000 7f8802112640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:07:30.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:30.943+0000 7f8802112640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T11:07:33.806 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T11:07:33.839 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:33.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:33.930+0000 7fba6bbb5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T11:07:33.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:33.930+0000 7fba6bbb5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T11:07:33.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:33.930+0000 7fba6bbb5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:07:33.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:33.930+0000 7fba6bbb5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:07:33.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:33.930+0000 7fba6bbb5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:07:33.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:33.931+0000 7fba6bbb5640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T11:07:36.792 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T11:07:36.823 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T11:07:36.856 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:36.857 INFO:tasks.ceph:role: osd.4 2023-11-28T11:07:36.857 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T11:07:36.857 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T11:07:36.935 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:36.935 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:36.935 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:36.935 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:36.936 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:36.936 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:36.936 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:36.936 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:36.936 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:36.936 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:37.367 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-11-28T11:07:37.368 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-11-28T11:07:37.369 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T11:07:37.543 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T11:07:37.582 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T11:07:37.652 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:37.653 INFO:tasks.ceph:role: osd.5 2023-11-28T11:07:37.653 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T11:07:37.653 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T11:07:37.731 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:37.731 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:37.731 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:37.731 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:37.731 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:37.731 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:37.731 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:37.731 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:37.732 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:37.732 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:38.155 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-11-28T11:07:38.157 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-11-28T11:07:38.158 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T11:07:38.277 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T11:07:38.311 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T11:07:38.381 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:38.381 INFO:tasks.ceph:role: osd.6 2023-11-28T11:07:38.382 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T11:07:38.382 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T11:07:38.458 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:38.458 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:38.458 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:38.458 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:38.458 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:38.458 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:38.458 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:38.458 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:38.459 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:38.459 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:38.903 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-11-28T11:07:38.904 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-11-28T11:07:38.905 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T11:07:39.065 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T11:07:39.098 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T11:07:39.167 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:39.167 INFO:tasks.ceph:role: osd.7 2023-11-28T11:07:39.167 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T11:07:39.167 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T11:07:39.242 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:39.242 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:39.243 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:39.243 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:39.243 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:39.243 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:39.243 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:39.243 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:39.243 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:39.243 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:39.689 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-11-28T11:07:39.691 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-11-28T11:07:39.691 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T11:07:39.879 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T11:07:39.916 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:40.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:40.027+0000 7f460e398640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T11:07:40.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:40.027+0000 7f460e398640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T11:07:40.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:40.027+0000 7f460e398640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:07:40.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:40.027+0000 7f460e398640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:07:40.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:40.027+0000 7f460e398640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:07:40.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:40.027+0000 7f460e398640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T11:07:42.903 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T11:07:42.934 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:43.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:43.027+0000 7f4c38624640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T11:07:43.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:43.027+0000 7f4c38624640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T11:07:43.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:43.027+0000 7f4c38624640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:07:43.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:43.027+0000 7f4c38624640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:07:43.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:43.027+0000 7f4c38624640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:07:43.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:43.028+0000 7f4c38624640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T11:07:45.894 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T11:07:45.926 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:46.017 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:46.016+0000 7f0f9e1d1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T11:07:46.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:46.017+0000 7f0f9e1d1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T11:07:46.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:46.017+0000 7f0f9e1d1640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:07:46.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:46.017+0000 7f0f9e1d1640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:07:46.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:46.017+0000 7f0f9e1d1640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:07:46.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:46.017+0000 7f0f9e1d1640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T11:07:48.891 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T11:07:48.920 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:49.010 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:49.008+0000 7f799217b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T11:07:49.010 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:49.008+0000 7f799217b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T11:07:49.010 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:49.008+0000 7f799217b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:07:49.010 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:49.008+0000 7f799217b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:07:49.010 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:49.008+0000 7f799217b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:07:49.010 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:49.008+0000 7f799217b640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T11:07:50.335 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:50.337 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:51.878 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T11:07:51.911 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T11:07:51.911 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:51.911 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T11:07:51.943 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:51.944 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T11:07:52.015 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:52.016 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T11:07:52.086 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:52.086 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T11:07:52.154 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:52.154 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T11:07:52.223 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:52.224 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T11:07:52.291 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:52.291 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T11:07:52.358 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:52.358 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T11:07:52.390 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:52.391 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T11:07:52.461 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:52.461 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T11:07:52.531 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:52.532 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T11:07:52.600 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:52.600 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T11:07:52.670 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:52.670 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T11:07:52.740 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:52.740 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T11:07:52.808 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:52.808 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T11:07:52.826 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:52.826 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-28T11:07:52.884 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:52.884 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-28T11:07:52.901 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:52.901 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-28T11:07:52.958 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T11:07:52.958 DEBUG:teuthology.orchestra.run.smithi156:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T11:07:52.960 DEBUG:teuthology.orchestra.run.smithi186:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T11:07:53.001 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.y] 2023-11-28T11:07:53.002 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDqyWVlI+zRMxAANSXxnBj90PtANlg25RwW1A== 2023-11-28T11:07:53.002 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2023-11-28T11:07:53.002 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDryWVlKLGIAhAAFJHYsC+LCoq5Qq1eh+MsgA== 2023-11-28T11:07:53.002 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2023-11-28T11:07:53.002 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDryWVlygniCRAAI6YYq5P02LiCGC4/tdGwqg== 2023-11-28T11:07:53.002 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.0] 2023-11-28T11:07:53.002 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDsyWVlhbi+ORAAG5EKPXefX8NjPR7TNi6Wgg== 2023-11-28T11:07:53.002 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.1] 2023-11-28T11:07:53.002 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDvyWVlu+nEOBAAmvcXFjC+MNeWkDsZsI30PA== 2023-11-28T11:07:53.002 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.2] 2023-11-28T11:07:53.003 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDyyWVlYMdIOBAABzlpv33IMXmgXqwROZ1n7A== 2023-11-28T11:07:53.006 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.3] 2023-11-28T11:07:53.006 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD1yWVltreINxAAcPoQjsD28WEyEYN7kZZEUg== 2023-11-28T11:07:53.006 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.x] 2023-11-28T11:07:53.006 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDqyWVly3m5OhAAWirKz8b4mHzNgLftULkWQg== 2023-11-28T11:07:53.006 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2023-11-28T11:07:53.006 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDryWVlr6ZnDxAAnJd4CtD5MUHauaV9GVqqVA== 2023-11-28T11:07:53.006 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2023-11-28T11:07:53.006 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDryWVlc4gEFxAA6J82vSOm+mN6hvRs7Ec8SA== 2023-11-28T11:07:53.006 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.4] 2023-11-28T11:07:53.007 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD8yWVlliukARAA3YVAwd8OePh6WT7H0Jykog== 2023-11-28T11:07:53.007 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.5] 2023-11-28T11:07:53.007 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD/yWVl6nilARAAfFTZi6p2kdxqOgtWonRRKA== 2023-11-28T11:07:53.007 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.6] 2023-11-28T11:07:53.007 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQACymVleb4IARAAXz8dDUDjem6eqAczZ5omDg== 2023-11-28T11:07:53.007 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.7] 2023-11-28T11:07:53.007 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAFymVlloyRABAAMa2gdE3weqqPWgiCI28jbQ== 2023-11-28T11:07:53.007 INFO:teuthology.orchestra.run.smithi156.stdout:[client.0] 2023-11-28T11:07:53.007 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDryWVl5LhsGxAA7WC9xFoSeBWdUPsmO5QaSg== 2023-11-28T11:07:53.007 INFO:teuthology.orchestra.run.smithi156.stdout:[client.2] 2023-11-28T11:07:53.007 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDryWVlnJDEHhAAEZX8g8t6Gbm6InUWOLqetw== 2023-11-28T11:07:53.008 INFO:teuthology.orchestra.run.smithi156.stdout:[client.1] 2023-11-28T11:07:53.008 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDryWVlXV40IhAAhx1t1uwqXPA2g7GmoSXVmA== 2023-11-28T11:07:53.008 INFO:teuthology.orchestra.run.smithi156.stdout:[client.3] 2023-11-28T11:07:53.008 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDryWVlPtmDJRAAoEvqSTpIIIWm42myVE3jVw== 2023-11-28T11:07:53.023 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr.y] 2023-11-28T11:07:53.024 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDqyWVlI+zRMxAANSXxnBj90PtANlg25RwW1A== 2023-11-28T11:07:53.024 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.a] 2023-11-28T11:07:53.024 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDryWVlKLGIAhAAFJHYsC+LCoq5Qq1eh+MsgA== 2023-11-28T11:07:53.024 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.c] 2023-11-28T11:07:53.024 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDryWVlygniCRAAI6YYq5P02LiCGC4/tdGwqg== 2023-11-28T11:07:53.024 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.0] 2023-11-28T11:07:53.024 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDsyWVlhbi+ORAAG5EKPXefX8NjPR7TNi6Wgg== 2023-11-28T11:07:53.024 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.1] 2023-11-28T11:07:53.024 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDvyWVlu+nEOBAAmvcXFjC+MNeWkDsZsI30PA== 2023-11-28T11:07:53.024 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.2] 2023-11-28T11:07:53.024 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDyyWVlYMdIOBAABzlpv33IMXmgXqwROZ1n7A== 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.3] 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD1yWVltreINxAAcPoQjsD28WEyEYN7kZZEUg== 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr.x] 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDqyWVly3m5OhAAWirKz8b4mHzNgLftULkWQg== 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.b] 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDryWVlr6ZnDxAAnJd4CtD5MUHauaV9GVqqVA== 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.d] 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDryWVlc4gEFxAA6J82vSOm+mN6hvRs7Ec8SA== 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.4] 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD8yWVlliukARAA3YVAwd8OePh6WT7H0Jykog== 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.5] 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD/yWVl6nilARAAfFTZi6p2kdxqOgtWonRRKA== 2023-11-28T11:07:53.025 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.6] 2023-11-28T11:07:53.026 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQACymVleb4IARAAXz8dDUDjem6eqAczZ5omDg== 2023-11-28T11:07:53.026 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.7] 2023-11-28T11:07:53.026 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAFymVlloyRABAAMa2gdE3weqqPWgiCI28jbQ== 2023-11-28T11:07:53.026 INFO:teuthology.orchestra.run.smithi186.stdout:[client.0] 2023-11-28T11:07:53.026 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDryWVl5LhsGxAA7WC9xFoSeBWdUPsmO5QaSg== 2023-11-28T11:07:53.026 INFO:teuthology.orchestra.run.smithi186.stdout:[client.2] 2023-11-28T11:07:53.026 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDryWVlnJDEHhAAEZX8g8t6Gbm6InUWOLqetw== 2023-11-28T11:07:53.026 INFO:teuthology.orchestra.run.smithi186.stdout:[client.1] 2023-11-28T11:07:53.026 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDryWVlXV40IhAAhx1t1uwqXPA2g7GmoSXVmA== 2023-11-28T11:07:53.026 INFO:teuthology.orchestra.run.smithi186.stdout:[client.3] 2023-11-28T11:07:53.026 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQDryWVlPtmDJRAAoEvqSTpIIIWm42myVE3jVw== 2023-11-28T11:07:53.028 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:07:53.052 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:07:53.122 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:07:53.124 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:07:53.180 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:07:53.182 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:07:53.237 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.239 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.292 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.295 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.350 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.353 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.406 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.409 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.464 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:07:53.467 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:07:53.521 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:07:53.524 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:07:53.581 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:07:53.584 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:07:53.640 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.642 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.696 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.699 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.751 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.753 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.807 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.809 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:07:53.862 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:07:53.864 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:07:53.921 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:07:53.923 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:07:53.979 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:07:53.982 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:07:54.039 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:07:54.041 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:07:54.097 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T11:07:54.098 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T11:07:54.124 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:07:54.223 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T11:07:54.249 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T11:07:54.277 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:07:54.375 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T11:07:54.401 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T11:07:54.469 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:07:54.568 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T11:07:54.594 DEBUG:teuthology.orchestra.run.smithi156:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:54.596 DEBUG:teuthology.orchestra.run.smithi186:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:54.651 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T11:07:54.651 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T11:07:54.651 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T11:07:54.654 INFO:tasks.ceph.mon.a:Started 2023-11-28T11:07:54.654 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T11:07:54.654 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T11:07:54.693 INFO:tasks.ceph.mon.b:Started 2023-11-28T11:07:54.693 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T11:07:54.694 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T11:07:54.695 INFO:tasks.ceph.mon.c:Started 2023-11-28T11:07:54.695 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T11:07:54.696 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T11:07:54.696 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T11:07:54.703 INFO:tasks.ceph.mgr.y:Started 2023-11-28T11:07:54.703 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T11:07:54.703 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T11:07:54.705 INFO:tasks.ceph.mgr.x:Started 2023-11-28T11:07:54.705 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T11:07:54.706 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T11:07:54.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.827+0000 7f5d65ffd700 1 Processor -- start 2023-11-28T11:07:54.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.827+0000 7f5d65ffd700 1 -- start start 2023-11-28T11:07:54.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.827+0000 7f5d65ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d60154e70 0x7f5d600cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:54.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.827+0000 7f5d65ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5d600cf360 0x7f5d60071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:54.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d65ffd700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d600719e0 con 0x7f5d60071580 2023-11-28T11:07:54.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d65ffd700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d60155240 con 0x7f5d600cf360 2023-11-28T11:07:54.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d65ffd700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6015b100 con 0x7f5d60154e70 2023-11-28T11:07:54.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d5effd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5d600cf360 0x7f5d60071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:54.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d5ffff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5d60071580 0x7f5d6015e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46012/0 (socket says 172.21.15.156:46012) 2023-11-28T11:07:54.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d5ffff700 1 -- 172.21.15.156:0/2822903479 learned_addr learned my addr 172.21.15.156:0/2822903479 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:07:54.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d5effd700 1 -- 172.21.15.156:0/2822903479 >> v2:172.21.15.156:3300/0 conn(0x7f5d600cf360 msgr2=0x7f5d60071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T11:07:54.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d5effd700 1 -- 172.21.15.156:0/2822903479 >> v2:172.21.15.156:3300/0 conn(0x7f5d600cf360 msgr2=0x7f5d60071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:07:54.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d5effd700 1 --2- 172.21.15.156:0/2822903479 >> v2:172.21.15.156:3300/0 conn(0x7f5d600cf360 0x7f5d60071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:07:54.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d5effd700 1 --2- 172.21.15.156:0/2822903479 >> v2:172.21.15.156:3300/0 conn(0x7f5d600cf360 0x7f5d60071040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:07:54.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d5f7fe700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d60154e70 0x7f5d600cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:54.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d5f7fe700 1 -- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d60154e70 msgr2=0x7f5d600cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T11:07:54.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d5f7fe700 1 -- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d60154e70 msgr2=0x7f5d600cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:07:54.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d5f7fe700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d60154e70 0x7f5d600cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:07:54.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.828+0000 7f5d5f7fe700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d60154e70 0x7f5d600cdbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:07:55.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.999+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956354717 0 0) 0x7f5d600719e0 con 0x7f5d60071580 2023-11-28T11:07:55.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:54.999+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d480035b0 con 0x7f5d60071580 2023-11-28T11:07:55.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3612606459 0 0) 0x7f5d480035b0 con 0x7f5d60071580 2023-11-28T11:07:55.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d60154e70 msgr2=0x7f5d600cdbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:07:55.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d60154e70 0x7f5d600cdbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 >> v2:172.21.15.156:3300/0 conn(0x7f5d600cf360 msgr2=0x7f5d60071040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:07:55.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 --2- 172.21.15.156:0/2822903479 >> v2:172.21.15.156:3300/0 conn(0x7f5d600cf360 0x7f5d60071040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6015b3e0 con 0x7f5d60071580 2023-11-28T11:07:55.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f5d580019f0 con 0x7f5d60071580 2023-11-28T11:07:55.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 >> v1:172.21.15.156:6789/0 conn(0x7f5d60071580 legacy=0x7f5d6015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d60154e70 0x7f5d48003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d48004030 0x7f5d48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48004d90 0x7f5d48005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d60155240 con 0x7f5d48004d90 2023-11-28T11:07:55.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6015b100 con 0x7f5d48004030 2023-11-28T11:07:55.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4800bba0 con 0x7f5d60154e70 2023-11-28T11:07:55.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5ffff700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48004d90 0x7f5d48005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5effd700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d48004030 0x7f5d48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5f7fe700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d60154e70 0x7f5d48003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5ffff700 1 -- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d60154e70 msgr2=0x7f5d48003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:07:55.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5ffff700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d60154e70 0x7f5d48003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5ffff700 1 -- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d48004030 msgr2=0x7f5d48004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:07:55.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5ffff700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d48004030 0x7f5d48004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.000+0000 7f5d5ffff700 1 -- 172.21.15.156:0/2822903479 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d480035b0 con 0x7f5d48004d90 2023-11-28T11:07:55.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.001+0000 7f5d5ffff700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48004d90 0x7f5d48005200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800a830 tx=0x7f5d48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc78313d3c0d8a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:55.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.002+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58012600 con 0x7f5d48004d90 2023-11-28T11:07:55.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.002+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58012760 con 0x7f5d48004d90 2023-11-28T11:07:55.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.002+0000 7f5d5e7fc700 1 -- 172.21.15.156:0/2822903479 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58012b10 con 0x7f5d48004d90 2023-11-28T11:07:55.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.002+0000 7f5d65ffd700 1 -- 172.21.15.156:0/2822903479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48004d90 msgr2=0x7f5d48005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.002+0000 7f5d65ffd700 1 --2- 172.21.15.156:0/2822903479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48004d90 0x7f5d48005200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800a830 tx=0x7f5d48005880 comp rx=0 tx=0).stop 2023-11-28T11:07:55.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.002+0000 7f5d5ffff700 1 -- 172.21.15.156:0/2822903479 reap_dead start 2023-11-28T11:07:55.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.002+0000 7f5d65ffd700 1 -- 172.21.15.156:0/2822903479 shutdown_connections 2023-11-28T11:07:55.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.002+0000 7f5d65ffd700 1 -- 172.21.15.156:0/2822903479 >> 172.21.15.156:0/2822903479 conn(0x7f5d600d0000 msgr2=0x7f5d600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:55.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.002+0000 7f5d65ffd700 1 -- 172.21.15.156:0/2822903479 shutdown_connections 2023-11-28T11:07:55.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.003+0000 7f5d65ffd700 1 -- 172.21.15.156:0/2822903479 wait complete. 2023-11-28T11:07:55.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.004+0000 7f5d65ffd700 1 Processor -- start 2023-11-28T11:07:55.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.005+0000 7f5d65ffd700 1 -- start start 2023-11-28T11:07:55.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.005+0000 7f5d65ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d48004d90 0x7f5d600c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.005+0000 7f5d65ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d600b2e30 0x7f5d600b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.005+0000 7f5d65ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d601540b0 0x7f5d60154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d65ffd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d60154a60 con 0x7f5d600b2e30 2023-11-28T11:07:55.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d65ffd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600c4a80 con 0x7f5d601540b0 2023-11-28T11:07:55.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d65ffd700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600c4d50 con 0x7f5d48004d90 2023-11-28T11:07:55.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d5effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d600b2e30 0x7f5d600b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d5effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d600b2e30 0x7f5d600b5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59396/0 (socket says 172.21.15.156:59396) 2023-11-28T11:07:55.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d5effd700 1 -- 172.21.15.156:0/947459871 learned_addr learned my addr 172.21.15.156:0/947459871 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:07:55.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d5f7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d48004d90 0x7f5d600c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d5ffff700 1 --2- 172.21.15.156:0/947459871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d601540b0 0x7f5d60154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d5effd700 1 -- 172.21.15.156:0/947459871 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d48004d90 msgr2=0x7f5d600c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d5effd700 1 --2- 172.21.15.156:0/947459871 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d48004d90 0x7f5d600c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d5effd700 1 -- 172.21.15.156:0/947459871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d601540b0 msgr2=0x7f5d60154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d5effd700 1 --2- 172.21.15.156:0/947459871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d601540b0 0x7f5d60154520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d5effd700 1 -- 172.21.15.156:0/947459871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6015b9a0 con 0x7f5d600b2e30 2023-11-28T11:07:55.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d5effd700 1 --2- 172.21.15.156:0/947459871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d600b2e30 0x7f5d600b5300 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5d54010920 tx=0x7f5d54010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:55.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d47fff700 1 -- 172.21.15.156:0/947459871 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400e8e0 con 0x7f5d600b2e30 2023-11-28T11:07:55.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d47fff700 1 -- 172.21.15.156:0/947459871 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5400ea40 con 0x7f5d600b2e30 2023-11-28T11:07:55.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d65ffd700 1 -- 172.21.15.156:0/947459871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6015b6c0 con 0x7f5d600b2e30 2023-11-28T11:07:55.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d47fff700 1 -- 172.21.15.156:0/947459871 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400edd0 con 0x7f5d600b2e30 2023-11-28T11:07:55.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d65ffd700 1 -- 172.21.15.156:0/947459871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d600af520 con 0x7f5d600b2e30 2023-11-28T11:07:55.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.006+0000 7f5d47fff700 1 -- 172.21.15.156:0/947459871 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f5d54026070 con 0x7f5d600b2e30 2023-11-28T11:07:55.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.007+0000 7f5d47fff700 1 -- 172.21.15.156:0/947459871 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5d54016420 con 0x7f5d600b2e30 2023-11-28T11:07:55.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.007+0000 7f5d65ffd700 1 -- 172.21.15.156:0/947459871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d600d0c60 con 0x7f5d600b2e30 2023-11-28T11:07:55.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.010+0000 7f5d47fff700 1 -- 172.21.15.156:0/947459871 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f5d5403eb00 con 0x7f5d600b2e30 2023-11-28T11:07:55.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.099+0000 7f5d65ffd700 1 -- 172.21.15.156:0/947459871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5d60072260 con 0x7f5d600b2e30 2023-11-28T11:07:55.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.102+0000 7f5d47fff700 1 -- 172.21.15.156:0/947459871 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5d5402d020 con 0x7f5d600b2e30 2023-11-28T11:07:55.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.102+0000 7f5d65ffd700 1 -- 172.21.15.156:0/947459871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d600b2e30 msgr2=0x7f5d600b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.102+0000 7f5d65ffd700 1 --2- 172.21.15.156:0/947459871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d600b2e30 0x7f5d600b5300 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5d54010920 tx=0x7f5d54010c30 comp rx=0 tx=0).stop 2023-11-28T11:07:55.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.103+0000 7f5d65ffd700 1 -- 172.21.15.156:0/947459871 shutdown_connections 2023-11-28T11:07:55.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.103+0000 7f5d65ffd700 1 --2- 172.21.15.156:0/947459871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d601540b0 0x7f5d60154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.103+0000 7f5d65ffd700 1 --2- 172.21.15.156:0/947459871 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d48004d90 0x7f5d600c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.103+0000 7f5d65ffd700 1 --2- 172.21.15.156:0/947459871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d600b2e30 0x7f5d600b5300 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.103+0000 7f5d65ffd700 1 -- 172.21.15.156:0/947459871 >> 172.21.15.156:0/947459871 conn(0x7f5d600d0000 msgr2=0x7f5d600b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:55.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.103+0000 7f5d65ffd700 1 -- 172.21.15.156:0/947459871 shutdown_connections 2023-11-28T11:07:55.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:55.103+0000 7f5d65ffd700 1 -- 172.21.15.156:0/947459871 wait complete. 2023-11-28T11:07:55.107 INFO:teuthology.orchestra.run.smithi156.stderr:adjusted tunables profile to default 2023-11-28T11:07:55.116 INFO:tasks.ceph:check_enable_crimson: False 2023-11-28T11:07:55.116 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T11:07:55.116 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:55.116 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T11:07:55.117 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:55.116+0000 7f075a483200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T11:07:55.131 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:55.129+0000 7f444597f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T11:07:55.142 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:55.143 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T11:07:55.183 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:55.181+0000 7f075a483200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T11:07:55.198 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:55.197+0000 7f444597f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T11:07:55.209 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:55.210 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T11:07:55.259 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:55.257+0000 7f075a483200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T11:07:55.276 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:07:55.276 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T11:07:55.277 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:55.276+0000 7f444597f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T11:07:55.303 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:55.303 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T11:07:55.333 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:55.333 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T11:07:55.343 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:55.342+0000 7f075a483200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T11:07:55.361 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:55.361+0000 7f444597f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T11:07:55.371 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:55.371 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T11:07:55.439 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:07:55.440 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T11:07:55.498 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:55.497+0000 7f075a483200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T11:07:55.503 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:55.502+0000 7f444597f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T11:07:55.506 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 0bd69671-96c1-4bb2-a04c-644fa7e2ba5c 0 2023-11-28T11:07:55.670 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.667+0000 7f56dc687700 1 Processor -- start 2023-11-28T11:07:55.670 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.667+0000 7f56dc687700 1 -- start start 2023-11-28T11:07:55.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.668+0000 7f56dc687700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f56d4071040 0x7f56d4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.668+0000 7f56dc687700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f56d406dee0 0x7f56d406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.668+0000 7f56dc687700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d4071950 con 0x7f56d406cdc0 2023-11-28T11:07:55.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.668+0000 7f56dc687700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d4161550 con 0x7f56d4071040 2023-11-28T11:07:55.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.668+0000 7f56dc687700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d415d060 con 0x7f56d406dee0 2023-11-28T11:07:55.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56d9c22700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f56d406dee0 0x7f56d406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.672 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56d9c22700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f56d406dee0 0x7f56d406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47428/0 (socket says 172.21.15.186:47428) 2023-11-28T11:07:55.672 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56d9c22700 1 -- 172.21.15.186:0/902176481 learned_addr learned my addr 172.21.15.186:0/902176481 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:55.672 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56dac24700 1 --1- 172.21.15.186:0/902176481 >> v1:172.21.15.186:6789/0 conn(0x7f56d406cdc0 0x7f56d406d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:38274/0 (socket says 172.21.15.186:38274) 2023-11-28T11:07:55.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56da423700 1 --2- 172.21.15.186:0/902176481 >> v2:172.21.15.156:3300/0 conn(0x7f56d4071040 0x7f56d4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56d9c22700 1 -- 172.21.15.186:0/902176481 >> v2:172.21.15.156:3300/0 conn(0x7f56d4071040 msgr2=0x7f56d4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:07:55.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56d9c22700 1 --2- 172.21.15.186:0/902176481 >> v2:172.21.15.156:3300/0 conn(0x7f56d4071040 0x7f56d4071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56d9c22700 1 -- 172.21.15.186:0/902176481 >> v1:172.21.15.186:6789/0 conn(0x7f56d406cdc0 legacy=0x7f56d406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56d9c22700 1 -- 172.21.15.186:0/902176481 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d415d340 con 0x7f56d406dee0 2023-11-28T11:07:55.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56d9c22700 1 --2- 172.21.15.186:0/902176481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d406dee0 0x7f56d406e330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56d406fb00 tx=0x7f56c4009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10357ea28f4414c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:55.676 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56d9421700 1 -- 172.21.15.186:0/902176481 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4011830 con 0x7f56d406dee0 2023-11-28T11:07:55.676 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56d9421700 1 -- 172.21.15.186:0/902176481 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4011990 con 0x7f56d406dee0 2023-11-28T11:07:55.676 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.669+0000 7f56d9421700 1 -- 172.21.15.186:0/902176481 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4011d20 con 0x7f56d406dee0 2023-11-28T11:07:55.676 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.670+0000 7f56dc687700 1 -- 172.21.15.186:0/902176481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d406dee0 msgr2=0x7f56d406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.676 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.670+0000 7f56dc687700 1 --2- 172.21.15.186:0/902176481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d406dee0 0x7f56d406e330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56d406fb00 tx=0x7f56c4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.676 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.670+0000 7f56dc687700 1 -- 172.21.15.186:0/902176481 shutdown_connections 2023-11-28T11:07:55.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.670+0000 7f56dc687700 1 --2- 172.21.15.186:0/902176481 >> v2:172.21.15.156:3300/0 conn(0x7f56d4071040 0x7f56d4071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.670+0000 7f56dc687700 1 --2- 172.21.15.186:0/902176481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d406dee0 0x7f56d406e330 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.670+0000 7f56dc687700 1 -- 172.21.15.186:0/902176481 >> 172.21.15.186:0/902176481 conn(0x7f56d40d0000 msgr2=0x7f56d40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:55.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.670+0000 7f56dc687700 1 -- 172.21.15.186:0/902176481 shutdown_connections 2023-11-28T11:07:55.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.670+0000 7f56dc687700 1 -- 172.21.15.186:0/902176481 wait complete. 2023-11-28T11:07:55.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.671+0000 7f56dc687700 1 Processor -- start 2023-11-28T11:07:55.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.671+0000 7f56dc687700 1 -- start start 2023-11-28T11:07:55.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56dc687700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d406cdc0 0x7f56d414c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56dc687700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56d406dee0 0x7f56d40c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56dc687700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56d4071040 0x7f56d406d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56dc687700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d41f28c0 con 0x7f56d4071040 2023-11-28T11:07:55.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56dc687700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d414c870 con 0x7f56d406cdc0 2023-11-28T11:07:55.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56dc687700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d40c6610 con 0x7f56d406dee0 2023-11-28T11:07:55.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56d9c22700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56d406dee0 0x7f56d40c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56da423700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d406cdc0 0x7f56d414c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56d9c22700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56d406dee0 0x7f56d40c3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:60990/0 (socket says 172.21.15.186:60990) 2023-11-28T11:07:55.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56d9c22700 1 -- 172.21.15.186:0/2724526346 learned_addr learned my addr 172.21.15.186:0/2724526346 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:55.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56dac24700 1 --2- 172.21.15.186:0/2724526346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56d4071040 0x7f56d406d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56d9c22700 1 -- 172.21.15.186:0/2724526346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d406cdc0 msgr2=0x7f56d414c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56d9c22700 1 --2- 172.21.15.186:0/2724526346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d406cdc0 0x7f56d414c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56d9c22700 1 -- 172.21.15.186:0/2724526346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56d4071040 msgr2=0x7f56d406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56d9c22700 1 --2- 172.21.15.186:0/2724526346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56d4071040 0x7f56d406d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56d9c22700 1 -- 172.21.15.186:0/2724526346 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c4009550 con 0x7f56d406dee0 2023-11-28T11:07:55.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56da423700 1 --2- 172.21.15.186:0/2724526346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d406cdc0 0x7f56d414c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:55.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.672+0000 7f56d9c22700 1 --2- 172.21.15.186:0/2724526346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56d406dee0 0x7f56d40c3aa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009c10 tx=0x7f56c4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:55.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.673+0000 7f56caffd700 1 -- 172.21.15.186:0/2724526346 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4004450 con 0x7f56d406dee0 2023-11-28T11:07:55.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.673+0000 7f56caffd700 1 -- 172.21.15.186:0/2724526346 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c401e070 con 0x7f56d406dee0 2023-11-28T11:07:55.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.673+0000 7f56caffd700 1 -- 172.21.15.186:0/2724526346 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c40038e0 con 0x7f56d406dee0 2023-11-28T11:07:55.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.673+0000 7f56dc687700 1 -- 172.21.15.186:0/2724526346 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d415d900 con 0x7f56d406dee0 2023-11-28T11:07:55.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.673+0000 7f56dc687700 1 -- 172.21.15.186:0/2724526346 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d415d620 con 0x7f56d406dee0 2023-11-28T11:07:55.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.673+0000 7f56caffd700 1 -- 172.21.15.186:0/2724526346 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f56c400f050 con 0x7f56d406dee0 2023-11-28T11:07:55.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.673+0000 7f56caffd700 1 -- 172.21.15.186:0/2724526346 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f56c402d630 con 0x7f56d406dee0 2023-11-28T11:07:55.680 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.674+0000 7f56dc687700 1 -- 172.21.15.186:0/2724526346 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d406e870 con 0x7f56d406dee0 2023-11-28T11:07:55.680 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.676+0000 7f56caffd700 1 -- 172.21.15.186:0/2724526346 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f56c4020070 con 0x7f56d406dee0 2023-11-28T11:07:55.762 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.760+0000 7f56dc687700 1 -- 172.21.15.186:0/2724526346 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0bd69671-96c1-4bb2-a04c-644fa7e2ba5c", "id": 0} v 0) v1 -- 0x7f56d40720a0 con 0x7f56d406dee0 2023-11-28T11:07:55.767 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.763+0000 7f56caffd700 1 -- 172.21.15.186:0/2724526346 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0bd69671-96c1-4bb2-a04c-644fa7e2ba5c", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f56c4015020 con 0x7f56d406dee0 2023-11-28T11:07:55.767 INFO:teuthology.orchestra.run.smithi186.stdout:0 2023-11-28T11:07:55.768 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.765+0000 7f56dc687700 1 -- 172.21.15.186:0/2724526346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56d406dee0 msgr2=0x7f56d40c3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.768 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.765+0000 7f56dc687700 1 --2- 172.21.15.186:0/2724526346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56d406dee0 0x7f56d40c3aa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009c10 tx=0x7f56c4004750 comp rx=0 tx=0).stop 2023-11-28T11:07:55.768 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.765+0000 7f56dc687700 1 -- 172.21.15.186:0/2724526346 shutdown_connections 2023-11-28T11:07:55.768 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.765+0000 7f56dc687700 1 --2- 172.21.15.186:0/2724526346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56d4071040 0x7f56d406d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.768 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.765+0000 7f56dc687700 1 --2- 172.21.15.186:0/2724526346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d406cdc0 0x7f56d414c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.768 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.765+0000 7f56dc687700 1 --2- 172.21.15.186:0/2724526346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56d406dee0 0x7f56d40c3aa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.769 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.765+0000 7f56dc687700 1 -- 172.21.15.186:0/2724526346 >> 172.21.15.186:0/2724526346 conn(0x7f56d40d0000 msgr2=0x7f56d406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:55.769 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.765+0000 7f56dc687700 1 -- 172.21.15.186:0/2724526346 shutdown_connections 2023-11-28T11:07:55.769 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.765+0000 7f56dc687700 1 -- 172.21.15.186:0/2724526346 wait complete. 2023-11-28T11:07:55.783 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new fa3e6ec6-caad-43ac-b667-9fe984bbf201 1 2023-11-28T11:07:55.907 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.905+0000 7f1cbd9c1700 1 Processor -- start 2023-11-28T11:07:55.907 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.906+0000 7f1cbd9c1700 1 -- start start 2023-11-28T11:07:55.909 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.906+0000 7f1cbd9c1700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1cb80cd8c0 0x7f1cb8140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.909 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.906+0000 7f1cbd9c1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1cb80cdc90 0x7f1cb80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.909 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.906+0000 7f1cbd9c1700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb813ff00 con 0x7f1cb80b2e70 2023-11-28T11:07:55.909 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.906+0000 7f1cbd9c1700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb81404c0 con 0x7f1cb80cdc90 2023-11-28T11:07:55.909 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.906+0000 7f1cbd9c1700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb815aa80 con 0x7f1cb80cd8c0 2023-11-28T11:07:55.909 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.906+0000 7f1cb6ffd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1cb80cd8c0 0x7f1cb8140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.909 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.906+0000 7f1cb6ffd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1cb80cd8c0 0x7f1cb8140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47438/0 (socket says 172.21.15.186:47438) 2023-11-28T11:07:55.910 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.907+0000 7f1cb6ffd700 1 -- 172.21.15.186:0/2367466120 learned_addr learned my addr 172.21.15.186:0/2367466120 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:55.910 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.907+0000 7f1cb67fc700 1 --2- 172.21.15.186:0/2367466120 >> v2:172.21.15.156:3300/0 conn(0x7f1cb80cdc90 0x7f1cb80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.910 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.907+0000 7f1cb6ffd700 1 -- 172.21.15.186:0/2367466120 >> v2:172.21.15.156:3300/0 conn(0x7f1cb80cdc90 msgr2=0x7f1cb80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:07:55.910 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.907+0000 7f1cb6ffd700 1 --2- 172.21.15.186:0/2367466120 >> v2:172.21.15.156:3300/0 conn(0x7f1cb80cdc90 0x7f1cb80b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.910 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.907+0000 7f1cb6ffd700 1 -- 172.21.15.186:0/2367466120 >> v1:172.21.15.156:6789/0 conn(0x7f1cb80b2e70 legacy=0x7f1cb806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.910 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.907+0000 7f1cb6ffd700 1 -- 172.21.15.186:0/2367466120 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb80b3310 con 0x7f1cb80cd8c0 2023-11-28T11:07:55.910 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.907+0000 7f1cb6ffd700 1 --2- 172.21.15.186:0/2367466120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb80cd8c0 0x7f1cb8140950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8070a20 tx=0x7f1ca8009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf5fd43abb872bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:55.910 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.907+0000 7f1cb5ffb700 1 -- 172.21.15.186:0/2367466120 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8005e30 con 0x7f1cb80cd8c0 2023-11-28T11:07:55.910 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.907+0000 7f1cb5ffb700 1 -- 172.21.15.186:0/2367466120 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca8004030 con 0x7f1cb80cd8c0 2023-11-28T11:07:55.910 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.907+0000 7f1cb5ffb700 1 -- 172.21.15.186:0/2367466120 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca80043e0 con 0x7f1cb80cd8c0 2023-11-28T11:07:55.911 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.908+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/2367466120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb80cd8c0 msgr2=0x7f1cb8140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.911 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.908+0000 7f1cbd9c1700 1 --2- 172.21.15.186:0/2367466120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb80cd8c0 0x7f1cb8140950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8070a20 tx=0x7f1ca8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.908+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/2367466120 shutdown_connections 2023-11-28T11:07:55.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.908+0000 7f1cbd9c1700 1 --2- 172.21.15.186:0/2367466120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb80cd8c0 0x7f1cb8140950 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.908+0000 7f1cbd9c1700 1 --2- 172.21.15.186:0/2367466120 >> v2:172.21.15.156:3300/0 conn(0x7f1cb80cdc90 0x7f1cb80b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.908+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/2367466120 >> 172.21.15.186:0/2367466120 conn(0x7f1cb806aed0 msgr2=0x7f1cb806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:55.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.908+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/2367466120 shutdown_connections 2023-11-28T11:07:55.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.908+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/2367466120 wait complete. 2023-11-28T11:07:55.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.909+0000 7f1cbd9c1700 1 Processor -- start 2023-11-28T11:07:55.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.910+0000 7f1cbd9c1700 1 -- start start 2023-11-28T11:07:55.914 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.910+0000 7f1cbd9c1700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1cb80b2e70 0x7f1cb80b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.914 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.910+0000 7f1cbd9c1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb80cd8c0 0x7f1cb80b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.914 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.910+0000 7f1cbd9c1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb80cdc90 0x7f1cb80b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:55.914 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.910+0000 7f1cbd9c1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb8158810 con 0x7f1cb80cd8c0 2023-11-28T11:07:55.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.910+0000 7f1cbd9c1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb80c5ee0 con 0x7f1cb80cdc90 2023-11-28T11:07:55.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.910+0000 7f1cbd9c1700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb80b3d00 con 0x7f1cb80b2e70 2023-11-28T11:07:55.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cb6ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1cb80b2e70 0x7f1cb80b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cb6ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1cb80b2e70 0x7f1cb80b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:32768/0 (socket says 172.21.15.186:32768) 2023-11-28T11:07:55.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cb6ffd700 1 -- 172.21.15.186:0/164855135 learned_addr learned my addr 172.21.15.186:0/164855135 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:55.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cb77fe700 1 --2- 172.21.15.186:0/164855135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb80cdc90 0x7f1cb80b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cb67fc700 1 --2- 172.21.15.186:0/164855135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb80cd8c0 0x7f1cb80b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:55.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cb6ffd700 1 -- 172.21.15.186:0/164855135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb80cdc90 msgr2=0x7f1cb80b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cb6ffd700 1 --2- 172.21.15.186:0/164855135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb80cdc90 0x7f1cb80b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cb6ffd700 1 -- 172.21.15.186:0/164855135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb80cd8c0 msgr2=0x7f1cb80b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:55.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cb6ffd700 1 --2- 172.21.15.186:0/164855135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb80cd8c0 0x7f1cb80b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:55.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cb6ffd700 1 -- 172.21.15.186:0/164855135 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8009500 con 0x7f1cb80b2e70 2023-11-28T11:07:55.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cb6ffd700 1 --2- 172.21.15.186:0/164855135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1cb80b2e70 0x7f1cb80b5550 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8004000 tx=0x7f1ca8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:55.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1c9ffff700 1 -- 172.21.15.186:0/164855135 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8003cb0 con 0x7f1cb80b2e70 2023-11-28T11:07:55.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1c9ffff700 1 -- 172.21.15.186:0/164855135 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca8003e10 con 0x7f1cb80b2e70 2023-11-28T11:07:55.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/164855135 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb8140150 con 0x7f1cb80b2e70 2023-11-28T11:07:55.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1c9ffff700 1 -- 172.21.15.186:0/164855135 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca801d670 con 0x7f1cb80b2e70 2023-11-28T11:07:55.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/164855135 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb80b35f0 con 0x7f1cb80b2e70 2023-11-28T11:07:55.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1c9ffff700 1 -- 172.21.15.186:0/164855135 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f1ca802d070 con 0x7f1cb80b2e70 2023-11-28T11:07:55.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.911+0000 7f1c9ffff700 1 -- 172.21.15.186:0/164855135 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f1ca8026a70 con 0x7f1cb80b2e70 2023-11-28T11:07:55.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.912+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/164855135 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb80ce100 con 0x7f1cb80b2e70 2023-11-28T11:07:55.917 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:55.916+0000 7f1c9ffff700 1 -- 172.21.15.186:0/164855135 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f1ca8045b00 con 0x7f1cb80b2e70 2023-11-28T11:07:56.010 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.007+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/164855135 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fa3e6ec6-caad-43ac-b667-9fe984bbf201", "id": 1} v 0) v1 -- 0x7f1cb8068690 con 0x7f1cb80b2e70 2023-11-28T11:07:56.013 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.009+0000 7f1c9ffff700 1 -- 172.21.15.186:0/164855135 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fa3e6ec6-caad-43ac-b667-9fe984bbf201", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1ca8034030 con 0x7f1cb80b2e70 2023-11-28T11:07:56.013 INFO:teuthology.orchestra.run.smithi186.stdout:1 2023-11-28T11:07:56.016 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.010+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/164855135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1cb80b2e70 msgr2=0x7f1cb80b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.016 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.010+0000 7f1cbd9c1700 1 --2- 172.21.15.186:0/164855135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1cb80b2e70 0x7f1cb80b5550 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8004000 tx=0x7f1ca8004750 comp rx=0 tx=0).stop 2023-11-28T11:07:56.017 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.010+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/164855135 shutdown_connections 2023-11-28T11:07:56.017 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.010+0000 7f1cbd9c1700 1 --2- 172.21.15.186:0/164855135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb80cdc90 0x7f1cb80b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.017 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.010+0000 7f1cbd9c1700 1 --2- 172.21.15.186:0/164855135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1cb80b2e70 0x7f1cb80b5550 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.017 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.010+0000 7f1cbd9c1700 1 --2- 172.21.15.186:0/164855135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb80cd8c0 0x7f1cb80b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.017 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.010+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/164855135 >> 172.21.15.186:0/164855135 conn(0x7f1cb806aed0 msgr2=0x7f1cb813ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:56.017 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.011+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/164855135 shutdown_connections 2023-11-28T11:07:56.017 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.011+0000 7f1cbd9c1700 1 -- 172.21.15.186:0/164855135 wait complete. 2023-11-28T11:07:56.024 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new c6cb5b07-8272-46e0-8551-afb3c654c34e 2 2023-11-28T11:07:56.148 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.146+0000 7f1dd5cda700 1 Processor -- start 2023-11-28T11:07:56.148 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.146+0000 7f1dd5cda700 1 -- start start 2023-11-28T11:07:56.156 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.147+0000 7f1dd5cda700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00ae9d0 0x7f1dd015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.156 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.147+0000 7f1dd5cda700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1dd006a5e0 0x7f1dd006aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.156 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.147+0000 7f1dd5cda700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd00aee40 con 0x7f1dd006e500 2023-11-28T11:07:56.156 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.147+0000 7f1dd5cda700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd015bf10 con 0x7f1dd006a5e0 2023-11-28T11:07:56.156 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.147+0000 7f1dd5cda700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd006c930 con 0x7f1dd00ae9d0 2023-11-28T11:07:56.156 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.147+0000 7f1dceffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00ae9d0 0x7f1dd015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.156 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.147+0000 7f1dcf7fe700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f1dd006e500 0x7f1dd006e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:38294/0 (socket says 172.21.15.186:38294) 2023-11-28T11:07:56.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.147+0000 7f1dcf7fe700 1 -- 172.21.15.186:0/4098767546 learned_addr learned my addr 172.21.15.186:0/4098767546 (peer_addr_for_me v1:172.21.15.186:0/0) 2023-11-28T11:07:56.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.147+0000 7f1dcffff700 1 --2- 172.21.15.186:0/4098767546 >> v2:172.21.15.156:3300/0 conn(0x7f1dd006a5e0 0x7f1dd006aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.148+0000 7f1dce7fc700 1 -- 172.21.15.186:0/4098767546 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924775618 0 0) 0x7f1dd00aee40 con 0x7f1dd006e500 2023-11-28T11:07:56.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.148+0000 7f1dceffd700 1 -- 172.21.15.186:0/4098767546 >> v2:172.21.15.156:3300/0 conn(0x7f1dd006a5e0 msgr2=0x7f1dd006aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:07:56.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.148+0000 7f1dceffd700 1 --2- 172.21.15.186:0/4098767546 >> v2:172.21.15.156:3300/0 conn(0x7f1dd006a5e0 0x7f1dd006aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.148+0000 7f1dceffd700 1 -- 172.21.15.186:0/4098767546 >> v1:172.21.15.186:6789/0 conn(0x7f1dd006e500 legacy=0x7f1dd006e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.148+0000 7f1dceffd700 1 -- 172.21.15.186:0/4098767546 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd006cc10 con 0x7f1dd00ae9d0 2023-11-28T11:07:56.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.148+0000 7f1dceffd700 1 --2- 172.21.15.186:0/4098767546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00ae9d0 0x7f1dd015b9d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009840 tx=0x7f1dc4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f21a3cbee8af16af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:56.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.148+0000 7f1dce7fc700 1 -- 172.21.15.186:0/4098767546 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4011830 con 0x7f1dd00ae9d0 2023-11-28T11:07:56.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.148+0000 7f1dce7fc700 1 -- 172.21.15.186:0/4098767546 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc4011990 con 0x7f1dd00ae9d0 2023-11-28T11:07:56.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.148+0000 7f1dce7fc700 1 -- 172.21.15.186:0/4098767546 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4011d20 con 0x7f1dd00ae9d0 2023-11-28T11:07:56.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.148+0000 7f1dd5cda700 1 -- 172.21.15.186:0/4098767546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00ae9d0 msgr2=0x7f1dd015b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.148+0000 7f1dd5cda700 1 --2- 172.21.15.186:0/4098767546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00ae9d0 0x7f1dd015b9d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009840 tx=0x7f1dc4009b50 comp rx=0 tx=0).stop 2023-11-28T11:07:56.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.149+0000 7f1dd5cda700 1 -- 172.21.15.186:0/4098767546 shutdown_connections 2023-11-28T11:07:56.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.149+0000 7f1dd5cda700 1 --2- 172.21.15.186:0/4098767546 >> v2:172.21.15.156:3300/0 conn(0x7f1dd006a5e0 0x7f1dd006aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.149+0000 7f1dd5cda700 1 --2- 172.21.15.186:0/4098767546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00ae9d0 0x7f1dd015b9d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.149+0000 7f1dd5cda700 1 -- 172.21.15.186:0/4098767546 >> 172.21.15.186:0/4098767546 conn(0x7f1dd00b12b0 msgr2=0x7f1dd00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:56.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.149+0000 7f1dd5cda700 1 -- 172.21.15.186:0/4098767546 shutdown_connections 2023-11-28T11:07:56.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.149+0000 7f1dd5cda700 1 -- 172.21.15.186:0/4098767546 wait complete. 2023-11-28T11:07:56.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.150+0000 7f1dd5cda700 1 Processor -- start 2023-11-28T11:07:56.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.150+0000 7f1dd5cda700 1 -- start start 2023-11-28T11:07:56.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dd5cda700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd006a5e0 0x7f1dd00cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dd5cda700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dd006e500 0x7f1dd00ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dd5cda700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00ae9d0 0x7f1dd01576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dd5cda700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00ace00 con 0x7f1dd006e500 2023-11-28T11:07:56.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dd5cda700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00ad0a0 con 0x7f1dd00ae9d0 2023-11-28T11:07:56.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dd5cda700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00ad380 con 0x7f1dd006a5e0 2023-11-28T11:07:56.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dcf7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd006a5e0 0x7f1dd00cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dcffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00ae9d0 0x7f1dd01576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dcf7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd006a5e0 0x7f1dd00cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:32780/0 (socket says 172.21.15.186:32780) 2023-11-28T11:07:56.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dcf7fe700 1 -- 172.21.15.186:0/2524269081 learned_addr learned my addr 172.21.15.186:0/2524269081 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:56.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dceffd700 1 --2- 172.21.15.186:0/2524269081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dd006e500 0x7f1dd00ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dcf7fe700 1 -- 172.21.15.186:0/2524269081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00ae9d0 msgr2=0x7f1dd01576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dcf7fe700 1 --2- 172.21.15.186:0/2524269081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00ae9d0 0x7f1dd01576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dcf7fe700 1 -- 172.21.15.186:0/2524269081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dd006e500 msgr2=0x7f1dd00ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dcf7fe700 1 --2- 172.21.15.186:0/2524269081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dd006e500 0x7f1dd00ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dcf7fe700 1 -- 172.21.15.186:0/2524269081 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc40094a0 con 0x7f1dd006a5e0 2023-11-28T11:07:56.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dcffff700 1 --2- 172.21.15.186:0/2524269081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00ae9d0 0x7f1dd01576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:56.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1dcf7fe700 1 --2- 172.21.15.186:0/2524269081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd006a5e0 0x7f1dd00cef80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00011e0 tx=0x7f1dc000ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:56.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1db7fff700 1 -- 172.21.15.186:0/2524269081 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0004c80 con 0x7f1dd006a5e0 2023-11-28T11:07:56.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.151+0000 7f1db7fff700 1 -- 172.21.15.186:0/2524269081 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc0004e00 con 0x7f1dd006a5e0 2023-11-28T11:07:56.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.152+0000 7f1db7fff700 1 -- 172.21.15.186:0/2524269081 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0007310 con 0x7f1dd006a5e0 2023-11-28T11:07:56.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.152+0000 7f1dd5cda700 1 -- 172.21.15.186:0/2524269081 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd006cef0 con 0x7f1dd006a5e0 2023-11-28T11:07:56.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.152+0000 7f1dd5cda700 1 -- 172.21.15.186:0/2524269081 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd00664c0 con 0x7f1dd006a5e0 2023-11-28T11:07:56.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.152+0000 7f1db7fff700 1 -- 172.21.15.186:0/2524269081 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f1dc001a020 con 0x7f1dd006a5e0 2023-11-28T11:07:56.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.152+0000 7f1db7fff700 1 -- 172.21.15.186:0/2524269081 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f1dc00137e0 con 0x7f1dd006a5e0 2023-11-28T11:07:56.163 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.152+0000 7f1dd5cda700 1 -- 172.21.15.186:0/2524269081 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd015ac80 con 0x7f1dd006a5e0 2023-11-28T11:07:56.163 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.154+0000 7f1db7fff700 1 -- 172.21.15.186:0/2524269081 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f1dc001d4a0 con 0x7f1dd006a5e0 2023-11-28T11:07:56.181 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:56.180+0000 7f444597f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T11:07:56.237 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.236+0000 7f1dd5cda700 1 -- 172.21.15.186:0/2524269081 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c6cb5b07-8272-46e0-8551-afb3c654c34e", "id": 2} v 0) v1 -- 0x7f1dd00ae510 con 0x7f1dd006a5e0 2023-11-28T11:07:56.240 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.239+0000 7f1db7fff700 1 -- 172.21.15.186:0/2524269081 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c6cb5b07-8272-46e0-8551-afb3c654c34e", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1dc0007540 con 0x7f1dd006a5e0 2023-11-28T11:07:56.241 INFO:teuthology.orchestra.run.smithi186.stdout:2 2023-11-28T11:07:56.242 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.240+0000 7f1dd5cda700 1 -- 172.21.15.186:0/2524269081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd006a5e0 msgr2=0x7f1dd00cef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.242 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.240+0000 7f1dd5cda700 1 --2- 172.21.15.186:0/2524269081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd006a5e0 0x7f1dd00cef80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00011e0 tx=0x7f1dc000ca30 comp rx=0 tx=0).stop 2023-11-28T11:07:56.242 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.240+0000 7f1dd5cda700 1 -- 172.21.15.186:0/2524269081 shutdown_connections 2023-11-28T11:07:56.242 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.240+0000 7f1dd5cda700 1 --2- 172.21.15.186:0/2524269081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00ae9d0 0x7f1dd01576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.242 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.240+0000 7f1dd5cda700 1 --2- 172.21.15.186:0/2524269081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd006a5e0 0x7f1dd00cef80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.242 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.240+0000 7f1dd5cda700 1 --2- 172.21.15.186:0/2524269081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dd006e500 0x7f1dd00ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.242 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.240+0000 7f1dd5cda700 1 -- 172.21.15.186:0/2524269081 >> 172.21.15.186:0/2524269081 conn(0x7f1dd00b12b0 msgr2=0x7f1dd0159e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:56.243 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.240+0000 7f1dd5cda700 1 -- 172.21.15.186:0/2524269081 shutdown_connections 2023-11-28T11:07:56.243 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.240+0000 7f1dd5cda700 1 -- 172.21.15.186:0/2524269081 wait complete. 2023-11-28T11:07:56.243 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:56.242+0000 7f075a483200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T11:07:56.253 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 769bf956-8498-49a7-8945-8cc91bcee69e 3 2023-11-28T11:07:56.256 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:56.255+0000 7f444597f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T11:07:56.325 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:56.323+0000 7f075a483200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T11:07:56.348 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:56.347+0000 7f444597f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T11:07:56.377 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.375+0000 7f927b207700 1 Processor -- start 2023-11-28T11:07:56.377 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.375+0000 7f927b207700 1 -- start start 2023-11-28T11:07:56.379 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f927b207700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f927406dee0 0x7f927406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.379 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f927b207700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9274160540 0x7f9274160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.379 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f927b207700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927415d060 con 0x7f9274071040 2023-11-28T11:07:56.379 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f927b207700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927415d340 con 0x7f927406dee0 2023-11-28T11:07:56.379 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f927b207700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927415d620 con 0x7f9274160540 2023-11-28T11:07:56.379 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f92797a4700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9274160540 0x7f9274160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.379 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f92797a4700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9274160540 0x7f9274160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47466/0 (socket says 172.21.15.186:47466) 2023-11-28T11:07:56.380 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f92797a4700 1 -- 172.21.15.186:0/3143932333 learned_addr learned my addr 172.21.15.186:0/3143932333 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:56.380 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f9273fff700 1 --2- 172.21.15.186:0/3143932333 >> v2:172.21.15.156:3300/0 conn(0x7f927406dee0 0x7f927406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.380 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f92797a4700 1 -- 172.21.15.186:0/3143932333 >> v2:172.21.15.156:3300/0 conn(0x7f927406dee0 msgr2=0x7f927406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:07:56.380 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f92797a4700 1 --2- 172.21.15.186:0/3143932333 >> v2:172.21.15.156:3300/0 conn(0x7f927406dee0 0x7f927406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.380 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f92797a4700 1 -- 172.21.15.186:0/3143932333 >> v1:172.21.15.156:6789/0 conn(0x7f9274071040 legacy=0x7f9274071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.380 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.376+0000 7f92797a4700 1 -- 172.21.15.186:0/3143932333 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927415d900 con 0x7f9274160540 2023-11-28T11:07:56.380 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.377+0000 7f92797a4700 1 --2- 172.21.15.186:0/3143932333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9274160540 0x7f9274160980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f92740c3a10 tx=0x7f926c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d603a073b9de99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:56.380 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.377+0000 7f92737fe700 1 -- 172.21.15.186:0/3143932333 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c00e070 con 0x7f9274160540 2023-11-28T11:07:56.380 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.377+0000 7f92737fe700 1 -- 172.21.15.186:0/3143932333 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926c00b910 con 0x7f9274160540 2023-11-28T11:07:56.380 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.377+0000 7f92737fe700 1 -- 172.21.15.186:0/3143932333 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c00bcc0 con 0x7f9274160540 2023-11-28T11:07:56.380 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.377+0000 7f927b207700 1 -- 172.21.15.186:0/3143932333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9274160540 msgr2=0x7f9274160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.380 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.377+0000 7f927b207700 1 --2- 172.21.15.186:0/3143932333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9274160540 0x7f9274160980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f92740c3a10 tx=0x7f926c00b330 comp rx=0 tx=0).stop 2023-11-28T11:07:56.381 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.377+0000 7f927b207700 1 -- 172.21.15.186:0/3143932333 shutdown_connections 2023-11-28T11:07:56.381 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.377+0000 7f927b207700 1 --2- 172.21.15.186:0/3143932333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9274160540 0x7f9274160980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.381 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.377+0000 7f927b207700 1 --2- 172.21.15.186:0/3143932333 >> v2:172.21.15.156:3300/0 conn(0x7f927406dee0 0x7f927406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.381 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.377+0000 7f927b207700 1 -- 172.21.15.186:0/3143932333 >> 172.21.15.186:0/3143932333 conn(0x7f92740d0000 msgr2=0x7f92740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:56.381 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.378+0000 7f927b207700 1 -- 172.21.15.186:0/3143932333 shutdown_connections 2023-11-28T11:07:56.381 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.378+0000 7f927b207700 1 -- 172.21.15.186:0/3143932333 wait complete. 2023-11-28T11:07:56.383 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.379+0000 7f927b207700 1 Processor -- start 2023-11-28T11:07:56.383 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.379+0000 7f927b207700 1 -- start start 2023-11-28T11:07:56.383 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f927b207700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f927406dee0 0x7f927415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.383 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f927b207700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9274071040 0x7f92741f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.384 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f927b207700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9274160540 0x7f92740b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.384 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f927b207700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92741f7010 con 0x7f9274071040 2023-11-28T11:07:56.384 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f927b207700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92741f6560 con 0x7f927406dee0 2023-11-28T11:07:56.384 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f927b207700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92740b4470 con 0x7f9274160540 2023-11-28T11:07:56.384 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f9278fa3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f927406dee0 0x7f927415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.384 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f9278fa3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f927406dee0 0x7f927415b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47468/0 (socket says 172.21.15.186:47468) 2023-11-28T11:07:56.384 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f92797a4700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9274160540 0x7f92740b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.384 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f9278fa3700 1 -- 172.21.15.186:0/4084976815 learned_addr learned my addr 172.21.15.186:0/4084976815 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:56.385 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f9273fff700 1 --2- 172.21.15.186:0/4084976815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9274071040 0x7f92741f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.385 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f9278fa3700 1 -- 172.21.15.186:0/4084976815 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9274160540 msgr2=0x7f92740b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.385 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f9278fa3700 1 --2- 172.21.15.186:0/4084976815 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9274160540 0x7f92740b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.385 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f9278fa3700 1 -- 172.21.15.186:0/4084976815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9274071040 msgr2=0x7f92741f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.385 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f9278fa3700 1 --2- 172.21.15.186:0/4084976815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9274071040 0x7f92741f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.385 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f9278fa3700 1 -- 172.21.15.186:0/4084976815 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c009b30 con 0x7f927406dee0 2023-11-28T11:07:56.386 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f92797a4700 1 --2- 172.21.15.186:0/4084976815 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9274160540 0x7f92740b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:56.386 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f9278fa3700 1 --2- 172.21.15.186:0/4084976815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f927406dee0 0x7f927415b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f926800c450 tx=0x7f926800c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:56.386 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f92717fa700 1 -- 172.21.15.186:0/4084976815 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92680052d0 con 0x7f927406dee0 2023-11-28T11:07:56.386 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f92717fa700 1 -- 172.21.15.186:0/4084976815 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9268005430 con 0x7f927406dee0 2023-11-28T11:07:56.386 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f927b207700 1 -- 172.21.15.186:0/4084976815 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927415ccd0 con 0x7f927406dee0 2023-11-28T11:07:56.386 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f92717fa700 1 -- 172.21.15.186:0/4084976815 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92680038c0 con 0x7f927406dee0 2023-11-28T11:07:56.386 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.380+0000 7f927b207700 1 -- 172.21.15.186:0/4084976815 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927406e870 con 0x7f927406dee0 2023-11-28T11:07:56.386 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.381+0000 7f92717fa700 1 -- 172.21.15.186:0/4084976815 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f9268019030 con 0x7f927406dee0 2023-11-28T11:07:56.386 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.381+0000 7f92717fa700 1 -- 172.21.15.186:0/4084976815 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f9268011bb0 con 0x7f927406dee0 2023-11-28T11:07:56.386 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.381+0000 7f927b207700 1 -- 172.21.15.186:0/4084976815 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927406a270 con 0x7f927406dee0 2023-11-28T11:07:56.387 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.384+0000 7f92717fa700 1 -- 172.21.15.186:0/4084976815 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f9268017070 con 0x7f927406dee0 2023-11-28T11:07:56.414 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:56.412+0000 7f075a483200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T11:07:56.467 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.466+0000 7f927b207700 1 -- 172.21.15.186:0/4084976815 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "769bf956-8498-49a7-8945-8cc91bcee69e", "id": 3} v 0) v1 -- 0x7f92740720a0 con 0x7f927406dee0 2023-11-28T11:07:56.470 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.469+0000 7f92717fa700 1 -- 172.21.15.186:0/4084976815 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "769bf956-8498-49a7-8945-8cc91bcee69e", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9268011e10 con 0x7f927406dee0 2023-11-28T11:07:56.471 INFO:teuthology.orchestra.run.smithi186.stdout:3 2023-11-28T11:07:56.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.470+0000 7f927b207700 1 -- 172.21.15.186:0/4084976815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f927406dee0 msgr2=0x7f927415b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.470+0000 7f927b207700 1 --2- 172.21.15.186:0/4084976815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f927406dee0 0x7f927415b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f926800c450 tx=0x7f926800c760 comp rx=0 tx=0).stop 2023-11-28T11:07:56.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.470+0000 7f927b207700 1 -- 172.21.15.186:0/4084976815 shutdown_connections 2023-11-28T11:07:56.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.470+0000 7f927b207700 1 --2- 172.21.15.186:0/4084976815 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9274160540 0x7f92740b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.470+0000 7f927b207700 1 --2- 172.21.15.186:0/4084976815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f927406dee0 0x7f927415b610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.470+0000 7f927b207700 1 --2- 172.21.15.186:0/4084976815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9274071040 0x7f92741f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.473 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.470+0000 7f927b207700 1 -- 172.21.15.186:0/4084976815 >> 172.21.15.186:0/4084976815 conn(0x7f92740d0000 msgr2=0x7f927415e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:56.473 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.470+0000 7f927b207700 1 -- 172.21.15.186:0/4084976815 shutdown_connections 2023-11-28T11:07:56.473 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.470+0000 7f927b207700 1 -- 172.21.15.186:0/4084976815 wait complete. 2023-11-28T11:07:56.485 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 5e44c417-5342-4241-9950-331ce1ceb42c 4 2023-11-28T11:07:56.485 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:56.484+0000 7f444597f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T11:07:56.558 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:56.557+0000 7f075a483200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T11:07:56.605 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.604+0000 7ff82f507700 1 Processor -- start 2023-11-28T11:07:56.606 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.604+0000 7ff82f507700 1 -- start start 2023-11-28T11:07:56.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82f507700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280ce040 0x7ff82815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82f507700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff8280d0a00 0x7ff82806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82f507700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82806b420 con 0x7ff8280b2930 2023-11-28T11:07:56.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82f507700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82806b700 con 0x7ff8280d0a00 2023-11-28T11:07:56.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82f507700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82806b9e0 con 0x7ff8280ce040 2023-11-28T11:07:56.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82caa2700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff8280d0a00 0x7ff82806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82d2a3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280ce040 0x7ff82815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82caa2700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff8280d0a00 0x7ff82806f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47474/0 (socket says 172.21.15.186:47474) 2023-11-28T11:07:56.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82d2a3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280ce040 0x7ff82815df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:32798/0 (socket says 172.21.15.186:32798) 2023-11-28T11:07:56.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82caa2700 1 -- 172.21.15.186:0/743109213 learned_addr learned my addr 172.21.15.186:0/743109213 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:56.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82d2a3700 1 -- 172.21.15.186:0/743109213 >> v2:172.21.15.186:3300/0 conn(0x7ff8280d0a00 msgr2=0x7ff82806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:07:56.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82d2a3700 1 --2- 172.21.15.186:0/743109213 >> v2:172.21.15.186:3300/0 conn(0x7ff8280d0a00 0x7ff82806f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82d2a3700 1 -- 172.21.15.186:0/743109213 >> v1:172.21.15.156:6789/0 conn(0x7ff8280b2930 legacy=0x7ff82806c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82d2a3700 1 -- 172.21.15.186:0/743109213 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82806bcc0 con 0x7ff8280ce040 2023-11-28T11:07:56.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82caa2700 1 --2- 172.21.15.186:0/743109213 >> v2:172.21.15.186:3300/0 conn(0x7ff8280d0a00 0x7ff82806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:56.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.605+0000 7ff82d2a3700 1 --2- 172.21.15.186:0/743109213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280ce040 0x7ff82815df10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff824009820 tx=0x7ff824009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76137656973039dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:56.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.606+0000 7ff81ffff700 1 -- 172.21.15.186:0/743109213 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824005e30 con 0x7ff8280ce040 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.606+0000 7ff81ffff700 1 -- 172.21.15.186:0/743109213 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff824006030 con 0x7ff8280ce040 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.606+0000 7ff81ffff700 1 -- 172.21.15.186:0/743109213 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8240063c0 con 0x7ff8280ce040 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.606+0000 7ff82f507700 1 -- 172.21.15.186:0/743109213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280ce040 msgr2=0x7ff82815df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.606+0000 7ff82f507700 1 --2- 172.21.15.186:0/743109213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280ce040 0x7ff82815df10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff824009820 tx=0x7ff824009b30 comp rx=0 tx=0).stop 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.606+0000 7ff82f507700 1 -- 172.21.15.186:0/743109213 shutdown_connections 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.606+0000 7ff82f507700 1 --2- 172.21.15.186:0/743109213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280ce040 0x7ff82815df10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.606+0000 7ff82f507700 1 --2- 172.21.15.186:0/743109213 >> v2:172.21.15.186:3300/0 conn(0x7ff8280d0a00 0x7ff82806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.606+0000 7ff82f507700 1 -- 172.21.15.186:0/743109213 >> 172.21.15.186:0/743109213 conn(0x7ff82806aed0 msgr2=0x7ff82806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.606+0000 7ff82f507700 1 -- 172.21.15.186:0/743109213 shutdown_connections 2023-11-28T11:07:56.610 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.606+0000 7ff82f507700 1 -- 172.21.15.186:0/743109213 wait complete. 2023-11-28T11:07:56.613 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.607+0000 7ff82f507700 1 Processor -- start 2023-11-28T11:07:56.613 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.607+0000 7ff82f507700 1 -- start start 2023-11-28T11:07:56.613 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.608+0000 7ff82f507700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280b2930 0x7ff8280c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.613 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.608+0000 7ff82f507700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8280ce040 0x7ff82814cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.613 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.608+0000 7ff82f507700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8280d0a00 0x7ff828153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.613 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.608+0000 7ff82f507700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82814d500 con 0x7ff8280d0a00 2023-11-28T11:07:56.613 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.608+0000 7ff82f507700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff828154840 con 0x7ff8280ce040 2023-11-28T11:07:56.613 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.608+0000 7ff82f507700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff828154560 con 0x7ff8280b2930 2023-11-28T11:07:56.614 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.608+0000 7ff82caa2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8280ce040 0x7ff82814cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.614 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.608+0000 7ff82caa2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8280ce040 0x7ff82814cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47482/0 (socket says 172.21.15.186:47482) 2023-11-28T11:07:56.614 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.608+0000 7ff82daa4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8280d0a00 0x7ff828153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.614 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.608+0000 7ff82caa2700 1 -- 172.21.15.186:0/103394062 learned_addr learned my addr 172.21.15.186:0/103394062 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:56.614 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.608+0000 7ff82d2a3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280b2930 0x7ff8280c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.614 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff82caa2700 1 -- 172.21.15.186:0/103394062 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280b2930 msgr2=0x7ff8280c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.614 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff82caa2700 1 --2- 172.21.15.186:0/103394062 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280b2930 0x7ff8280c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.614 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff82caa2700 1 -- 172.21.15.186:0/103394062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8280d0a00 msgr2=0x7ff828153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.614 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff82caa2700 1 --2- 172.21.15.186:0/103394062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8280d0a00 0x7ff828153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.614 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff82caa2700 1 -- 172.21.15.186:0/103394062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff818009550 con 0x7ff8280ce040 2023-11-28T11:07:56.614 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff82daa4700 1 --2- 172.21.15.186:0/103394062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8280d0a00 0x7ff828153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:56.615 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff82d2a3700 1 --2- 172.21.15.186:0/103394062 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280b2930 0x7ff8280c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:56.615 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff82caa2700 1 --2- 172.21.15.186:0/103394062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8280ce040 0x7ff82814cfc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff81800ebe0 tx=0x7ff81800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:56.615 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff81dffb700 1 -- 172.21.15.186:0/103394062 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800c850 con 0x7ff8280ce040 2023-11-28T11:07:56.615 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff81dffb700 1 -- 172.21.15.186:0/103394062 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81800c9b0 con 0x7ff8280ce040 2023-11-28T11:07:56.615 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff82f507700 1 -- 172.21.15.186:0/103394062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff824009430 con 0x7ff8280ce040 2023-11-28T11:07:56.615 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff81dffb700 1 -- 172.21.15.186:0/103394062 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800cd40 con 0x7ff8280ce040 2023-11-28T11:07:56.615 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff82f507700 1 -- 172.21.15.186:0/103394062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82806fd90 con 0x7ff8280ce040 2023-11-28T11:07:56.616 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.609+0000 7ff81dffb700 1 -- 172.21.15.186:0/103394062 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff818019030 con 0x7ff8280ce040 2023-11-28T11:07:56.616 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.610+0000 7ff81dffb700 1 -- 172.21.15.186:0/103394062 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ff8180062a0 con 0x7ff8280ce040 2023-11-28T11:07:56.616 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.611+0000 7ff82f507700 1 -- 172.21.15.186:0/103394062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82806bfa0 con 0x7ff8280ce040 2023-11-28T11:07:56.616 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.613+0000 7ff81dffb700 1 -- 172.21.15.186:0/103394062 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff818021060 con 0x7ff8280ce040 2023-11-28T11:07:56.697 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.694+0000 7ff82f507700 1 -- 172.21.15.186:0/103394062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5e44c417-5342-4241-9950-331ce1ceb42c", "id": 4} v 0) v1 -- 0x7ff82806ff20 con 0x7ff8280ce040 2023-11-28T11:07:56.699 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.697+0000 7ff81dffb700 1 -- 172.21.15.186:0/103394062 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5e44c417-5342-4241-9950-331ce1ceb42c", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff818029020 con 0x7ff8280ce040 2023-11-28T11:07:56.699 INFO:teuthology.orchestra.run.smithi186.stdout:4 2023-11-28T11:07:56.700 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.698+0000 7ff82f507700 1 -- 172.21.15.186:0/103394062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8280ce040 msgr2=0x7ff82814cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.700 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.698+0000 7ff82f507700 1 --2- 172.21.15.186:0/103394062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8280ce040 0x7ff82814cfc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff81800ebe0 tx=0x7ff81800eef0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.700 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.698+0000 7ff82f507700 1 -- 172.21.15.186:0/103394062 shutdown_connections 2023-11-28T11:07:56.700 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.698+0000 7ff82f507700 1 --2- 172.21.15.186:0/103394062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8280d0a00 0x7ff828153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.700 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.698+0000 7ff82f507700 1 --2- 172.21.15.186:0/103394062 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8280b2930 0x7ff8280c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.700 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.698+0000 7ff82f507700 1 --2- 172.21.15.186:0/103394062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8280ce040 0x7ff82814cfc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.700 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.698+0000 7ff82f507700 1 -- 172.21.15.186:0/103394062 >> 172.21.15.186:0/103394062 conn(0x7ff82806aed0 msgr2=0x7ff8280cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:56.700 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.698+0000 7ff82f507700 1 -- 172.21.15.186:0/103394062 shutdown_connections 2023-11-28T11:07:56.701 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.698+0000 7ff82f507700 1 -- 172.21.15.186:0/103394062 wait complete. 2023-11-28T11:07:56.711 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 9927448f-0e01-4a87-9525-5052812549fd 5 2023-11-28T11:07:56.836 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.834+0000 7fe7fc8ef700 1 Processor -- start 2023-11-28T11:07:56.836 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.834+0000 7fe7fc8ef700 1 -- start start 2023-11-28T11:07:56.839 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.835+0000 7fe7fc8ef700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe7f80d01b0 0x7fe7f80d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.839 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.835+0000 7fe7fc8ef700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe7f80b16d0 0x7fe7f80d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.839 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.835+0000 7fe7fc8ef700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f80b0230 con 0x7fe7f80b20b0 2023-11-28T11:07:56.839 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.835+0000 7fe7fc8ef700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f815cec0 con 0x7fe7f80b16d0 2023-11-28T11:07:56.839 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.835+0000 7fe7fc8ef700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f80b4570 con 0x7fe7f80d01b0 2023-11-28T11:07:56.839 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.835+0000 7fe7f659c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe7f80d01b0 0x7fe7f80d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.839 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.835+0000 7fe7f659c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe7f80d01b0 0x7fe7f80d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47486/0 (socket says 172.21.15.186:47486) 2023-11-28T11:07:56.839 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.835+0000 7fe7f5d9b700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fe7f80b20b0 0x7fe7f80b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:38330/0 (socket says 172.21.15.186:38330) 2023-11-28T11:07:56.840 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.835+0000 7fe7f659c700 1 -- 172.21.15.186:0/2261716827 learned_addr learned my addr 172.21.15.186:0/2261716827 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:56.840 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.836+0000 7fe7f6d9d700 1 --2- 172.21.15.186:0/2261716827 >> v2:172.21.15.156:3300/0 conn(0x7fe7f80b16d0 0x7fe7f80d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.840 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.836+0000 7fe7f659c700 1 -- 172.21.15.186:0/2261716827 >> v2:172.21.15.156:3300/0 conn(0x7fe7f80b16d0 msgr2=0x7fe7f80d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:07:56.840 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.836+0000 7fe7f659c700 1 --2- 172.21.15.186:0/2261716827 >> v2:172.21.15.156:3300/0 conn(0x7fe7f80b16d0 0x7fe7f80d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.840 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.836+0000 7fe7f659c700 1 -- 172.21.15.186:0/2261716827 >> v1:172.21.15.186:6789/0 conn(0x7fe7f80b20b0 legacy=0x7fe7f80b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.840 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.836+0000 7fe7f659c700 1 -- 172.21.15.186:0/2261716827 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f80b4850 con 0x7fe7f80d01b0 2023-11-28T11:07:56.840 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.836+0000 7fe7f659c700 1 --2- 172.21.15.186:0/2261716827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f80d01b0 0x7fe7f80d0c20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80b5bb0 tx=0x7fe7e8009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ba6c03ab292d49b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:56.840 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.836+0000 7fe7f559a700 1 -- 172.21.15.186:0/2261716827 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8011830 con 0x7fe7f80d01b0 2023-11-28T11:07:56.840 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.836+0000 7fe7f559a700 1 -- 172.21.15.186:0/2261716827 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e8011990 con 0x7fe7f80d01b0 2023-11-28T11:07:56.840 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.836+0000 7fe7f559a700 1 -- 172.21.15.186:0/2261716827 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8011d20 con 0x7fe7f80d01b0 2023-11-28T11:07:56.840 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.837+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/2261716827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f80d01b0 msgr2=0x7fe7f80d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.841 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.837+0000 7fe7fc8ef700 1 --2- 172.21.15.186:0/2261716827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f80d01b0 0x7fe7f80d0c20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80b5bb0 tx=0x7fe7e8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.841 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.837+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/2261716827 shutdown_connections 2023-11-28T11:07:56.841 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.837+0000 7fe7fc8ef700 1 --2- 172.21.15.186:0/2261716827 >> v2:172.21.15.156:3300/0 conn(0x7fe7f80b16d0 0x7fe7f80d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.841 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.837+0000 7fe7fc8ef700 1 --2- 172.21.15.186:0/2261716827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f80d01b0 0x7fe7f80d0c20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.841 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.837+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/2261716827 >> 172.21.15.186:0/2261716827 conn(0x7fe7f809c3f0 msgr2=0x7fe7f80d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:56.841 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.837+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/2261716827 shutdown_connections 2023-11-28T11:07:56.841 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.837+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/2261716827 wait complete. 2023-11-28T11:07:56.843 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.838+0000 7fe7fc8ef700 1 Processor -- start 2023-11-28T11:07:56.844 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.838+0000 7fe7fc8ef700 1 -- start start 2023-11-28T11:07:56.844 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7fc8ef700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f80b16d0 0x7fe7f8156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.844 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7fc8ef700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f80b20b0 0x7fe7f8157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.844 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7fc8ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7f80d01b0 0x7fe7f8155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:56.844 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7fc8ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f81e4160 con 0x7fe7f80d01b0 2023-11-28T11:07:56.844 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7fc8ef700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f8158310 con 0x7fe7f80b16d0 2023-11-28T11:07:56.844 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7fc8ef700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f81585e0 con 0x7fe7f80b20b0 2023-11-28T11:07:56.844 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7f659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f80b16d0 0x7fe7f8156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.845 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7f5d9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f80b20b0 0x7fe7f8157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.845 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7f659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f80b16d0 0x7fe7f8156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47492/0 (socket says 172.21.15.186:47492) 2023-11-28T11:07:56.845 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7f659c700 1 -- 172.21.15.186:0/1052322764 learned_addr learned my addr 172.21.15.186:0/1052322764 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:56.845 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7f5d9b700 1 --2- 172.21.15.186:0/1052322764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f80b20b0 0x7fe7f8157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:32818/0 (socket says 172.21.15.186:32818) 2023-11-28T11:07:56.845 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7f6d9d700 1 --2- 172.21.15.186:0/1052322764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7f80d01b0 0x7fe7f8155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:56.845 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7f659c700 1 -- 172.21.15.186:0/1052322764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f80b20b0 msgr2=0x7fe7f8157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.845 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7f659c700 1 --2- 172.21.15.186:0/1052322764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f80b20b0 0x7fe7f8157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.845 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7f659c700 1 -- 172.21.15.186:0/1052322764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7f80d01b0 msgr2=0x7fe7f8155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.846 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7f659c700 1 --2- 172.21.15.186:0/1052322764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7f80d01b0 0x7fe7f8155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.846 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.839+0000 7fe7f659c700 1 -- 172.21.15.186:0/1052322764 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e8009550 con 0x7fe7f80b16d0 2023-11-28T11:07:56.846 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.840+0000 7fe7f6d9d700 1 --2- 172.21.15.186:0/1052322764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7f80d01b0 0x7fe7f8155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:56.846 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.840+0000 7fe7f5d9b700 1 --2- 172.21.15.186:0/1052322764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f80b20b0 0x7fe7f8157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:56.846 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.840+0000 7fe7f659c700 1 --2- 172.21.15.186:0/1052322764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f80b16d0 0x7fe7f8156ed0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009130 tx=0x7fe7e8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:56.846 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.840+0000 7fe7e6ffd700 1 -- 172.21.15.186:0/1052322764 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8004510 con 0x7fe7f80b16d0 2023-11-28T11:07:56.846 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.840+0000 7fe7e6ffd700 1 -- 172.21.15.186:0/1052322764 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e801e070 con 0x7fe7f80b16d0 2023-11-28T11:07:56.846 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.840+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/1052322764 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f80afec0 con 0x7fe7f80b16d0 2023-11-28T11:07:56.846 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.840+0000 7fe7e6ffd700 1 -- 172.21.15.186:0/1052322764 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80038a0 con 0x7fe7f80b16d0 2023-11-28T11:07:56.846 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.840+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/1052322764 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f80afbe0 con 0x7fe7f80b16d0 2023-11-28T11:07:56.846 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.840+0000 7fe7e6ffd700 1 -- 172.21.15.186:0/1052322764 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fe7e800f050 con 0x7fe7f80b16d0 2023-11-28T11:07:56.847 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.840+0000 7fe7e6ffd700 1 -- 172.21.15.186:0/1052322764 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fe7e802dbb0 con 0x7fe7f80b16d0 2023-11-28T11:07:56.847 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.841+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/1052322764 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f80b2600 con 0x7fe7f80b16d0 2023-11-28T11:07:56.847 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.846+0000 7fe7e6ffd700 1 -- 172.21.15.186:0/1052322764 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fe7e8020070 con 0x7fe7f80b16d0 2023-11-28T11:07:56.875 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:56.874+0000 7f444597f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T11:07:56.931 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.930+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/1052322764 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9927448f-0e01-4a87-9525-5052812549fd", "id": 5} v 0) v1 -- 0x7fe7f80686f0 con 0x7fe7f80b16d0 2023-11-28T11:07:56.935 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.933+0000 7fe7e6ffd700 1 -- 172.21.15.186:0/1052322764 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9927448f-0e01-4a87-9525-5052812549fd", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe7e8015020 con 0x7fe7f80b16d0 2023-11-28T11:07:56.935 INFO:teuthology.orchestra.run.smithi186.stdout:5 2023-11-28T11:07:56.936 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.934+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/1052322764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f80b16d0 msgr2=0x7fe7f8156ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:56.937 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.934+0000 7fe7fc8ef700 1 --2- 172.21.15.186:0/1052322764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f80b16d0 0x7fe7f8156ed0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009130 tx=0x7fe7e8004750 comp rx=0 tx=0).stop 2023-11-28T11:07:56.937 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.934+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/1052322764 shutdown_connections 2023-11-28T11:07:56.937 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.934+0000 7fe7fc8ef700 1 --2- 172.21.15.186:0/1052322764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7f80d01b0 0x7fe7f8155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.937 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.934+0000 7fe7fc8ef700 1 --2- 172.21.15.186:0/1052322764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f80b16d0 0x7fe7f8156ed0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.937 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.934+0000 7fe7fc8ef700 1 --2- 172.21.15.186:0/1052322764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f80b20b0 0x7fe7f8157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:56.937 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.934+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/1052322764 >> 172.21.15.186:0/1052322764 conn(0x7fe7f809c3f0 msgr2=0x7fe7f806ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:56.937 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.935+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/1052322764 shutdown_connections 2023-11-28T11:07:56.937 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:56.935+0000 7fe7fc8ef700 1 -- 172.21.15.186:0/1052322764 wait complete. 2023-11-28T11:07:56.948 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 0890154e-fd59-440c-8e5c-58495a4cf47d 6 2023-11-28T11:07:56.975 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:56.973+0000 7f075a483200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T11:07:57.069 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.068+0000 7f03a0770700 1 Processor -- start 2023-11-28T11:07:57.069 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.068+0000 7f03a0770700 1 -- start start 2023-11-28T11:07:57.072 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f03a0770700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f039806b060 0x7f039806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.072 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f03a0770700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03980c9e10 con 0x7f0398160740 2023-11-28T11:07:57.072 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f03a0770700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039806efc0 con 0x7f0398160b10 2023-11-28T11:07:57.072 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f03a0770700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03980ca720 con 0x7f039806b060 2023-11-28T11:07:57.072 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f039ed0d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f039806b060 0x7f039806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.072 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f039dd0b700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f0398160b10 0x7f03980cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:38342/0 (socket says 172.21.15.186:38342) 2023-11-28T11:07:57.072 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f039dd0b700 1 -- 172.21.15.186:0/3018466099 learned_addr learned my addr 172.21.15.186:0/3018466099 (peer_addr_for_me v1:172.21.15.186:0/0) 2023-11-28T11:07:57.072 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f039d50a700 1 -- 172.21.15.186:0/3018466099 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138361657 0 0) 0x7f039806efc0 con 0x7f0398160b10 2023-11-28T11:07:57.072 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f039d50a700 1 -- 172.21.15.186:0/3018466099 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03840035b0 con 0x7f0398160b10 2023-11-28T11:07:57.072 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f039ed0d700 1 -- 172.21.15.186:0/3018466099 >> v1:172.21.15.186:6789/0 conn(0x7f0398160b10 legacy=0x7f03980cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.073 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f039ed0d700 1 -- 172.21.15.186:0/3018466099 >> v1:172.21.15.156:6789/0 conn(0x7f0398160740 legacy=0x7f03980cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.073 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f039ed0d700 1 -- 172.21.15.186:0/3018466099 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03980ca0d0 con 0x7f039806b060 2023-11-28T11:07:57.073 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f039ed0d700 1 --2- 172.21.15.186:0/3018466099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f039806b060 0x7f039806b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0390009dc0 tx=0x7f039000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fe23d4e1cc556f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:57.073 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f039d50a700 1 -- 172.21.15.186:0/3018466099 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000e070 con 0x7f039806b060 2023-11-28T11:07:57.073 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f039d50a700 1 -- 172.21.15.186:0/3018466099 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039000bb20 con 0x7f039806b060 2023-11-28T11:07:57.073 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.069+0000 7f039d50a700 1 -- 172.21.15.186:0/3018466099 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390012430 con 0x7f039806b060 2023-11-28T11:07:57.073 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.070+0000 7f03a0770700 1 -- 172.21.15.186:0/3018466099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f039806b060 msgr2=0x7f039806b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.073 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.070+0000 7f03a0770700 1 --2- 172.21.15.186:0/3018466099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f039806b060 0x7f039806b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0390009dc0 tx=0x7f039000b330 comp rx=0 tx=0).stop 2023-11-28T11:07:57.073 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.070+0000 7f03a0770700 1 -- 172.21.15.186:0/3018466099 shutdown_connections 2023-11-28T11:07:57.073 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.070+0000 7f03a0770700 1 --2- 172.21.15.186:0/3018466099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f039806b060 0x7f039806b4e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.073 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.070+0000 7f03a0770700 1 -- 172.21.15.186:0/3018466099 >> 172.21.15.186:0/3018466099 conn(0x7f03980b12b0 msgr2=0x7f03980b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:57.073 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.070+0000 7f03a0770700 1 -- 172.21.15.186:0/3018466099 shutdown_connections 2023-11-28T11:07:57.074 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.070+0000 7f03a0770700 1 -- 172.21.15.186:0/3018466099 wait complete. 2023-11-28T11:07:57.076 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.071+0000 7f03a0770700 1 Processor -- start 2023-11-28T11:07:57.076 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.071+0000 7f03a0770700 1 -- start start 2023-11-28T11:07:57.076 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.072+0000 7f03a0770700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039806b060 0x7f03980ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.072+0000 7f03a0770700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0398160740 0x7f039806d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.072+0000 7f03a0770700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0398160b10 0x7f0398157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.072+0000 7f03a0770700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039806d770 con 0x7f0398160740 2023-11-28T11:07:57.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.072+0000 7f03a0770700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03981e7450 con 0x7f039806b060 2023-11-28T11:07:57.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.072+0000 7f03a0770700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03980acf80 con 0x7f0398160b10 2023-11-28T11:07:57.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.072+0000 7f039e50c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039806b060 0x7f03980ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.072+0000 7f039ed0d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0398160b10 0x7f0398157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.072+0000 7f039e50c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039806b060 0x7f03980ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47504/0 (socket says 172.21.15.186:47504) 2023-11-28T11:07:57.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.072+0000 7f039e50c700 1 -- 172.21.15.186:0/1628800686 learned_addr learned my addr 172.21.15.186:0/1628800686 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:57.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.072+0000 7f039dd0b700 1 --2- 172.21.15.186:0/1628800686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0398160740 0x7f039806d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f039ed0d700 1 -- 172.21.15.186:0/1628800686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039806b060 msgr2=0x7f03980ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f039ed0d700 1 --2- 172.21.15.186:0/1628800686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039806b060 0x7f03980ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f039ed0d700 1 -- 172.21.15.186:0/1628800686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0398160740 msgr2=0x7f039806d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f039ed0d700 1 --2- 172.21.15.186:0/1628800686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0398160740 0x7f039806d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f039ed0d700 1 -- 172.21.15.186:0/1628800686 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390009a70 con 0x7f0398160b10 2023-11-28T11:07:57.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f039e50c700 1 --2- 172.21.15.186:0/1628800686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039806b060 0x7f03980ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:57.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f039dd0b700 1 --2- 172.21.15.186:0/1628800686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0398160740 0x7f039806d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:57.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f039ed0d700 1 --2- 172.21.15.186:0/1628800686 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0398160b10 0x7f0398157cc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0390016000 tx=0x7f0390004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:57.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f038effd700 1 -- 172.21.15.186:0/1628800686 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390012430 con 0x7f0398160b10 2023-11-28T11:07:57.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f038effd700 1 -- 172.21.15.186:0/1628800686 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0390020070 con 0x7f0398160b10 2023-11-28T11:07:57.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f03a0770700 1 -- 172.21.15.186:0/1628800686 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0398160f60 con 0x7f0398160b10 2023-11-28T11:07:57.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f038effd700 1 -- 172.21.15.186:0/1628800686 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390004280 con 0x7f0398160b10 2023-11-28T11:07:57.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f03a0770700 1 -- 172.21.15.186:0/1628800686 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03980ca3b0 con 0x7f0398160b10 2023-11-28T11:07:57.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f038effd700 1 -- 172.21.15.186:0/1628800686 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f03900046d0 con 0x7f0398160b10 2023-11-28T11:07:57.079 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.073+0000 7f038effd700 1 -- 172.21.15.186:0/1628800686 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f0390022070 con 0x7f0398160b10 2023-11-28T11:07:57.079 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.074+0000 7f03a0770700 1 -- 172.21.15.186:0/1628800686 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039806ebb0 con 0x7f0398160b10 2023-11-28T11:07:57.079 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.076+0000 7f038effd700 1 -- 172.21.15.186:0/1628800686 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f0390027480 con 0x7f0398160b10 2023-11-28T11:07:57.168 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.167+0000 7f03a0770700 1 -- 172.21.15.186:0/1628800686 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0890154e-fd59-440c-8e5c-58495a4cf47d", "id": 6} v 0) v1 -- 0x7f03980b5740 con 0x7f0398160b10 2023-11-28T11:07:57.176 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.175+0000 7f038effd700 1 -- 172.21.15.186:0/1628800686 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0890154e-fd59-440c-8e5c-58495a4cf47d", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0390027890 con 0x7f0398160b10 2023-11-28T11:07:57.177 INFO:teuthology.orchestra.run.smithi186.stdout:6 2023-11-28T11:07:57.178 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.176+0000 7f03a0770700 1 -- 172.21.15.186:0/1628800686 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0398160b10 msgr2=0x7f0398157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.178 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.176+0000 7f03a0770700 1 --2- 172.21.15.186:0/1628800686 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0398160b10 0x7f0398157cc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0390016000 tx=0x7f0390004ad0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.178 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.176+0000 7f03a0770700 1 -- 172.21.15.186:0/1628800686 shutdown_connections 2023-11-28T11:07:57.178 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.176+0000 7f03a0770700 1 --2- 172.21.15.186:0/1628800686 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0398160b10 0x7f0398157cc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.178 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.176+0000 7f03a0770700 1 --2- 172.21.15.186:0/1628800686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039806b060 0x7f03980ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.178 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.176+0000 7f03a0770700 1 --2- 172.21.15.186:0/1628800686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0398160740 0x7f039806d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.178 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.176+0000 7f03a0770700 1 -- 172.21.15.186:0/1628800686 >> 172.21.15.186:0/1628800686 conn(0x7f03980b12b0 msgr2=0x7f03980b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:57.178 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.176+0000 7f03a0770700 1 -- 172.21.15.186:0/1628800686 shutdown_connections 2023-11-28T11:07:57.178 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.176+0000 7f03a0770700 1 -- 172.21.15.186:0/1628800686 wait complete. 2023-11-28T11:07:57.190 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new f639cc58-c503-44a9-aa39-4b000df009cc 7 2023-11-28T11:07:57.353 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.351+0000 7ff09848b700 1 Processor -- start 2023-11-28T11:07:57.353 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.351+0000 7ff09848b700 1 -- start start 2023-11-28T11:07:57.356 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.352+0000 7ff09848b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff090160740 0x7ff0900cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.356 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.352+0000 7ff09848b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff090160b10 0x7ff0900cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.356 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.352+0000 7ff09848b700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09006bbf0 con 0x7ff09006b060 2023-11-28T11:07:57.356 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.352+0000 7ff09848b700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0900ca720 con 0x7ff090160b10 2023-11-28T11:07:57.356 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.352+0000 7ff09848b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900cc030 con 0x7ff090160740 2023-11-28T11:07:57.357 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.352+0000 7ff096227700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff090160740 0x7ff0900cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.357 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.352+0000 7ff095a26700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff090160b10 0x7ff0900cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.357 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.352+0000 7ff095a26700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff090160b10 0x7ff0900cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47510/0 (socket says 172.21.15.186:47510) 2023-11-28T11:07:57.357 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.352+0000 7ff096227700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff090160740 0x7ff0900cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:32834/0 (socket says 172.21.15.186:32834) 2023-11-28T11:07:57.357 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.353+0000 7ff095a26700 1 -- 172.21.15.186:0/82804405 learned_addr learned my addr 172.21.15.186:0/82804405 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:57.357 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.353+0000 7ff095a26700 1 -- 172.21.15.186:0/82804405 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff090160740 msgr2=0x7ff0900cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:07:57.357 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.353+0000 7ff095a26700 1 --2- 172.21.15.186:0/82804405 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff090160740 0x7ff0900cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.357 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.353+0000 7ff095a26700 1 -- 172.21.15.186:0/82804405 >> v1:172.21.15.186:6789/0 conn(0x7ff09006b060 legacy=0x7ff09006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.357 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.353+0000 7ff095a26700 1 -- 172.21.15.186:0/82804405 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0900c9e10 con 0x7ff090160b10 2023-11-28T11:07:57.358 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.353+0000 7ff095225700 1 -- 172.21.15.186:0/82804405 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031295011 0 0) 0x7ff09006bbf0 con 0x7ff09006b060 2023-11-28T11:07:57.358 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.353+0000 7ff096227700 1 --2- 172.21.15.186:0/82804405 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff090160740 0x7ff0900cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:57.358 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.353+0000 7ff095a26700 1 --2- 172.21.15.186:0/82804405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff090160b10 0x7ff0900cd390 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0098f0 tx=0x7ff08c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a215d91b886c87e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:57.358 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.353+0000 7ff095225700 1 -- 172.21.15.186:0/82804405 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c011830 con 0x7ff090160b10 2023-11-28T11:07:57.358 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.353+0000 7ff095225700 1 -- 172.21.15.186:0/82804405 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c011990 con 0x7ff090160b10 2023-11-28T11:07:57.358 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.353+0000 7ff095225700 1 -- 172.21.15.186:0/82804405 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c011d20 con 0x7ff090160b10 2023-11-28T11:07:57.358 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.354+0000 7ff09848b700 1 -- 172.21.15.186:0/82804405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff090160b10 msgr2=0x7ff0900cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.358 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.354+0000 7ff09848b700 1 --2- 172.21.15.186:0/82804405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff090160b10 0x7ff0900cd390 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0098f0 tx=0x7ff08c009c00 comp rx=0 tx=0).stop 2023-11-28T11:07:57.358 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.354+0000 7ff09848b700 1 -- 172.21.15.186:0/82804405 shutdown_connections 2023-11-28T11:07:57.359 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.354+0000 7ff09848b700 1 --2- 172.21.15.186:0/82804405 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff090160740 0x7ff0900cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.359 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.354+0000 7ff09848b700 1 --2- 172.21.15.186:0/82804405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff090160b10 0x7ff0900cd390 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.359 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.354+0000 7ff09848b700 1 -- 172.21.15.186:0/82804405 >> 172.21.15.186:0/82804405 conn(0x7ff0900b12b0 msgr2=0x7ff0900b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:57.359 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.354+0000 7ff09848b700 1 -- 172.21.15.186:0/82804405 shutdown_connections 2023-11-28T11:07:57.359 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.354+0000 7ff09848b700 1 -- 172.21.15.186:0/82804405 wait complete. 2023-11-28T11:07:57.362 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.355+0000 7ff09848b700 1 Processor -- start 2023-11-28T11:07:57.362 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.355+0000 7ff09848b700 1 -- start start 2023-11-28T11:07:57.362 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff09848b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09006b060 0x7ff0901f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.362 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff09848b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff090160740 0x7ff09006dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.363 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff09848b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff090160b10 0x7ff09015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.363 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff09848b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900cd350 con 0x7ff090160740 2023-11-28T11:07:57.363 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff09848b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901f69d0 con 0x7ff09006b060 2023-11-28T11:07:57.363 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff09848b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900cd580 con 0x7ff090160b10 2023-11-28T11:07:57.363 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff096227700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09006b060 0x7ff0901f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.363 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff096227700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09006b060 0x7ff0901f6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47516/0 (socket says 172.21.15.186:47516) 2023-11-28T11:07:57.363 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff096a28700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff090160b10 0x7ff09015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.363 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff096227700 1 -- 172.21.15.186:0/1257482864 learned_addr learned my addr 172.21.15.186:0/1257482864 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T11:07:57.363 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff095a26700 1 --2- 172.21.15.186:0/1257482864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff090160740 0x7ff09006dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.364 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff096227700 1 -- 172.21.15.186:0/1257482864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff090160b10 msgr2=0x7ff09015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.364 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff096227700 1 --2- 172.21.15.186:0/1257482864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff090160b10 0x7ff09015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.364 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff096227700 1 -- 172.21.15.186:0/1257482864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff090160740 msgr2=0x7ff09006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.364 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff096227700 1 --2- 172.21.15.186:0/1257482864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff090160740 0x7ff09006dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.364 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff096227700 1 -- 172.21.15.186:0/1257482864 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0800094a0 con 0x7ff09006b060 2023-11-28T11:07:57.364 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff095a26700 1 --2- 172.21.15.186:0/1257482864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff090160740 0x7ff09006dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:07:57.364 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff096227700 1 --2- 172.21.15.186:0/1257482864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09006b060 0x7ff0901f6490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff08000ebe0 tx=0x7ff08000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:57.364 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.356+0000 7ff086ffd700 1 -- 172.21.15.186:0/1257482864 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080009800 con 0x7ff09006b060 2023-11-28T11:07:57.364 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.357+0000 7ff086ffd700 1 -- 172.21.15.186:0/1257482864 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08000c910 con 0x7ff09006b060 2023-11-28T11:07:57.365 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.357+0000 7ff09848b700 1 -- 172.21.15.186:0/1257482864 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08c009550 con 0x7ff09006b060 2023-11-28T11:07:57.365 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.357+0000 7ff086ffd700 1 -- 172.21.15.186:0/1257482864 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08000cc80 con 0x7ff09006b060 2023-11-28T11:07:57.365 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.357+0000 7ff09848b700 1 -- 172.21.15.186:0/1257482864 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0900ca0d0 con 0x7ff09006b060 2023-11-28T11:07:57.365 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.357+0000 7ff086ffd700 1 -- 172.21.15.186:0/1257482864 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff080019030 con 0x7ff09006b060 2023-11-28T11:07:57.365 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.357+0000 7ff086ffd700 1 -- 172.21.15.186:0/1257482864 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7ff080013470 con 0x7ff09006b060 2023-11-28T11:07:57.365 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.358+0000 7ff09848b700 1 -- 172.21.15.186:0/1257482864 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09006efc0 con 0x7ff09006b060 2023-11-28T11:07:57.365 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.360+0000 7ff086ffd700 1 -- 172.21.15.186:0/1257482864 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff080021060 con 0x7ff09006b060 2023-11-28T11:07:57.438 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:57.437+0000 7f444597f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T11:07:57.478 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.477+0000 7ff09848b700 1 -- 172.21.15.186:0/1257482864 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f639cc58-c503-44a9-aa39-4b000df009cc", "id": 7} v 0) v1 -- 0x7ff0900ca3b0 con 0x7ff09006b060 2023-11-28T11:07:57.484 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.482+0000 7ff086ffd700 1 -- 172.21.15.186:0/1257482864 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f639cc58-c503-44a9-aa39-4b000df009cc", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff080017070 con 0x7ff09006b060 2023-11-28T11:07:57.484 INFO:teuthology.orchestra.run.smithi186.stdout:7 2023-11-28T11:07:57.485 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.484+0000 7ff09848b700 1 -- 172.21.15.186:0/1257482864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09006b060 msgr2=0x7ff0901f6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.485 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.484+0000 7ff09848b700 1 --2- 172.21.15.186:0/1257482864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09006b060 0x7ff0901f6490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff08000ebe0 tx=0x7ff08000eef0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.485 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.484+0000 7ff09848b700 1 -- 172.21.15.186:0/1257482864 shutdown_connections 2023-11-28T11:07:57.486 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.484+0000 7ff09848b700 1 --2- 172.21.15.186:0/1257482864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff090160b10 0x7ff09015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.486 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.484+0000 7ff09848b700 1 --2- 172.21.15.186:0/1257482864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09006b060 0x7ff0901f6490 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.486 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.484+0000 7ff09848b700 1 --2- 172.21.15.186:0/1257482864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff090160740 0x7ff09006dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.486 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.484+0000 7ff09848b700 1 -- 172.21.15.186:0/1257482864 >> 172.21.15.186:0/1257482864 conn(0x7ff0900b12b0 msgr2=0x7ff0900ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:57.486 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.484+0000 7ff09848b700 1 -- 172.21.15.186:0/1257482864 shutdown_connections 2023-11-28T11:07:57.486 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T11:07:57.484+0000 7ff09848b700 1 -- 172.21.15.186:0/1257482864 wait complete. 2023-11-28T11:07:57.501 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T11:07:57.501 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T11:07:57.503 INFO:tasks.ceph.osd.0:Started 2023-11-28T11:07:57.503 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T11:07:57.503 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T11:07:57.505 INFO:tasks.ceph.osd.1:Started 2023-11-28T11:07:57.505 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T11:07:57.506 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T11:07:57.507 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:57.506+0000 7f444597f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T11:07:57.507 INFO:tasks.ceph.osd.2:Started 2023-11-28T11:07:57.508 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T11:07:57.508 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T11:07:57.510 INFO:tasks.ceph.osd.3:Started 2023-11-28T11:07:57.510 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T11:07:57.510 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T11:07:57.512 INFO:tasks.ceph.osd.4:Started 2023-11-28T11:07:57.512 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T11:07:57.512 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T11:07:57.514 INFO:tasks.ceph.osd.5:Started 2023-11-28T11:07:57.514 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T11:07:57.514 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T11:07:57.516 INFO:tasks.ceph.osd.6:Started 2023-11-28T11:07:57.516 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T11:07:57.516 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T11:07:57.524 INFO:tasks.ceph.osd.7:Started 2023-11-28T11:07:57.524 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:07:57.619 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:57.618+0000 7f075a483200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T11:07:57.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.662+0000 7fd84a2c7700 1 Processor -- start 2023-11-28T11:07:57.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.662+0000 7fd84a2c7700 1 -- start start 2023-11-28T11:07:57.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.662+0000 7fd84a2c7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd8440d01b0 0x7fd8440d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.662+0000 7fd84a2c7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8440b16d0 0x7fd8440d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.662+0000 7fd84a2c7700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8440b0230 con 0x7fd8440b20b0 2023-11-28T11:07:57.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.662+0000 7fd84a2c7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd84415cec0 con 0x7fd8440d01b0 2023-11-28T11:07:57.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.662+0000 7fd84a2c7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440b4570 con 0x7fd8440b16d0 2023-11-28T11:07:57.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd843fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd8440d01b0 0x7fd8440d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd843fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd8440d01b0 0x7fd8440d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59424/0 (socket says 172.21.15.156:59424) 2023-11-28T11:07:57.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd843fff700 1 -- 172.21.15.156:0/753471561 learned_addr learned my addr 172.21.15.156:0/753471561 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:07:57.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd8437fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd8440b20b0 0x7fd8440b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46070/0 (socket says 172.21.15.156:46070) 2023-11-28T11:07:57.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd848864700 1 --2- 172.21.15.156:0/753471561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8440b16d0 0x7fd8440d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd843fff700 1 -- 172.21.15.156:0/753471561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8440b16d0 msgr2=0x7fd8440d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:07:57.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd843fff700 1 --2- 172.21.15.156:0/753471561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8440b16d0 0x7fd8440d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd843fff700 1 -- 172.21.15.156:0/753471561 >> v1:172.21.15.156:6789/0 conn(0x7fd8440b20b0 legacy=0x7fd8440b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd843fff700 1 -- 172.21.15.156:0/753471561 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8440b4850 con 0x7fd8440d01b0 2023-11-28T11:07:57.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd842ffd700 1 -- 172.21.15.156:0/753471561 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215515691 0 0) 0x7fd8440b0230 con 0x7fd8440b20b0 2023-11-28T11:07:57.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd843fff700 1 --2- 172.21.15.156:0/753471561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8440d01b0 0x7fd8440d0c20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8340098f0 tx=0x7fd834009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b6dc9a3bc1da6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:57.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd842ffd700 1 -- 172.21.15.156:0/753471561 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834011830 con 0x7fd8440d01b0 2023-11-28T11:07:57.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd842ffd700 1 -- 172.21.15.156:0/753471561 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd834011990 con 0x7fd8440d01b0 2023-11-28T11:07:57.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.663+0000 7fd842ffd700 1 -- 172.21.15.156:0/753471561 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834011d20 con 0x7fd8440d01b0 2023-11-28T11:07:57.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.664+0000 7fd84a2c7700 1 -- 172.21.15.156:0/753471561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8440d01b0 msgr2=0x7fd8440d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.664+0000 7fd84a2c7700 1 --2- 172.21.15.156:0/753471561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8440d01b0 0x7fd8440d0c20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8340098f0 tx=0x7fd834009c00 comp rx=0 tx=0).stop 2023-11-28T11:07:57.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.664+0000 7fd84a2c7700 1 -- 172.21.15.156:0/753471561 shutdown_connections 2023-11-28T11:07:57.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.664+0000 7fd84a2c7700 1 --2- 172.21.15.156:0/753471561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8440b16d0 0x7fd8440d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.664+0000 7fd84a2c7700 1 --2- 172.21.15.156:0/753471561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8440d01b0 0x7fd8440d0c20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.664+0000 7fd84a2c7700 1 -- 172.21.15.156:0/753471561 >> 172.21.15.156:0/753471561 conn(0x7fd84409c3f0 msgr2=0x7fd8440d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:57.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.664+0000 7fd84a2c7700 1 -- 172.21.15.156:0/753471561 shutdown_connections 2023-11-28T11:07:57.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.664+0000 7fd84a2c7700 1 -- 172.21.15.156:0/753471561 wait complete. 2023-11-28T11:07:57.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.665+0000 7fd84a2c7700 1 Processor -- start 2023-11-28T11:07:57.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.665+0000 7fd84a2c7700 1 -- start start 2023-11-28T11:07:57.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd84a2c7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8440b16d0 0x7fd8441e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd84a2c7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8440b20b0 0x7fd8440b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd84a2c7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8440d01b0 0x7fd84415a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:57.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd84a2c7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd844158da0 con 0x7fd8440d01b0 2023-11-28T11:07:57.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd84a2c7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440b8d00 con 0x7fd8440b20b0 2023-11-28T11:07:57.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd84a2c7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8441e48b0 con 0x7fd8440b16d0 2023-11-28T11:07:57.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd848864700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8440d01b0 0x7fd84415a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd848864700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8440d01b0 0x7fd84415a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59434/0 (socket says 172.21.15.156:59434) 2023-11-28T11:07:57.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd848864700 1 -- 172.21.15.156:0/2845335043 learned_addr learned my addr 172.21.15.156:0/2845335043 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:07:57.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd843fff700 1 --2- 172.21.15.156:0/2845335043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8440b16d0 0x7fd8441e3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd8437fe700 1 --2- 172.21.15.156:0/2845335043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8440b20b0 0x7fd8440b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:57.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd848864700 1 -- 172.21.15.156:0/2845335043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8440b16d0 msgr2=0x7fd8441e3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd848864700 1 --2- 172.21.15.156:0/2845335043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8440b16d0 0x7fd8441e3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd848864700 1 -- 172.21.15.156:0/2845335043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8440b20b0 msgr2=0x7fd8440b87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd848864700 1 --2- 172.21.15.156:0/2845335043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8440b20b0 0x7fd8440b87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd848864700 1 -- 172.21.15.156:0/2845335043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd834009550 con 0x7fd8440d01b0 2023-11-28T11:07:57.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.666+0000 7fd843fff700 1 --2- 172.21.15.156:0/2845335043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8440b16d0 0x7fd8441e3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:57.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.667+0000 7fd848864700 1 --2- 172.21.15.156:0/2845335043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8440d01b0 0x7fd84415a5d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00efc0 tx=0x7fd83c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:57.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.667+0000 7fd840ff9700 1 -- 172.21.15.156:0/2845335043 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c009ea0 con 0x7fd8440d01b0 2023-11-28T11:07:57.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.667+0000 7fd840ff9700 1 -- 172.21.15.156:0/2845335043 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c00f040 con 0x7fd8440d01b0 2023-11-28T11:07:57.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.667+0000 7fd84a2c7700 1 -- 172.21.15.156:0/2845335043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8440afec0 con 0x7fd8440d01b0 2023-11-28T11:07:57.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.667+0000 7fd840ff9700 1 -- 172.21.15.156:0/2845335043 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00ced0 con 0x7fd8440d01b0 2023-11-28T11:07:57.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.667+0000 7fd84a2c7700 1 -- 172.21.15.156:0/2845335043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8440afbe0 con 0x7fd8440d01b0 2023-11-28T11:07:57.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.667+0000 7fd840ff9700 1 -- 172.21.15.156:0/2845335043 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fd83c014480 con 0x7fd8440d01b0 2023-11-28T11:07:57.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.667+0000 7fd840ff9700 1 -- 172.21.15.156:0/2845335043 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd83c013ab0 con 0x7fd8440d01b0 2023-11-28T11:07:57.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.668+0000 7fd84a2c7700 1 -- 172.21.15.156:0/2845335043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8440b2600 con 0x7fd8440d01b0 2023-11-28T11:07:57.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.672+0000 7fd840ff9700 1 -- 172.21.15.156:0/2845335043 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fd83c018020 con 0x7fd8440d01b0 2023-11-28T11:07:57.690 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:57.688+0000 7f075a483200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T11:07:57.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.741+0000 7fd84a2c7700 1 -- 172.21.15.156:0/2845335043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd844068690 con 0x7fd8440d01b0 2023-11-28T11:07:57.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.742+0000 7fd840ff9700 1 -- 172.21.15.156:0/2845335043 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd83c013e40 con 0x7fd8440d01b0 2023-11-28T11:07:57.743 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:07:57.743 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":10,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:07:57.479373+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:07:57.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.742+0000 7fd84a2c7700 1 -- 172.21.15.156:0/2845335043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8440d01b0 msgr2=0x7fd84415a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:57.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.742+0000 7fd84a2c7700 1 --2- 172.21.15.156:0/2845335043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8440d01b0 0x7fd84415a5d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00efc0 tx=0x7fd83c00c3a0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.743+0000 7fd84a2c7700 1 -- 172.21.15.156:0/2845335043 shutdown_connections 2023-11-28T11:07:57.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.743+0000 7fd84a2c7700 1 --2- 172.21.15.156:0/2845335043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8440d01b0 0x7fd84415a5d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.743+0000 7fd84a2c7700 1 --2- 172.21.15.156:0/2845335043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8440b16d0 0x7fd8441e3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.743+0000 7fd84a2c7700 1 --2- 172.21.15.156:0/2845335043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8440b20b0 0x7fd8440b87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:57.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.743+0000 7fd84a2c7700 1 -- 172.21.15.156:0/2845335043 >> 172.21.15.156:0/2845335043 conn(0x7fd84409c3f0 msgr2=0x7fd8440b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:57.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.743+0000 7fd84a2c7700 1 -- 172.21.15.156:0/2845335043 shutdown_connections 2023-11-28T11:07:57.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:07:57.743+0000 7fd84a2c7700 1 -- 172.21.15.156:0/2845335043 wait complete. 2023-11-28T11:07:57.756 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T11:07:57.756 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T11:07:58.116 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-11-28T11:07:58.114+0000 7f053ab0d640 -1 Falling back to public interface 2023-11-28T11:07:58.116 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-11-28T11:07:58.115+0000 7f45b185b640 -1 Falling back to public interface 2023-11-28T11:07:58.125 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-11-28T11:07:58.123+0000 7fc3e7848640 -1 Falling back to public interface 2023-11-28T11:07:58.129 INFO:tasks.ceph.osd.0.smithi156.stderr:2023-11-28T11:07:58.128+0000 7fb47bbac640 -1 Falling back to public interface 2023-11-28T11:07:58.130 INFO:tasks.ceph.osd.2.smithi156.stderr:2023-11-28T11:07:58.128+0000 7f240823f640 -1 Falling back to public interface 2023-11-28T11:07:58.130 INFO:tasks.ceph.osd.0.smithi156.stderr:2023-11-28T11:07:58.128+0000 7fb472f6a700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-11-28T11:07:58.130 INFO:tasks.ceph.osd.0.smithi156.stderr:2023-11-28T11:07:58.128+0000 7fb472f6a700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2023-11-28T11:07:58.138 INFO:tasks.ceph.osd.3.smithi156.stderr:2023-11-28T11:07:58.137+0000 7f4e6ab52640 -1 Falling back to public interface 2023-11-28T11:07:58.139 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-11-28T11:07:58.138+0000 7f33a34b6640 -1 Falling back to public interface 2023-11-28T11:07:58.139 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-11-28T11:07:58.139+0000 7f339a874700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6819/0: (98) Address already in use 2023-11-28T11:07:58.141 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-11-28T11:07:58.140+0000 7fa35ac9c640 -1 Falling back to public interface 2023-11-28T11:07:58.241 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:58.240+0000 7f444597f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:07:58.319 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:58.318+0000 7f444597f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T11:07:58.378 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:58.377+0000 7f075a483200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:07:58.458 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:58.456+0000 7f075a483200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T11:07:58.459 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:58.458+0000 7f444597f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:07:58.563 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:07:58.596 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:58.595+0000 7f075a483200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:07:58.680 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.679+0000 7f2516bbe700 1 Processor -- start 2023-11-28T11:07:58.680 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.679+0000 7f2516bbe700 1 -- start start 2023-11-28T11:07:58.682 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.679+0000 7f2516bbe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f251015cef0 0x7f25100af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:58.682 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.679+0000 7f2516bbe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f25100cbdf0 0x7f251006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:58.682 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.679+0000 7f2516bbe700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25100cc5f0 con 0x7f251006b8c0 2023-11-28T11:07:58.682 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.679+0000 7f2516bbe700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251006bd40 con 0x7f251015cef0 2023-11-28T11:07:58.682 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.679+0000 7f2516bbe700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251015d470 con 0x7f25100cbdf0 2023-11-28T11:07:58.682 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.680+0000 7f250ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f25100cbdf0 0x7f251006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:58.682 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.680+0000 7f251515b700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f251006b8c0 0x7f2510071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:56444/0 (socket says 172.21.15.156:56444) 2023-11-28T11:07:58.682 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.680+0000 7f251515b700 1 -- 172.21.15.156:0/1164840099 learned_addr learned my addr 172.21.15.156:0/1164840099 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:07:58.682 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.680+0000 7f250ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f25100cbdf0 0x7f251006b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:35072/0 (socket says 172.21.15.156:35072) 2023-11-28T11:07:58.682 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.680+0000 7f250f7fe700 1 -- 172.21.15.156:0/1164840099 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731623866 0 0) 0x7f25100cc5f0 con 0x7f251006b8c0 2023-11-28T11:07:58.682 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.680+0000 7f250ffff700 1 -- 172.21.15.156:0/1164840099 >> v2:172.21.15.186:3300/0 conn(0x7f251015cef0 msgr2=0x7f25100af400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:07:58.683 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.680+0000 7f250ffff700 1 --2- 172.21.15.156:0/1164840099 >> v2:172.21.15.186:3300/0 conn(0x7f251015cef0 0x7f25100af400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:58.683 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.680+0000 7f250ffff700 1 -- 172.21.15.156:0/1164840099 >> v1:172.21.15.186:6789/0 conn(0x7f251006b8c0 legacy=0x7f2510071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:58.683 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.680+0000 7f250ffff700 1 -- 172.21.15.156:0/1164840099 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251015d750 con 0x7f25100cbdf0 2023-11-28T11:07:58.683 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.680+0000 7f250ffff700 1 --2- 172.21.15.156:0/1164840099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f25100cbdf0 0x7f251006b380 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009840 tx=0x7f24fc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd6cfd04c718d18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:58.683 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.680+0000 7f250f7fe700 1 -- 172.21.15.156:0/1164840099 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc011830 con 0x7f25100cbdf0 2023-11-28T11:07:58.683 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.681+0000 7f250f7fe700 1 -- 172.21.15.156:0/1164840099 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc011990 con 0x7f25100cbdf0 2023-11-28T11:07:58.683 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.681+0000 7f250f7fe700 1 -- 172.21.15.156:0/1164840099 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc011d20 con 0x7f25100cbdf0 2023-11-28T11:07:58.687 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.681+0000 7f2516bbe700 1 -- 172.21.15.156:0/1164840099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f25100cbdf0 msgr2=0x7f251006b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:58.687 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.681+0000 7f2516bbe700 1 --2- 172.21.15.156:0/1164840099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f25100cbdf0 0x7f251006b380 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009840 tx=0x7f24fc009b50 comp rx=0 tx=0).stop 2023-11-28T11:07:58.687 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.681+0000 7f2516bbe700 1 -- 172.21.15.156:0/1164840099 shutdown_connections 2023-11-28T11:07:58.687 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.681+0000 7f2516bbe700 1 --2- 172.21.15.156:0/1164840099 >> v2:172.21.15.186:3300/0 conn(0x7f251015cef0 0x7f25100af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:58.687 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.681+0000 7f2516bbe700 1 --2- 172.21.15.156:0/1164840099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f25100cbdf0 0x7f251006b380 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:58.688 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.681+0000 7f2516bbe700 1 -- 172.21.15.156:0/1164840099 >> 172.21.15.156:0/1164840099 conn(0x7f25100d0000 msgr2=0x7f25100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:58.688 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.681+0000 7f2516bbe700 1 -- 172.21.15.156:0/1164840099 shutdown_connections 2023-11-28T11:07:58.688 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.681+0000 7f2516bbe700 1 -- 172.21.15.156:0/1164840099 wait complete. 2023-11-28T11:07:58.688 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.682+0000 7f2516bbe700 1 Processor -- start 2023-11-28T11:07:58.688 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.682+0000 7f2516bbe700 1 -- start start 2023-11-28T11:07:58.688 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.683+0000 7f2516bbe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f251006b8c0 0x7f25101f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:58.688 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.683+0000 7f2516bbe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25100cbdf0 0x7f251014ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:58.688 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.683+0000 7f2516bbe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f251015cef0 0x7f25100cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:07:58.688 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.683+0000 7f2516bbe700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251014c580 con 0x7f25100cbdf0 2023-11-28T11:07:58.688 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.683+0000 7f2516bbe700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25101f29f0 con 0x7f251015cef0 2023-11-28T11:07:58.688 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.683+0000 7f2516bbe700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25101f2cc0 con 0x7f251006b8c0 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.683+0000 7f250ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25100cbdf0 0x7f251014ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.683+0000 7f251515b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f251015cef0 0x7f25100cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25100cbdf0 0x7f251014ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59444/0 (socket says 172.21.15.156:59444) 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250ffff700 1 -- 172.21.15.156:0/3767837769 learned_addr learned my addr 172.21.15.156:0/3767837769 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f251495a700 1 --2- 172.21.15.156:0/3767837769 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f251006b8c0 0x7f25101f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250ffff700 1 -- 172.21.15.156:0/3767837769 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f251006b8c0 msgr2=0x7f25101f24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250ffff700 1 --2- 172.21.15.156:0/3767837769 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f251006b8c0 0x7f25101f24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250ffff700 1 -- 172.21.15.156:0/3767837769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f251015cef0 msgr2=0x7f25100cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250ffff700 1 --2- 172.21.15.156:0/3767837769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f251015cef0 0x7f25100cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250ffff700 1 -- 172.21.15.156:0/3767837769 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc0094a0 con 0x7f25100cbdf0 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f251515b700 1 --2- 172.21.15.156:0/3767837769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f251015cef0 0x7f25100cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250ffff700 1 --2- 172.21.15.156:0/3767837769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25100cbdf0 0x7f251014ccf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0090b0 tx=0x7f24fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:07:58.689 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250d7fa700 1 -- 172.21.15.156:0/3767837769 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc0036a0 con 0x7f25100cbdf0 2023-11-28T11:07:58.690 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250d7fa700 1 -- 172.21.15.156:0/3767837769 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc01f070 con 0x7f25100cbdf0 2023-11-28T11:07:58.690 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f2516bbe700 1 -- 172.21.15.156:0/3767837769 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251015dd10 con 0x7f25100cbdf0 2023-11-28T11:07:58.690 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250d7fa700 1 -- 172.21.15.156:0/3767837769 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc003a00 con 0x7f25100cbdf0 2023-11-28T11:07:58.690 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f2516bbe700 1 -- 172.21.15.156:0/3767837769 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251015da30 con 0x7f25100cbdf0 2023-11-28T11:07:58.690 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250d7fa700 1 -- 172.21.15.156:0/3767837769 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f24fc00f050 con 0x7f25100cbdf0 2023-11-28T11:07:58.690 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.684+0000 7f250d7fa700 1 -- 172.21.15.156:0/3767837769 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f24fc021070 con 0x7f25100cbdf0 2023-11-28T11:07:58.690 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.685+0000 7f2516bbe700 1 -- 172.21.15.156:0/3767837769 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251015dff0 con 0x7f25100cbdf0 2023-11-28T11:07:58.690 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.687+0000 7f250d7fa700 1 -- 172.21.15.156:0/3767837769 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f24fc045b00 con 0x7f25100cbdf0 2023-11-28T11:07:58.759 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.757+0000 7f2516bbe700 1 -- 172.21.15.156:0/3767837769 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25100d0e60 con 0x7f25100cbdf0 2023-11-28T11:07:58.759 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.758+0000 7f250d7fa700 1 -- 172.21.15.156:0/3767837769 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f24fc034070 con 0x7f25100cbdf0 2023-11-28T11:07:58.759 INFO:teuthology.misc.health.smithi156.stdout: 2023-11-28T11:07:58.759 INFO:teuthology.misc.health.smithi156.stdout:{"epoch":10,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:07:57.479373+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:07:58.760 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.759+0000 7f2516bbe700 1 -- 172.21.15.156:0/3767837769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25100cbdf0 msgr2=0x7f251014ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:07:58.760 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.759+0000 7f2516bbe700 1 --2- 172.21.15.156:0/3767837769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25100cbdf0 0x7f251014ccf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0090b0 tx=0x7f24fc004750 comp rx=0 tx=0).stop 2023-11-28T11:07:58.760 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.759+0000 7f2516bbe700 1 -- 172.21.15.156:0/3767837769 shutdown_connections 2023-11-28T11:07:58.760 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.759+0000 7f2516bbe700 1 --2- 172.21.15.156:0/3767837769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f251015cef0 0x7f25100cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:58.761 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.759+0000 7f2516bbe700 1 --2- 172.21.15.156:0/3767837769 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f251006b8c0 0x7f25101f24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:58.761 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.759+0000 7f2516bbe700 1 --2- 172.21.15.156:0/3767837769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25100cbdf0 0x7f251014ccf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:07:58.761 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.759+0000 7f2516bbe700 1 -- 172.21.15.156:0/3767837769 >> 172.21.15.156:0/3767837769 conn(0x7f25100d0000 msgr2=0x7f25100b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:07:58.761 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.759+0000 7f2516bbe700 1 -- 172.21.15.156:0/3767837769 shutdown_connections 2023-11-28T11:07:58.761 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:07:58.759+0000 7f2516bbe700 1 -- 172.21.15.156:0/3767837769 wait complete. 2023-11-28T11:07:58.771 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T11:07:58.772 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:58.771+0000 7f444597f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T11:07:58.900 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:58.898+0000 7f075a483200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T11:07:58.904 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:58.903+0000 7f444597f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T11:07:59.036 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:59.034+0000 7f075a483200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T11:07:59.121 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:59.120+0000 7f444597f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T11:07:59.253 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:59.251+0000 7f075a483200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T11:07:59.275 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:59.274+0000 7f444597f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T11:07:59.388 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:59.387+0000 7f444597f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T11:07:59.408 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:59.407+0000 7f075a483200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T11:07:59.525 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:59.523+0000 7f075a483200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T11:07:59.709 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:59.708+0000 7f444597f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T11:07:59.809 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:59.808+0000 7f444597f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T11:07:59.850 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:59.848+0000 7f075a483200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T11:07:59.948 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:07:59.946+0000 7f075a483200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T11:07:59.989 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:07:59.988+0000 7f444597f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T11:08:00.132 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:08:00.130+0000 7f075a483200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T11:08:00.505 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:08:00.504+0000 7f444597f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T11:08:00.663 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T11:08:00.662+0000 7f075a483200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T11:08:01.527 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:08:01.526+0000 7f43fcbe6700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T11:08:01.528 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T11:08:01.527+0000 7f43fcbe6700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T11:08:01.620 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-11-28T11:08:01.618+0000 7f053ab0d640 -1 osd.6 0 log_to_monitors true 2023-11-28T11:08:01.647 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-11-28T11:08:01.645+0000 7f45b185b640 -1 osd.5 0 log_to_monitors true 2023-11-28T11:08:01.659 INFO:tasks.ceph.osd.2.smithi156.stderr:2023-11-28T11:08:01.658+0000 7f240823f640 -1 osd.2 0 log_to_monitors true 2023-11-28T11:08:01.667 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-11-28T11:08:01.666+0000 7f33a34b6640 -1 osd.1 0 log_to_monitors true 2023-11-28T11:08:01.668 INFO:tasks.ceph.osd.0.smithi156.stderr:2023-11-28T11:08:01.667+0000 7fb47bbac640 -1 osd.0 0 log_to_monitors true 2023-11-28T11:08:01.703 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-11-28T11:08:01.702+0000 7fa35ac9c640 -1 osd.4 0 log_to_monitors true 2023-11-28T11:08:01.703 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-11-28T11:08:01.702+0000 7fc3e7848640 -1 osd.7 0 log_to_monitors true 2023-11-28T11:08:01.708 INFO:tasks.ceph.osd.3.smithi156.stderr:2023-11-28T11:08:01.708+0000 7f4e6ab52640 -1 osd.3 0 log_to_monitors true 2023-11-28T11:08:04.838 INFO:tasks.ceph.osd.0.smithi156.stderr:2023-11-28T11:08:04.836+0000 7fb46f763700 -1 osd.0 0 waiting for initial osdmap 2023-11-28T11:08:08.923 INFO:tasks.ceph.osd.3.smithi156.stderr:2023-11-28T11:08:04.841+0000 7f4e5e709700 -1 osd.3 0 waiting for initial osdmap 2023-11-28T11:08:08.923 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-11-28T11:08:05.491+0000 7f052e6c4700 -1 osd.6 0 waiting for initial osdmap 2023-11-28T11:08:08.924 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-11-28T11:08:05.506+0000 7f0526a9f700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:08.924 INFO:tasks.ceph.osd.0.smithi156.stderr:2023-11-28T11:08:04.848+0000 7fb467b3e700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:08.925 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-11-28T11:08:04.851+0000 7f339706d700 -1 osd.1 0 waiting for initial osdmap 2023-11-28T11:08:08.925 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-11-28T11:08:05.523+0000 7fa34e853700 -1 osd.4 0 waiting for initial osdmap 2023-11-28T11:08:08.925 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-11-28T11:08:05.535+0000 7fa346c2e700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:08.925 INFO:tasks.ceph.osd.3.smithi156.stderr:2023-11-28T11:08:04.854+0000 7f4e56ae4700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:08.949 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-11-28T11:08:05.553+0000 7fc3db3ff700 -1 osd.7 0 waiting for initial osdmap 2023-11-28T11:08:08.949 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-11-28T11:08:05.568+0000 7fc3d37da700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:08.949 INFO:tasks.ceph.osd.2.smithi156.stderr:2023-11-28T11:08:04.861+0000 7f23fbdf6700 -1 osd.2 0 waiting for initial osdmap 2023-11-28T11:08:08.949 INFO:tasks.ceph.osd.2.smithi156.stderr:2023-11-28T11:08:04.918+0000 7f23f41d1700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:08.949 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-11-28T11:08:04.922+0000 7f338f448700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:08.950 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-11-28T11:08:05.585+0000 7f45a5412700 -1 osd.5 0 waiting for initial osdmap 2023-11-28T11:08:08.950 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-11-28T11:08:05.597+0000 7f459d7ed700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:09.627 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:09.740 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.739+0000 7f0f02f65700 1 Processor -- start 2023-11-28T11:08:09.740 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.739+0000 7f0f02f65700 1 -- start start 2023-11-28T11:08:09.742 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.739+0000 7f0f02f65700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0efc071040 0x7f0efc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:09.743 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.739+0000 7f0f02f65700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0efc160540 0x7f0efc1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:09.743 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.739+0000 7f0f02f65700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0efc071950 con 0x7f0efc06dee0 2023-11-28T11:08:09.743 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.739+0000 7f0f02f65700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0efc15d060 con 0x7f0efc160540 2023-11-28T11:08:09.743 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.739+0000 7f0f02f65700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc15d340 con 0x7f0efc071040 2023-11-28T11:08:09.743 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.740+0000 7f0f01502700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0efc160540 0x7f0efc1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:09.743 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.740+0000 7f0f01502700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0efc160540 0x7f0efc1609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:42550/0 (socket says 172.21.15.156:42550) 2023-11-28T11:08:09.743 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.740+0000 7f0efbfff700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f0efc06dee0 0x7f0efc06e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:56770/0 (socket says 172.21.15.156:56770) 2023-11-28T11:08:09.743 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.740+0000 7f0f01502700 1 -- 172.21.15.156:0/1632814451 learned_addr learned my addr 172.21.15.156:0/1632814451 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:09.744 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.740+0000 7f0f00d01700 1 --2- 172.21.15.156:0/1632814451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0efc071040 0x7f0efc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:09.744 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.740+0000 7f0f01502700 1 -- 172.21.15.156:0/1632814451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0efc071040 msgr2=0x7f0efc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:09.744 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.740+0000 7f0f01502700 1 --2- 172.21.15.156:0/1632814451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0efc071040 0x7f0efc071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:09.744 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.740+0000 7f0f01502700 1 -- 172.21.15.156:0/1632814451 >> v1:172.21.15.186:6789/0 conn(0x7f0efc06dee0 legacy=0x7f0efc06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:09.744 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.740+0000 7f0f01502700 1 -- 172.21.15.156:0/1632814451 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0efc15d620 con 0x7f0efc160540 2023-11-28T11:08:09.744 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.740+0000 7f0f01502700 1 --2- 172.21.15.156:0/1632814451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0efc160540 0x7f0efc1609c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009e50 tx=0x7f0ef400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7bcf1500c1d1ee8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:09.744 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.741+0000 7f0efb7fe700 1 -- 172.21.15.156:0/1632814451 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef400e070 con 0x7f0efc160540 2023-11-28T11:08:09.744 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.741+0000 7f0efb7fe700 1 -- 172.21.15.156:0/1632814451 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef400b910 con 0x7f0efc160540 2023-11-28T11:08:09.744 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.741+0000 7f0efb7fe700 1 -- 172.21.15.156:0/1632814451 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef400bcc0 con 0x7f0efc160540 2023-11-28T11:08:09.744 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.741+0000 7f0f02f65700 1 -- 172.21.15.156:0/1632814451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0efc160540 msgr2=0x7f0efc1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:09.745 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.741+0000 7f0f02f65700 1 --2- 172.21.15.156:0/1632814451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0efc160540 0x7f0efc1609c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009e50 tx=0x7f0ef400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:09.745 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.741+0000 7f0f02f65700 1 -- 172.21.15.156:0/1632814451 shutdown_connections 2023-11-28T11:08:09.745 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.741+0000 7f0f02f65700 1 --2- 172.21.15.156:0/1632814451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0efc160540 0x7f0efc1609c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:09.745 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.741+0000 7f0f02f65700 1 --2- 172.21.15.156:0/1632814451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0efc071040 0x7f0efc071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:09.745 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.741+0000 7f0f02f65700 1 -- 172.21.15.156:0/1632814451 >> 172.21.15.156:0/1632814451 conn(0x7f0efc0d0000 msgr2=0x7f0efc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:09.745 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.741+0000 7f0f02f65700 1 -- 172.21.15.156:0/1632814451 shutdown_connections 2023-11-28T11:08:09.745 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.741+0000 7f0f02f65700 1 -- 172.21.15.156:0/1632814451 wait complete. 2023-11-28T11:08:09.749 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.742+0000 7f0f02f65700 1 Processor -- start 2023-11-28T11:08:09.749 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.742+0000 7f0f02f65700 1 -- start start 2023-11-28T11:08:09.749 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.743+0000 7f0f02f65700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0efc06dee0 0x7f0efc14d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:09.749 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.743+0000 7f0f02f65700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0efc071040 0x7f0efc0c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:09.749 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.743+0000 7f0f02f65700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc160540 0x7f0efc159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:09.749 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.743+0000 7f0f02f65700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc156eb0 con 0x7f0efc160540 2023-11-28T11:08:09.749 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.743+0000 7f0f02f65700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc0b4ee0 con 0x7f0efc071040 2023-11-28T11:08:09.749 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.743+0000 7f0f02f65700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc14d880 con 0x7f0efc06dee0 2023-11-28T11:08:09.749 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.743+0000 7f0f01502700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc160540 0x7f0efc159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:09.749 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.743+0000 7f0f01502700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc160540 0x7f0efc159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59774/0 (socket says 172.21.15.156:59774) 2023-11-28T11:08:09.750 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.743+0000 7f0f01502700 1 -- 172.21.15.156:0/1715687740 learned_addr learned my addr 172.21.15.156:0/1715687740 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:09.750 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0efbfff700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0efc071040 0x7f0efc0c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:09.750 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0f00d01700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0efc06dee0 0x7f0efc14d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:09.750 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0f01502700 1 -- 172.21.15.156:0/1715687740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0efc06dee0 msgr2=0x7f0efc14d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:09.750 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0f01502700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0efc06dee0 0x7f0efc14d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:09.750 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0f01502700 1 -- 172.21.15.156:0/1715687740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0efc071040 msgr2=0x7f0efc0c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:09.750 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0f01502700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0efc071040 0x7f0efc0c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:09.750 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0f01502700 1 -- 172.21.15.156:0/1715687740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef4009b30 con 0x7f0efc160540 2023-11-28T11:08:09.750 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0efbfff700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0efc071040 0x7f0efc0c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:09.750 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0f01502700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc160540 0x7f0efc159100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0efc06c7b0 tx=0x7f0ef4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:09.751 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0ef97fa700 1 -- 172.21.15.156:0/1715687740 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef40099b0 con 0x7f0efc160540 2023-11-28T11:08:09.751 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0ef97fa700 1 -- 172.21.15.156:0/1715687740 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef40092e0 con 0x7f0efc160540 2023-11-28T11:08:09.751 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0f02f65700 1 -- 172.21.15.156:0/1715687740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0efc15ccd0 con 0x7f0efc160540 2023-11-28T11:08:09.751 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0ef97fa700 1 -- 172.21.15.156:0/1715687740 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef401c640 con 0x7f0efc160540 2023-11-28T11:08:09.751 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.744+0000 7f0f02f65700 1 -- 172.21.15.156:0/1715687740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0efc15d900 con 0x7f0efc160540 2023-11-28T11:08:09.751 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.745+0000 7f0f02f65700 1 -- 172.21.15.156:0/1715687740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0efc06a270 con 0x7f0efc160540 2023-11-28T11:08:09.751 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.745+0000 7f0ef97fa700 1 -- 172.21.15.156:0/1715687740 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0ef402c070 con 0x7f0efc160540 2023-11-28T11:08:09.751 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.746+0000 7f0ef97fa700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0ee00682f0 0x7f0ee006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:09.751 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.746+0000 7f0ef97fa700 1 -- 172.21.15.156:0/1715687740 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0ef4087430 con 0x7f0efc160540 2023-11-28T11:08:09.751 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.746+0000 7f0f00d01700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0ee00682f0 0x7f0ee006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:09.751 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.746+0000 7f0f00d01700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0ee00682f0 0x7f0ee006a7a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009ad0 tx=0x7f0ef0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:09.752 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.750+0000 7f0ef97fa700 1 -- 172.21.15.156:0/1715687740 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0ef4054140 con 0x7f0efc160540 2023-11-28T11:08:09.939 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.938+0000 7f0f02f65700 1 -- 172.21.15.156:0/1715687740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0efc0b5020 con 0x7f0efc160540 2023-11-28T11:08:09.939 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.939+0000 7f0ef97fa700 1 -- 172.21.15.156:0/1715687740 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12526 (secure 0 0 0) 0x7f0ef4057760 con 0x7f0efc160540 2023-11-28T11:08:09.940 INFO:teuthology.misc.health.smithi156.stdout: 2023-11-28T11:08:09.940 INFO:teuthology.misc.health.smithi156.stdout:{"epoch":16,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:08.924835+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:09.942 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.941+0000 7f0f02f65700 1 -- 172.21.15.156:0/1715687740 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0ee00682f0 msgr2=0x7f0ee006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:09.942 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.941+0000 7f0f02f65700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0ee00682f0 0x7f0ee006a7a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009ad0 tx=0x7f0ef0008040 comp rx=0 tx=0).stop 2023-11-28T11:08:09.943 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.941+0000 7f0f02f65700 1 -- 172.21.15.156:0/1715687740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc160540 msgr2=0x7f0efc159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:09.943 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.941+0000 7f0f02f65700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc160540 0x7f0efc159100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0efc06c7b0 tx=0x7f0ef4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:09.943 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.941+0000 7f0f02f65700 1 -- 172.21.15.156:0/1715687740 shutdown_connections 2023-11-28T11:08:09.943 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.941+0000 7f0f02f65700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0efc160540 0x7f0efc159100 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:09.943 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.941+0000 7f0f02f65700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0ee00682f0 0x7f0ee006a7a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:09.943 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.941+0000 7f0f02f65700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0efc06dee0 0x7f0efc14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:09.943 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.941+0000 7f0f02f65700 1 --2- 172.21.15.156:0/1715687740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0efc071040 0x7f0efc0c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:09.943 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.941+0000 7f0f02f65700 1 -- 172.21.15.156:0/1715687740 >> 172.21.15.156:0/1715687740 conn(0x7f0efc0d0000 msgr2=0x7f0efc15f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:09.943 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.941+0000 7f0f02f65700 1 -- 172.21.15.156:0/1715687740 shutdown_connections 2023-11-28T11:08:09.943 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T11:08:09.941+0000 7f0f02f65700 1 -- 172.21.15.156:0/1715687740 wait complete. 2023-11-28T11:08:09.953 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T11:08:09.953 INFO:tasks.ceph:Creating RBD pool 2023-11-28T11:08:09.954 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-28T11:08:10.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.072+0000 7fa4fdd55700 1 Processor -- start 2023-11-28T11:08:10.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.072+0000 7fa4fdd55700 1 -- start start 2023-11-28T11:08:10.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.073+0000 7fa4fdd55700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa4f806dee0 0x7fa4f806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:10.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.073+0000 7fa4fdd55700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa4f8160540 0x7fa4f8160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:10.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.073+0000 7fa4fdd55700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f815d060 con 0x7fa4f8071040 2023-11-28T11:08:10.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.073+0000 7fa4fdd55700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f815d340 con 0x7fa4f806dee0 2023-11-28T11:08:10.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.073+0000 7fa4fdd55700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f815d620 con 0x7fa4f8160540 2023-11-28T11:08:10.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.073+0000 7fa4f6ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa4f806dee0 0x7fa4f806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:10.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.073+0000 7fa4f6ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa4f806dee0 0x7fa4f806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59780/0 (socket says 172.21.15.156:59780) 2023-11-28T11:08:10.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.073+0000 7fa4f6ffd700 1 -- 172.21.15.156:0/413036502 learned_addr learned my addr 172.21.15.156:0/413036502 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:10.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.073+0000 7fa4f7fff700 1 --2- 172.21.15.156:0/413036502 >> v2:172.21.15.186:3300/0 conn(0x7fa4f8160540 0x7fa4f8160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:10.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.074+0000 7fa4f6ffd700 1 -- 172.21.15.156:0/413036502 >> v2:172.21.15.186:3300/0 conn(0x7fa4f8160540 msgr2=0x7fa4f8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:10.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.074+0000 7fa4f6ffd700 1 --2- 172.21.15.156:0/413036502 >> v2:172.21.15.186:3300/0 conn(0x7fa4f8160540 0x7fa4f8160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:10.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.074+0000 7fa4f6ffd700 1 -- 172.21.15.156:0/413036502 >> v1:172.21.15.186:6789/0 conn(0x7fa4f8071040 legacy=0x7fa4f8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:10.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.074+0000 7fa4f6ffd700 1 -- 172.21.15.156:0/413036502 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f815d900 con 0x7fa4f806dee0 2023-11-28T11:08:10.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.074+0000 7fa4f6ffd700 1 --2- 172.21.15.156:0/413036502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4f806dee0 0x7fa4f806e330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80c3a10 tx=0x7fa4e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb1abb8f43ca6d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:10.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.074+0000 7fa4f67fc700 1 -- 172.21.15.156:0/413036502 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8011830 con 0x7fa4f806dee0 2023-11-28T11:08:10.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.074+0000 7fa4f67fc700 1 -- 172.21.15.156:0/413036502 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e8011990 con 0x7fa4f806dee0 2023-11-28T11:08:10.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.074+0000 7fa4f67fc700 1 -- 172.21.15.156:0/413036502 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8011d20 con 0x7fa4f806dee0 2023-11-28T11:08:10.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.074+0000 7fa4fdd55700 1 -- 172.21.15.156:0/413036502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4f806dee0 msgr2=0x7fa4f806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:10.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.074+0000 7fa4fdd55700 1 --2- 172.21.15.156:0/413036502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4f806dee0 0x7fa4f806e330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80c3a10 tx=0x7fa4e8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:10.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.075+0000 7fa4fdd55700 1 -- 172.21.15.156:0/413036502 shutdown_connections 2023-11-28T11:08:10.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.075+0000 7fa4fdd55700 1 --2- 172.21.15.156:0/413036502 >> v2:172.21.15.186:3300/0 conn(0x7fa4f8160540 0x7fa4f8160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:10.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.075+0000 7fa4fdd55700 1 --2- 172.21.15.156:0/413036502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4f806dee0 0x7fa4f806e330 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:10.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.075+0000 7fa4fdd55700 1 -- 172.21.15.156:0/413036502 >> 172.21.15.156:0/413036502 conn(0x7fa4f80d0000 msgr2=0x7fa4f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:10.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.075+0000 7fa4fdd55700 1 -- 172.21.15.156:0/413036502 shutdown_connections 2023-11-28T11:08:10.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.075+0000 7fa4fdd55700 1 -- 172.21.15.156:0/413036502 wait complete. 2023-11-28T11:08:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.076+0000 7fa4fdd55700 1 Processor -- start 2023-11-28T11:08:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.076+0000 7fa4fdd55700 1 -- start start 2023-11-28T11:08:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4fdd55700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4f806dee0 0x7fa4f814a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4fdd55700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4f8071040 0x7fa4f80b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4fdd55700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4f8160540 0x7fa4f80b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4fdd55700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f81f7190 con 0x7fa4f8160540 2023-11-28T11:08:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4fdd55700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f81f6610 con 0x7fa4f8071040 2023-11-28T11:08:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4fdd55700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f81f68f0 con 0x7fa4f806dee0 2023-11-28T11:08:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4f7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4f8160540 0x7fa4f80b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4f7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4f8160540 0x7fa4f80b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59788/0 (socket says 172.21.15.156:59788) 2023-11-28T11:08:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4f7fff700 1 -- 172.21.15.156:0/2993576905 learned_addr learned my addr 172.21.15.156:0/2993576905 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4f6ffd700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4f8071040 0x7fa4f80b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4f77fe700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4f806dee0 0x7fa4f814a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:10.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4f7fff700 1 -- 172.21.15.156:0/2993576905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4f806dee0 msgr2=0x7fa4f814a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:10.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4f7fff700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4f806dee0 0x7fa4f814a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:10.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4f7fff700 1 -- 172.21.15.156:0/2993576905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4f8071040 msgr2=0x7fa4f80b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:10.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4f7fff700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4f8071040 0x7fa4f80b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:10.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4f7fff700 1 -- 172.21.15.156:0/2993576905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e8009550 con 0x7fa4f8160540 2023-11-28T11:08:10.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4f7fff700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4f8160540 0x7fa4f80b0150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000efc0 tx=0x7fa4f000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:10.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4effff700 1 -- 172.21.15.156:0/2993576905 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000cb80 con 0x7fa4f8160540 2023-11-28T11:08:10.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4effff700 1 -- 172.21.15.156:0/2993576905 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f000f040 con 0x7fa4f8160540 2023-11-28T11:08:10.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4fdd55700 1 -- 172.21.15.156:0/2993576905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f815ccd0 con 0x7fa4f8160540 2023-11-28T11:08:10.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4effff700 1 -- 172.21.15.156:0/2993576905 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0014480 con 0x7fa4f8160540 2023-11-28T11:08:10.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.077+0000 7fa4fdd55700 1 -- 172.21.15.156:0/2993576905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f806e870 con 0x7fa4f8160540 2023-11-28T11:08:10.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.078+0000 7fa4fdd55700 1 -- 172.21.15.156:0/2993576905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f806a270 con 0x7fa4f8160540 2023-11-28T11:08:10.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.079+0000 7fa4effff700 1 -- 172.21.15.156:0/2993576905 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa4f00145e0 con 0x7fa4f8160540 2023-11-28T11:08:10.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.079+0000 7fa4effff700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4d80682f0 0x7fa4d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:10.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.079+0000 7fa4effff700 1 -- 172.21.15.156:0/2993576905 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa4f0086130 con 0x7fa4f8160540 2023-11-28T11:08:10.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.079+0000 7fa4f77fe700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4d80682f0 0x7fa4d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:10.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.080+0000 7fa4f77fe700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4d80682f0 0x7fa4d806a7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000f60 tx=0x7fa4e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.083+0000 7fa4effff700 1 -- 172.21.15.156:0/2993576905 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa4f0052e40 con 0x7fa4f8160540 2023-11-28T11:08:10.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.264+0000 7fa4fdd55700 1 -- 172.21.15.156:0/2993576905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fa4f80c5050 con 0x7fa4f8160540 2023-11-28T11:08:10.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.940+0000 7fa4effff700 1 -- 172.21.15.156:0/2993576905 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fa4f80c5050 con 0x7fa4f8160540 2023-11-28T11:08:10.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.942+0000 7fa4fdd55700 1 -- 172.21.15.156:0/2993576905 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4d80682f0 msgr2=0x7fa4d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:10.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.942+0000 7fa4fdd55700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4d80682f0 0x7fa4d806a7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000f60 tx=0x7fa4e0009040 comp rx=0 tx=0).stop 2023-11-28T11:08:10.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.943+0000 7fa4fdd55700 1 -- 172.21.15.156:0/2993576905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4f8160540 msgr2=0x7fa4f80b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:10.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.943+0000 7fa4fdd55700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4f8160540 0x7fa4f80b0150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000efc0 tx=0x7fa4f000c0c0 comp rx=0 tx=0).stop 2023-11-28T11:08:10.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.943+0000 7fa4fdd55700 1 -- 172.21.15.156:0/2993576905 shutdown_connections 2023-11-28T11:08:10.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.943+0000 7fa4fdd55700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4f8160540 0x7fa4f80b0150 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:10.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.943+0000 7fa4fdd55700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4d80682f0 0x7fa4d806a7a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:10.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.943+0000 7fa4fdd55700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4f806dee0 0x7fa4f814a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:10.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.943+0000 7fa4fdd55700 1 --2- 172.21.15.156:0/2993576905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4f8071040 0x7fa4f80b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:10.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.943+0000 7fa4fdd55700 1 -- 172.21.15.156:0/2993576905 >> 172.21.15.156:0/2993576905 conn(0x7fa4f80d0000 msgr2=0x7fa4f815f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:10.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.943+0000 7fa4fdd55700 1 -- 172.21.15.156:0/2993576905 shutdown_connections 2023-11-28T11:08:10.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:10.943+0000 7fa4fdd55700 1 -- 172.21.15.156:0/2993576905 wait complete. 2023-11-28T11:08:10.945 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'rbd' created 2023-11-28T11:08:10.958 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-28T11:08:11.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.084+0000 7faa80e40700 1 Processor -- start 2023-11-28T11:08:11.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.084+0000 7faa80e40700 1 -- start start 2023-11-28T11:08:11.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa80e40700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faa7c0cd910 0x7faa7c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:11.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa80e40700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c06b380 con 0x7faa7c06c4e0 2023-11-28T11:08:11.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa80e40700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c0ce220 con 0x7faa7c0cfef0 2023-11-28T11:08:11.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa80e40700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7c06b9b0 con 0x7faa7c0cd910 2023-11-28T11:08:11.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa7a59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faa7c0cd910 0x7faa7c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:11.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa7a59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faa7c0cd910 0x7faa7c0cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59792/0 (socket says 172.21.15.156:59792) 2023-11-28T11:08:11.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa7a59c700 1 -- 172.21.15.156:0/1874927294 learned_addr learned my addr 172.21.15.156:0/1874927294 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:11.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa7a59c700 1 -- 172.21.15.156:0/1874927294 >> v1:172.21.15.186:6789/0 conn(0x7faa7c0cfef0 legacy=0x7faa7c06bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:11.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa7a59c700 1 -- 172.21.15.156:0/1874927294 >> v1:172.21.15.156:6789/0 conn(0x7faa7c06c4e0 legacy=0x7faa7c15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:11.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa7959a700 1 -- 172.21.15.156:0/1874927294 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060314839 0 0) 0x7faa7c06b380 con 0x7faa7c06c4e0 2023-11-28T11:08:11.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa7a59c700 1 -- 172.21.15.156:0/1874927294 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c15de80 con 0x7faa7c0cd910 2023-11-28T11:08:11.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa7a59c700 1 --2- 172.21.15.156:0/1874927294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa7c0cd910 0x7faa7c0cdce0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faa640098f0 tx=0x7faa64009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd595a94f5313d7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:11.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa7959a700 1 -- 172.21.15.156:0/1874927294 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa64004230 con 0x7faa7c0cd910 2023-11-28T11:08:11.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa7959a700 1 -- 172.21.15.156:0/1874927294 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa64004390 con 0x7faa7c0cd910 2023-11-28T11:08:11.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.085+0000 7faa7959a700 1 -- 172.21.15.156:0/1874927294 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa640036a0 con 0x7faa7c0cd910 2023-11-28T11:08:11.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.086+0000 7faa80e40700 1 -- 172.21.15.156:0/1874927294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa7c0cd910 msgr2=0x7faa7c0cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:11.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.086+0000 7faa80e40700 1 --2- 172.21.15.156:0/1874927294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa7c0cd910 0x7faa7c0cdce0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faa640098f0 tx=0x7faa64009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:11.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.086+0000 7faa80e40700 1 -- 172.21.15.156:0/1874927294 shutdown_connections 2023-11-28T11:08:11.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.086+0000 7faa80e40700 1 --2- 172.21.15.156:0/1874927294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa7c0cd910 0x7faa7c0cdce0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:11.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.086+0000 7faa80e40700 1 -- 172.21.15.156:0/1874927294 >> 172.21.15.156:0/1874927294 conn(0x7faa7c06df70 msgr2=0x7faa7c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:11.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.086+0000 7faa80e40700 1 -- 172.21.15.156:0/1874927294 shutdown_connections 2023-11-28T11:08:11.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.086+0000 7faa80e40700 1 -- 172.21.15.156:0/1874927294 wait complete. 2023-11-28T11:08:11.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.088+0000 7faa80e40700 1 Processor -- start 2023-11-28T11:08:11.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.088+0000 7faa80e40700 1 -- start start 2023-11-28T11:08:11.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.088+0000 7faa80e40700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa7c06c4e0 0x7faa7c0afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:11.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa80e40700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa7c0cd910 0x7faa7c0c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:11.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa80e40700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa7c0cfef0 0x7faa7c0c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:11.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa80e40700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c15c820 con 0x7faa7c0cd910 2023-11-28T11:08:11.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa80e40700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0ad8b0 con 0x7faa7c06c4e0 2023-11-28T11:08:11.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa80e40700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c1e7c90 con 0x7faa7c0cfef0 2023-11-28T11:08:11.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa79d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa7c0cd910 0x7faa7c0c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:11.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa79d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa7c0cd910 0x7faa7c0c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59800/0 (socket says 172.21.15.156:59800) 2023-11-28T11:08:11.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa79d9b700 1 -- 172.21.15.156:0/1066563393 learned_addr learned my addr 172.21.15.156:0/1066563393 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:11.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa7a59c700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa7c06c4e0 0x7faa7c0afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:11.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa7ad9d700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa7c0cfef0 0x7faa7c0c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:11.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa79d9b700 1 -- 172.21.15.156:0/1066563393 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa7c0cfef0 msgr2=0x7faa7c0c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:11.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa79d9b700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa7c0cfef0 0x7faa7c0c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:11.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa79d9b700 1 -- 172.21.15.156:0/1066563393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa7c06c4e0 msgr2=0x7faa7c0afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:11.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa79d9b700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa7c06c4e0 0x7faa7c0afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:11.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa79d9b700 1 -- 172.21.15.156:0/1066563393 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa64009550 con 0x7faa7c0cd910 2023-11-28T11:08:11.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa7ad9d700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa7c0cfef0 0x7faa7c0c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:11.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa79d9b700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa7c0cd910 0x7faa7c0c6aa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00b4a0 tx=0x7faa6c00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:11.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa72ffd700 1 -- 172.21.15.156:0/1066563393 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c011830 con 0x7faa7c0cd910 2023-11-28T11:08:11.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa72ffd700 1 -- 172.21.15.156:0/1066563393 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c011990 con 0x7faa7c0cd910 2023-11-28T11:08:11.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa72ffd700 1 -- 172.21.15.156:0/1066563393 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c011d40 con 0x7faa7c0cd910 2023-11-28T11:08:11.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa80e40700 1 -- 172.21.15.156:0/1066563393 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7c06b5d0 con 0x7faa7c0cd910 2023-11-28T11:08:11.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.089+0000 7faa80e40700 1 -- 172.21.15.156:0/1066563393 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c15e160 con 0x7faa7c0cd910 2023-11-28T11:08:11.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.091+0000 7faa72ffd700 1 -- 172.21.15.156:0/1066563393 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faa6c00f050 con 0x7faa7c0cd910 2023-11-28T11:08:11.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.091+0000 7faa80e40700 1 -- 172.21.15.156:0/1066563393 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c0b04c0 con 0x7faa7c0cd910 2023-11-28T11:08:11.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.091+0000 7faa72ffd700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faa5c0682f0 0x7faa5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:11.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.091+0000 7faa72ffd700 1 -- 172.21.15.156:0/1066563393 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7faa6c085410 con 0x7faa7c0cd910 2023-11-28T11:08:11.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.091+0000 7faa7a59c700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faa5c0682f0 0x7faa5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:11.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.092+0000 7faa7a59c700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faa5c0682f0 0x7faa5c06a7a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faa64009130 tx=0x7faa64008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:11.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.096+0000 7faa72ffd700 1 -- 172.21.15.156:0/1066563393 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faa6c051f70 con 0x7faa7c0cd910 2023-11-28T11:08:11.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.281+0000 7faa80e40700 1 -- 172.21.15.156:0/1066563393 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faa7c068690 con 0x7faa7c0cd910 2023-11-28T11:08:11.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.944+0000 7faa72ffd700 1 -- 172.21.15.156:0/1066563393 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7faa6c055590 con 0x7faa7c0cd910 2023-11-28T11:08:11.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.946+0000 7faa80e40700 1 -- 172.21.15.156:0/1066563393 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faa5c0682f0 msgr2=0x7faa5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:11.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.946+0000 7faa80e40700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faa5c0682f0 0x7faa5c06a7a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faa64009130 tx=0x7faa64008030 comp rx=0 tx=0).stop 2023-11-28T11:08:11.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.946+0000 7faa80e40700 1 -- 172.21.15.156:0/1066563393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa7c0cd910 msgr2=0x7faa7c0c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:11.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.946+0000 7faa80e40700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa7c0cd910 0x7faa7c0c6aa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00b4a0 tx=0x7faa6c00b7b0 comp rx=0 tx=0).stop 2023-11-28T11:08:11.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.946+0000 7faa80e40700 1 -- 172.21.15.156:0/1066563393 shutdown_connections 2023-11-28T11:08:11.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.946+0000 7faa80e40700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faa5c0682f0 0x7faa5c06a7a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:11.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.946+0000 7faa80e40700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa7c0cfef0 0x7faa7c0c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:11.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.946+0000 7faa80e40700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa7c06c4e0 0x7faa7c0afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:11.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.946+0000 7faa80e40700 1 --2- 172.21.15.156:0/1066563393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa7c0cd910 0x7faa7c0c6aa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:11.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.946+0000 7faa80e40700 1 -- 172.21.15.156:0/1066563393 >> 172.21.15.156:0/1066563393 conn(0x7faa7c06df70 msgr2=0x7faa7c0b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:11.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.946+0000 7faa80e40700 1 -- 172.21.15.156:0/1066563393 shutdown_connections 2023-11-28T11:08:11.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:11.946+0000 7faa80e40700 1 -- 172.21.15.156:0/1066563393 wait complete. 2023-11-28T11:08:11.949 INFO:teuthology.orchestra.run.smithi156.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-28T11:08:11.962 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T11:08:11.962 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:08:11.963 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:08:11.967 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:08:11.967 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:08:11.967 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:08:11.969 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:08:11.969 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:08:11.970 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:08:11.972 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:08:11.972 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:08:11.972 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:08:11.974 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:08:11.974 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T11:08:11.975 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:12.047 INFO:tasks.ceph.mds.a.smithi156.stdout:starting mds.a at 2023-11-28T11:08:12.050 INFO:tasks.ceph.mds.c.smithi156.stdout:starting mds.c at 2023-11-28T11:08:12.062 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-11-28T11:08:12.062 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2023-11-28T11:08:12.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.104+0000 7fc986c72700 1 Processor -- start 2023-11-28T11:08:12.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.104+0000 7fc986c72700 1 -- start start 2023-11-28T11:08:12.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc986c72700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc980161340 0x7fc9800d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc986c72700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9800d1950 con 0x7fc98015c9b0 2023-11-28T11:08:12.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc986c72700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9800b5520 con 0x7fc9800d2320 2023-11-28T11:08:12.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc986c72700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9800b5800 con 0x7fc980161340 2023-11-28T11:08:12.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc984a0e700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc98015c9b0 0x7fc9800b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46478/0 (socket says 172.21.15.156:46478) 2023-11-28T11:08:12.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc984a0e700 1 -- 172.21.15.156:0/2881063482 learned_addr learned my addr 172.21.15.156:0/2881063482 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:12.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc97ffff700 1 --2- 172.21.15.156:0/2881063482 >> v2:172.21.15.186:3300/0 conn(0x7fc980161340 0x7fc9800d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc97f7fe700 1 -- 172.21.15.156:0/2881063482 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039651919 0 0) 0x7fc9800d1950 con 0x7fc98015c9b0 2023-11-28T11:08:12.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc97f7fe700 1 -- 172.21.15.156:0/2881063482 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9700035b0 con 0x7fc98015c9b0 2023-11-28T11:08:12.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc97f7fe700 1 -- 172.21.15.156:0/2881063482 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766739372 0 0) 0x7fc9800b5520 con 0x7fc9800d2320 2023-11-28T11:08:12.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc97f7fe700 1 -- 172.21.15.156:0/2881063482 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9800d1950 con 0x7fc9800d2320 2023-11-28T11:08:12.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc97f7fe700 1 -- 172.21.15.156:0/2881063482 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 202487229 0 0) 0x7fc9700035b0 con 0x7fc98015c9b0 2023-11-28T11:08:12.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc97ffff700 1 -- 172.21.15.156:0/2881063482 >> v1:172.21.15.186:6789/0 conn(0x7fc9800d2320 legacy=0x7fc9800cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc97ffff700 1 -- 172.21.15.156:0/2881063482 >> v1:172.21.15.156:6789/0 conn(0x7fc98015c9b0 legacy=0x7fc9800b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.105+0000 7fc97ffff700 1 -- 172.21.15.156:0/2881063482 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800d1300 con 0x7fc980161340 2023-11-28T11:08:12.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.106+0000 7fc97ffff700 1 --2- 172.21.15.156:0/2881063482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc980161340 0x7fc9800d1de0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009840 tx=0x7fc96c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e3ea3596bb2b71b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:12.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.106+0000 7fc97f7fe700 1 -- 172.21.15.156:0/2881063482 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974001670 con 0x7fc980161340 2023-11-28T11:08:12.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.106+0000 7fc97f7fe700 1 -- 172.21.15.156:0/2881063482 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c004030 con 0x7fc980161340 2023-11-28T11:08:12.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.106+0000 7fc97f7fe700 1 -- 172.21.15.156:0/2881063482 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0043c0 con 0x7fc980161340 2023-11-28T11:08:12.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.106+0000 7fc986c72700 1 -- 172.21.15.156:0/2881063482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc980161340 msgr2=0x7fc9800d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.106+0000 7fc986c72700 1 --2- 172.21.15.156:0/2881063482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc980161340 0x7fc9800d1de0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009840 tx=0x7fc96c009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:12.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.106+0000 7fc986c72700 1 -- 172.21.15.156:0/2881063482 shutdown_connections 2023-11-28T11:08:12.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.106+0000 7fc986c72700 1 --2- 172.21.15.156:0/2881063482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc980161340 0x7fc9800d1de0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.106+0000 7fc986c72700 1 -- 172.21.15.156:0/2881063482 >> 172.21.15.156:0/2881063482 conn(0x7fc980072600 msgr2=0x7fc9800cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:12.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.106+0000 7fc986c72700 1 -- 172.21.15.156:0/2881063482 shutdown_connections 2023-11-28T11:08:12.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.106+0000 7fc986c72700 1 -- 172.21.15.156:0/2881063482 wait complete. 2023-11-28T11:08:12.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.108+0000 7fc986c72700 1 Processor -- start 2023-11-28T11:08:12.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.108+0000 7fc986c72700 1 -- start start 2023-11-28T11:08:12.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.108+0000 7fc986c72700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9800d2320 0x7fc9801578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.108+0000 7fc986c72700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc98015c9b0 0x7fc9800c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.108+0000 7fc986c72700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc980161340 0x7fc9801f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.108+0000 7fc986c72700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9800c71b0 con 0x7fc98015c9b0 2023-11-28T11:08:12.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.108+0000 7fc986c72700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9801f2fe0 con 0x7fc980161340 2023-11-28T11:08:12.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.108+0000 7fc986c72700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9801f32c0 con 0x7fc9800d2320 2023-11-28T11:08:12.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc97ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc98015c9b0 0x7fc9800c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc97ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc98015c9b0 0x7fc9800c7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59840/0 (socket says 172.21.15.156:59840) 2023-11-28T11:08:12.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc97ffff700 1 -- 172.21.15.156:0/53474690 learned_addr learned my addr 172.21.15.156:0/53474690 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:12.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc984a0e700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9800d2320 0x7fc9801578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc98520f700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc980161340 0x7fc9801f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc97ffff700 1 -- 172.21.15.156:0/53474690 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9800d2320 msgr2=0x7fc9801578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc97ffff700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9800d2320 0x7fc9801578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc97ffff700 1 -- 172.21.15.156:0/53474690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc980161340 msgr2=0x7fc9801f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc97ffff700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc980161340 0x7fc9801f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc97ffff700 1 -- 172.21.15.156:0/53474690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c0094a0 con 0x7fc98015c9b0 2023-11-28T11:08:12.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc97ffff700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc98015c9b0 0x7fc9800c7640 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00a010 tx=0x7fc96c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:12.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc97d7fa700 1 -- 172.21.15.156:0/53474690 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c003c50 con 0x7fc98015c9b0 2023-11-28T11:08:12.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc97d7fa700 1 -- 172.21.15.156:0/53474690 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c01f070 con 0x7fc98015c9b0 2023-11-28T11:08:12.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc986c72700 1 -- 172.21.15.156:0/53474690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98015cd80 con 0x7fc98015c9b0 2023-11-28T11:08:12.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc97d7fa700 1 -- 172.21.15.156:0/53474690 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c011630 con 0x7fc98015c9b0 2023-11-28T11:08:12.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.109+0000 7fc986c72700 1 -- 172.21.15.156:0/53474690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9800d15e0 con 0x7fc98015c9b0 2023-11-28T11:08:12.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.110+0000 7fc986c72700 1 -- 172.21.15.156:0/53474690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98015cf10 con 0x7fc98015c9b0 2023-11-28T11:08:12.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.111+0000 7fc97d7fa700 1 -- 172.21.15.156:0/53474690 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc96c02a020 con 0x7fc98015c9b0 2023-11-28T11:08:12.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.111+0000 7fc97d7fa700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc9640682f0 0x7fc96406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.111+0000 7fc97d7fa700 1 -- 172.21.15.156:0/53474690 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc96c086cf0 con 0x7fc98015c9b0 2023-11-28T11:08:12.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.111+0000 7fc984a0e700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc9640682f0 0x7fc96406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.111+0000 7fc984a0e700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc9640682f0 0x7fc96406a7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc974000fd0 tx=0x7fc974000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:12.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.115+0000 7fc97d7fa700 1 -- 172.21.15.156:0/53474690 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc96c0538a0 con 0x7fc98015c9b0 2023-11-28T11:08:12.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.299+0000 7fc986c72700 1 -- 172.21.15.156:0/53474690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc98015cf10 con 0x7fc98015c9b0 2023-11-28T11:08:12.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.300+0000 7fc97d7fa700 1 -- 172.21.15.156:0/53474690 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14325 (secure 0 0 0) 0x7fc96c056ec0 con 0x7fc98015c9b0 2023-11-28T11:08:12.301 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:12.302 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":18,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:11.938594+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:12.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.302+0000 7fc986c72700 1 -- 172.21.15.156:0/53474690 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc9640682f0 msgr2=0x7fc96406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.302+0000 7fc986c72700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc9640682f0 0x7fc96406a7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc974000fd0 tx=0x7fc974000ed0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.302+0000 7fc986c72700 1 -- 172.21.15.156:0/53474690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc98015c9b0 msgr2=0x7fc9800c7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.302+0000 7fc986c72700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc98015c9b0 0x7fc9800c7640 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00a010 tx=0x7fc96c004750 comp rx=0 tx=0).stop 2023-11-28T11:08:12.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.302+0000 7fc986c72700 1 -- 172.21.15.156:0/53474690 shutdown_connections 2023-11-28T11:08:12.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.302+0000 7fc986c72700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc980161340 0x7fc9801f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.302+0000 7fc986c72700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc9640682f0 0x7fc96406a7a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.302+0000 7fc986c72700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9800d2320 0x7fc9801578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.302+0000 7fc986c72700 1 --2- 172.21.15.156:0/53474690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc98015c9b0 0x7fc9800c7640 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.302+0000 7fc986c72700 1 -- 172.21.15.156:0/53474690 >> 172.21.15.156:0/53474690 conn(0x7fc980072600 msgr2=0x7fc9800b6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:12.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.302+0000 7fc986c72700 1 -- 172.21.15.156:0/53474690 shutdown_connections 2023-11-28T11:08:12.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.302+0000 7fc986c72700 1 -- 172.21.15.156:0/53474690 wait complete. 2023-11-28T11:08:12.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:12.317 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:12.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.439+0000 7f32f7040700 1 Processor -- start 2023-11-28T11:08:12.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.439+0000 7f32f7040700 1 -- start start 2023-11-28T11:08:12.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32f7040700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0140950 0x7f32f00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32f7040700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f013ff00 con 0x7f32f00b2e70 2023-11-28T11:08:12.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32f7040700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f01404c0 con 0x7f32f00cd8c0 2023-11-28T11:08:12.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32f7040700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f0140dc0 con 0x7f32f0140950 2023-11-28T11:08:12.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32f55dd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f32f00b2e70 0x7f32f006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46496/0 (socket says 172.21.15.156:46496) 2023-11-28T11:08:12.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32f55dd700 1 -- 172.21.15.156:0/2451865676 learned_addr learned my addr 172.21.15.156:0/2451865676 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:12.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32effff700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0140950 0x7f32f00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290996391 0 0) 0x7f32f013ff00 con 0x7f32f00b2e70 2023-11-28T11:08:12.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e00035b0 con 0x7f32f00b2e70 2023-11-28T11:08:12.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441857714 0 0) 0x7f32f01404c0 con 0x7f32f00cd8c0 2023-11-28T11:08:12.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f013ff00 con 0x7f32f00cd8c0 2023-11-28T11:08:12.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2117696637 0 0) 0x7f32e00035b0 con 0x7f32f00b2e70 2023-11-28T11:08:12.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0140950 msgr2=0x7f32f00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:12.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0140950 0x7f32f00b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 >> v1:172.21.15.186:6789/0 conn(0x7f32f00cd8c0 legacy=0x7f32f00cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f015aa80 con 0x7f32f00b2e70 2023-11-28T11:08:12.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f32e80019f0 con 0x7f32f00b2e70 2023-11-28T11:08:12.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32effff700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0140950 0x7f32f00b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:12.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 >> v1:172.21.15.156:6789/0 conn(0x7f32f00b2e70 legacy=0x7f32f006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32e0003bc0 0x7f32e0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32e0004850 0x7f32e0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0140950 0x7f32e0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f0140dc0 con 0x7f32e0003bc0 2023-11-28T11:08:12.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e000bb60 con 0x7f32e0004850 2023-11-28T11:08:12.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e000bd20 con 0x7f32f0140950 2023-11-28T11:08:12.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32f4ddc700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32e0003bc0 0x7f32e0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32effff700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32e0004850 0x7f32e0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.440+0000 7f32f55dd700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0140950 0x7f32e0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.441+0000 7f32f4ddc700 1 -- 172.21.15.156:0/2451865676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0140950 msgr2=0x7f32e0005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:12.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.441+0000 7f32f4ddc700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0140950 0x7f32e0005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.441+0000 7f32f4ddc700 1 -- 172.21.15.156:0/2451865676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32e0004850 msgr2=0x7f32e0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:12.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.441+0000 7f32f4ddc700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32e0004850 0x7f32e0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.441+0000 7f32f4ddc700 1 -- 172.21.15.156:0/2451865676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc0094a0 con 0x7f32e0003bc0 2023-11-28T11:08:12.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.441+0000 7f32f4ddc700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32e0003bc0 0x7f32e0003fb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32e40019e0 tx=0x7f32e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d9c7b5162615e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:12.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.441+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e40014a0 con 0x7f32e0003bc0 2023-11-28T11:08:12.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.441+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e40052d0 con 0x7f32e0003bc0 2023-11-28T11:08:12.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.441+0000 7f32ef7fe700 1 -- 172.21.15.156:0/2451865676 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e40036d0 con 0x7f32e0003bc0 2023-11-28T11:08:12.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.441+0000 7f32f7040700 1 -- 172.21.15.156:0/2451865676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32e0003bc0 msgr2=0x7f32e0003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.441+0000 7f32f7040700 1 --2- 172.21.15.156:0/2451865676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32e0003bc0 0x7f32e0003fb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32e40019e0 tx=0x7f32e0005880 comp rx=0 tx=0).stop 2023-11-28T11:08:12.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.441+0000 7f32f55dd700 1 -- 172.21.15.156:0/2451865676 reap_dead start 2023-11-28T11:08:12.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.442+0000 7f32f7040700 1 -- 172.21.15.156:0/2451865676 shutdown_connections 2023-11-28T11:08:12.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.442+0000 7f32f7040700 1 -- 172.21.15.156:0/2451865676 >> 172.21.15.156:0/2451865676 conn(0x7f32f006aed0 msgr2=0x7f32f006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:12.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.442+0000 7f32f7040700 1 -- 172.21.15.156:0/2451865676 shutdown_connections 2023-11-28T11:08:12.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.442+0000 7f32f7040700 1 -- 172.21.15.156:0/2451865676 wait complete. 2023-11-28T11:08:12.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.443+0000 7f32f7040700 1 Processor -- start 2023-11-28T11:08:12.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.443+0000 7f32f7040700 1 -- start start 2023-11-28T11:08:12.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f7040700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32e0003bc0 0x7f32f00b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f7040700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f00c49e0 0x7f32f01f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f7040700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f013ccf0 0x7f32f013d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f7040700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f013d6a0 con 0x7f32f013ccf0 2023-11-28T11:08:12.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f7040700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f01f2260 con 0x7f32f00c49e0 2023-11-28T11:08:12.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f7040700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f006b520 con 0x7f32e0003bc0 2023-11-28T11:08:12.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f55dd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f013ccf0 0x7f32f013d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f55dd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f013ccf0 0x7f32f013d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59862/0 (socket says 172.21.15.156:59862) 2023-11-28T11:08:12.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f55dd700 1 -- 172.21.15.156:0/1629603190 learned_addr learned my addr 172.21.15.156:0/1629603190 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:12.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32effff700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f00c49e0 0x7f32f01f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f4ddc700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32e0003bc0 0x7f32f00b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f55dd700 1 -- 172.21.15.156:0/1629603190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32e0003bc0 msgr2=0x7f32f00b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f55dd700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32e0003bc0 0x7f32f00b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f55dd700 1 -- 172.21.15.156:0/1629603190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f00c49e0 msgr2=0x7f32f01f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f55dd700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f00c49e0 0x7f32f01f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f55dd700 1 -- 172.21.15.156:0/1629603190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f00b35f0 con 0x7f32f013ccf0 2023-11-28T11:08:12.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32effff700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f00c49e0 0x7f32f01f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:12.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f55dd700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f013ccf0 0x7f32f013d160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f32e8002230 tx=0x7f32e800a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:12.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32ed7fa700 1 -- 172.21.15.156:0/1629603190 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e800ad40 con 0x7f32f013ccf0 2023-11-28T11:08:12.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32ed7fa700 1 -- 172.21.15.156:0/1629603190 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e80133f0 con 0x7f32f013ccf0 2023-11-28T11:08:12.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f7040700 1 -- 172.21.15.156:0/1629603190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f00b3310 con 0x7f32f013ccf0 2023-11-28T11:08:12.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32ed7fa700 1 -- 172.21.15.156:0/1629603190 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e800aea0 con 0x7f32f013ccf0 2023-11-28T11:08:12.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.444+0000 7f32f7040700 1 -- 172.21.15.156:0/1629603190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f0140150 con 0x7f32f013ccf0 2023-11-28T11:08:12.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.445+0000 7f32f7040700 1 -- 172.21.15.156:0/1629603190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f0068690 con 0x7f32f013ccf0 2023-11-28T11:08:12.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.446+0000 7f32ed7fa700 1 -- 172.21.15.156:0/1629603190 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f32e8023070 con 0x7f32f013ccf0 2023-11-28T11:08:12.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.446+0000 7f32ed7fa700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f32d40682f0 0x7f32d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.446+0000 7f32ed7fa700 1 -- 172.21.15.156:0/1629603190 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f32e8086400 con 0x7f32f013ccf0 2023-11-28T11:08:12.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.446+0000 7f32f4ddc700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f32d40682f0 0x7f32d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.446+0000 7f32f4ddc700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f32d40682f0 0x7f32d406a7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f32e4008fd0 tx=0x7f32e400a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:12.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.450+0000 7f32ed7fa700 1 -- 172.21.15.156:0/1629603190 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f32e8052f50 con 0x7f32f013ccf0 2023-11-28T11:08:12.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.632+0000 7f32f7040700 1 -- 172.21.15.156:0/1629603190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32f015a3a0 con 0x7f32f013ccf0 2023-11-28T11:08:12.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.633+0000 7f32ed7fa700 1 -- 172.21.15.156:0/1629603190 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32e80170a0 con 0x7f32f013ccf0 2023-11-28T11:08:12.634 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:08:12.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.634+0000 7f32f7040700 1 -- 172.21.15.156:0/1629603190 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f32d40682f0 msgr2=0x7f32d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.634+0000 7f32f7040700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f32d40682f0 0x7f32d406a7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f32e4008fd0 tx=0x7f32e400a000 comp rx=0 tx=0).stop 2023-11-28T11:08:12.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.634+0000 7f32f7040700 1 -- 172.21.15.156:0/1629603190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f013ccf0 msgr2=0x7f32f013d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.634+0000 7f32f7040700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f013ccf0 0x7f32f013d160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f32e8002230 tx=0x7f32e800a510 comp rx=0 tx=0).stop 2023-11-28T11:08:12.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.635+0000 7f32f7040700 1 -- 172.21.15.156:0/1629603190 shutdown_connections 2023-11-28T11:08:12.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.635+0000 7f32f7040700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f013ccf0 0x7f32f013d160 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.635+0000 7f32f7040700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f32d40682f0 0x7f32d406a7a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.635+0000 7f32f7040700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32e0003bc0 0x7f32f00b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.635+0000 7f32f7040700 1 --2- 172.21.15.156:0/1629603190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f00c49e0 0x7f32f01f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.635+0000 7f32f7040700 1 -- 172.21.15.156:0/1629603190 >> 172.21.15.156:0/1629603190 conn(0x7f32f006aed0 msgr2=0x7f32f013db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.635+0000 7f32f7040700 1 -- 172.21.15.156:0/1629603190 shutdown_connections 2023-11-28T11:08:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.635+0000 7f32f7040700 1 -- 172.21.15.156:0/1629603190 wait complete. 2023-11-28T11:08:12.648 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:12.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.770+0000 7fc5f03ee700 1 Processor -- start 2023-11-28T11:08:12.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.770+0000 7fc5f03ee700 1 -- start start 2023-11-28T11:08:12.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5f03ee700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5e80cd8c0 0x7fc5e8140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5f03ee700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc5e80cdc90 0x7fc5e80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5f03ee700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e813ff00 con 0x7fc5e80b2e70 2023-11-28T11:08:12.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5f03ee700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e81404c0 con 0x7fc5e80cdc90 2023-11-28T11:08:12.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5f03ee700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e815aa80 con 0x7fc5e80cd8c0 2023-11-28T11:08:12.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5ee98b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc5e80b2e70 0x7fc5e806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46516/0 (socket says 172.21.15.156:46516) 2023-11-28T11:08:12.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5ee98b700 1 -- 172.21.15.156:0/247601001 learned_addr learned my addr 172.21.15.156:0/247601001 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:12.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5ee18a700 1 --2- 172.21.15.156:0/247601001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5e80cd8c0 0x7fc5e8140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5ed989700 1 --2- 172.21.15.156:0/247601001 >> v2:172.21.15.186:3300/0 conn(0x7fc5e80cdc90 0x7fc5e80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5ed188700 1 -- 172.21.15.156:0/247601001 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380654041 0 0) 0x7fc5e813ff00 con 0x7fc5e80b2e70 2023-11-28T11:08:12.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5ed188700 1 -- 172.21.15.156:0/247601001 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5d40035b0 con 0x7fc5e80b2e70 2023-11-28T11:08:12.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5ee18a700 1 -- 172.21.15.156:0/247601001 >> v2:172.21.15.186:3300/0 conn(0x7fc5e80cdc90 msgr2=0x7fc5e80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:12.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5ee18a700 1 --2- 172.21.15.156:0/247601001 >> v2:172.21.15.186:3300/0 conn(0x7fc5e80cdc90 0x7fc5e80b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5ee18a700 1 -- 172.21.15.156:0/247601001 >> v1:172.21.15.156:6789/0 conn(0x7fc5e80b2e70 legacy=0x7fc5e806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5ee18a700 1 -- 172.21.15.156:0/247601001 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e80b3310 con 0x7fc5e80cd8c0 2023-11-28T11:08:12.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.771+0000 7fc5ee18a700 1 --2- 172.21.15.156:0/247601001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5e80cd8c0 0x7fc5e8140950 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009820 tx=0x7fc5e4009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3915eb003797ed80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:12.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.772+0000 7fc5ed188700 1 -- 172.21.15.156:0/247601001 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4005e30 con 0x7fc5e80cd8c0 2023-11-28T11:08:12.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.772+0000 7fc5ed188700 1 -- 172.21.15.156:0/247601001 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e4006030 con 0x7fc5e80cd8c0 2023-11-28T11:08:12.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.772+0000 7fc5ed188700 1 -- 172.21.15.156:0/247601001 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40063c0 con 0x7fc5e80cd8c0 2023-11-28T11:08:12.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.772+0000 7fc5f03ee700 1 -- 172.21.15.156:0/247601001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5e80cd8c0 msgr2=0x7fc5e8140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.772+0000 7fc5f03ee700 1 --2- 172.21.15.156:0/247601001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5e80cd8c0 0x7fc5e8140950 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009820 tx=0x7fc5e4009b30 comp rx=0 tx=0).stop 2023-11-28T11:08:12.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.772+0000 7fc5f03ee700 1 -- 172.21.15.156:0/247601001 shutdown_connections 2023-11-28T11:08:12.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.772+0000 7fc5f03ee700 1 --2- 172.21.15.156:0/247601001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5e80cd8c0 0x7fc5e8140950 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.772+0000 7fc5f03ee700 1 --2- 172.21.15.156:0/247601001 >> v2:172.21.15.186:3300/0 conn(0x7fc5e80cdc90 0x7fc5e80b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.772+0000 7fc5f03ee700 1 -- 172.21.15.156:0/247601001 >> 172.21.15.156:0/247601001 conn(0x7fc5e806aed0 msgr2=0x7fc5e806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:12.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.772+0000 7fc5f03ee700 1 -- 172.21.15.156:0/247601001 shutdown_connections 2023-11-28T11:08:12.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.772+0000 7fc5f03ee700 1 -- 172.21.15.156:0/247601001 wait complete. 2023-11-28T11:08:12.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.773+0000 7fc5f03ee700 1 Processor -- start 2023-11-28T11:08:12.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.773+0000 7fc5f03ee700 1 -- start start 2023-11-28T11:08:12.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.774+0000 7fc5f03ee700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e80b2e70 0x7fc5e81551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.774+0000 7fc5f03ee700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5e80cd8c0 0x7fc5e813f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.774+0000 7fc5f03ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5e80cdc90 0x7fc5e80c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.774+0000 7fc5f03ee700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e813f2a0 con 0x7fc5e80cdc90 2023-11-28T11:08:12.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.774+0000 7fc5f03ee700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e81556f0 con 0x7fc5e80b2e70 2023-11-28T11:08:12.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.774+0000 7fc5f03ee700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e813fc00 con 0x7fc5e80cd8c0 2023-11-28T11:08:12.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.774+0000 7fc5ee98b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5e80cdc90 0x7fc5e80c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.774+0000 7fc5ee18a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e80b2e70 0x7fc5e81551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.774+0000 7fc5ee18a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e80b2e70 0x7fc5e81551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:42654/0 (socket says 172.21.15.156:42654) 2023-11-28T11:08:12.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.774+0000 7fc5ee98b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5e80cdc90 0x7fc5e80c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59876/0 (socket says 172.21.15.156:59876) 2023-11-28T11:08:12.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.774+0000 7fc5ee18a700 1 -- 172.21.15.156:0/817788455 learned_addr learned my addr 172.21.15.156:0/817788455 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:12.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.774+0000 7fc5ed989700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5e80cd8c0 0x7fc5e813f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.775+0000 7fc5ee98b700 1 -- 172.21.15.156:0/817788455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5e80cd8c0 msgr2=0x7fc5e813f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.775+0000 7fc5ee98b700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5e80cd8c0 0x7fc5e813f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.775+0000 7fc5ee98b700 1 -- 172.21.15.156:0/817788455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e80b2e70 msgr2=0x7fc5e81551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.775+0000 7fc5ee98b700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e80b2e70 0x7fc5e81551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.775+0000 7fc5ee98b700 1 -- 172.21.15.156:0/817788455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e4009430 con 0x7fc5e80cdc90 2023-11-28T11:08:12.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.775+0000 7fc5ee18a700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e80b2e70 0x7fc5e81551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:12.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.775+0000 7fc5ee98b700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5e80cdc90 0x7fc5e80c9350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000ca80 tx=0x7fc5e000cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:12.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.775+0000 7fc5deffd700 1 -- 172.21.15.156:0/817788455 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e000a9a0 con 0x7fc5e80cdc90 2023-11-28T11:08:12.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.775+0000 7fc5deffd700 1 -- 172.21.15.156:0/817788455 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e000ab00 con 0x7fc5e80cdc90 2023-11-28T11:08:12.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.775+0000 7fc5f03ee700 1 -- 172.21.15.156:0/817788455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e8140150 con 0x7fc5e80cdc90 2023-11-28T11:08:12.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.775+0000 7fc5deffd700 1 -- 172.21.15.156:0/817788455 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e000ae70 con 0x7fc5e80cdc90 2023-11-28T11:08:12.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.775+0000 7fc5f03ee700 1 -- 172.21.15.156:0/817788455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e80b35f0 con 0x7fc5e80cdc90 2023-11-28T11:08:12.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.776+0000 7fc5f03ee700 1 -- 172.21.15.156:0/817788455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e80ce100 con 0x7fc5e80cdc90 2023-11-28T11:08:12.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.776+0000 7fc5deffd700 1 -- 172.21.15.156:0/817788455 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc5e0019030 con 0x7fc5e80cdc90 2023-11-28T11:08:12.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.777+0000 7fc5deffd700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc5cc0682f0 0x7fc5cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:12.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.777+0000 7fc5ee18a700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc5cc0682f0 0x7fc5cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:12.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.777+0000 7fc5deffd700 1 -- 172.21.15.156:0/817788455 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc5e00852b0 con 0x7fc5e80cdc90 2023-11-28T11:08:12.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.777+0000 7fc5ee18a700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc5cc0682f0 0x7fc5cc06a7a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4007fd0 tx=0x7fc5e4008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:12.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.781+0000 7fc5deffd700 1 -- 172.21.15.156:0/817788455 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc5e004b5e0 con 0x7fc5e80cdc90 2023-11-28T11:08:12.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.963+0000 7fc5f03ee700 1 -- 172.21.15.156:0/817788455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc5e813f3e0 con 0x7fc5e80cdc90 2023-11-28T11:08:12.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.964+0000 7fc5deffd700 1 -- 172.21.15.156:0/817788455 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc5e813f3e0 con 0x7fc5e80cdc90 2023-11-28T11:08:12.965 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:08:12.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.966+0000 7fc5f03ee700 1 -- 172.21.15.156:0/817788455 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc5cc0682f0 msgr2=0x7fc5cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.966+0000 7fc5f03ee700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc5cc0682f0 0x7fc5cc06a7a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4007fd0 tx=0x7fc5e4008070 comp rx=0 tx=0).stop 2023-11-28T11:08:12.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.966+0000 7fc5f03ee700 1 -- 172.21.15.156:0/817788455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5e80cdc90 msgr2=0x7fc5e80c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:12.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.966+0000 7fc5f03ee700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5e80cdc90 0x7fc5e80c9350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000ca80 tx=0x7fc5e000cd90 comp rx=0 tx=0).stop 2023-11-28T11:08:12.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.966+0000 7fc5f03ee700 1 -- 172.21.15.156:0/817788455 shutdown_connections 2023-11-28T11:08:12.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.966+0000 7fc5f03ee700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5e80cdc90 0x7fc5e80c9350 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.966+0000 7fc5f03ee700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5e80b2e70 0x7fc5e81551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.966+0000 7fc5f03ee700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc5cc0682f0 0x7fc5cc06a7a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.966+0000 7fc5f03ee700 1 --2- 172.21.15.156:0/817788455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5e80cd8c0 0x7fc5e813f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:12.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.966+0000 7fc5f03ee700 1 -- 172.21.15.156:0/817788455 >> 172.21.15.156:0/817788455 conn(0x7fc5e806aed0 msgr2=0x7fc5e80b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:12.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.966+0000 7fc5f03ee700 1 -- 172.21.15.156:0/817788455 shutdown_connections 2023-11-28T11:08:12.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:12.966+0000 7fc5f03ee700 1 -- 172.21.15.156:0/817788455 wait complete. 2023-11-28T11:08:12.979 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:08:13.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.100+0000 7fdd2620d700 1 Processor -- start 2023-11-28T11:08:13.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.100+0000 7fdd2620d700 1 -- start start 2023-11-28T11:08:13.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd2620d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdd2015c9b0 0x7fdd200b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd2620d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd200cd7c0 0x7fdd200cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd2620d700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd200d1950 con 0x7fdd20161340 2023-11-28T11:08:13.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd2620d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd200b5800 con 0x7fdd2015c9b0 2023-11-28T11:08:13.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd2620d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd200ce140 con 0x7fdd200cd7c0 2023-11-28T11:08:13.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd1f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdd2015c9b0 0x7fdd200b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd1f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdd2015c9b0 0x7fdd200b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59882/0 (socket says 172.21.15.156:59882) 2023-11-28T11:08:13.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd1f7fe700 1 -- 172.21.15.156:0/1977034875 learned_addr learned my addr 172.21.15.156:0/1977034875 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:13.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd1ffff700 1 --2- 172.21.15.156:0/1977034875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd200cd7c0 0x7fdd200cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd1f7fe700 1 -- 172.21.15.156:0/1977034875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd200cd7c0 msgr2=0x7fdd200cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:13.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd1f7fe700 1 --2- 172.21.15.156:0/1977034875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd200cd7c0 0x7fdd200cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd1f7fe700 1 -- 172.21.15.156:0/1977034875 >> v1:172.21.15.186:6789/0 conn(0x7fdd20161340 legacy=0x7fdd200d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd1f7fe700 1 -- 172.21.15.156:0/1977034875 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd200d24f0 con 0x7fdd2015c9b0 2023-11-28T11:08:13.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd1ffff700 1 --2- 172.21.15.156:0/1977034875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd200cd7c0 0x7fdd200cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:13.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd1f7fe700 1 --2- 172.21.15.156:0/1977034875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd2015c9b0 0x7fdd200b4bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdd100098f0 tx=0x7fdd10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f942298fe8688570 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:13.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd1e7fc700 1 -- 172.21.15.156:0/1977034875 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000b8a0 con 0x7fdd2015c9b0 2023-11-28T11:08:13.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.101+0000 7fdd1e7fc700 1 -- 172.21.15.156:0/1977034875 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1000ba00 con 0x7fdd2015c9b0 2023-11-28T11:08:13.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.102+0000 7fdd1e7fc700 1 -- 172.21.15.156:0/1977034875 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000bd90 con 0x7fdd2015c9b0 2023-11-28T11:08:13.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.102+0000 7fdd2620d700 1 -- 172.21.15.156:0/1977034875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd2015c9b0 msgr2=0x7fdd200b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.102+0000 7fdd2620d700 1 --2- 172.21.15.156:0/1977034875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd2015c9b0 0x7fdd200b4bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdd100098f0 tx=0x7fdd10009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.102+0000 7fdd2620d700 1 -- 172.21.15.156:0/1977034875 shutdown_connections 2023-11-28T11:08:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.102+0000 7fdd2620d700 1 --2- 172.21.15.156:0/1977034875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd200cd7c0 0x7fdd200cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.102+0000 7fdd2620d700 1 --2- 172.21.15.156:0/1977034875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd2015c9b0 0x7fdd200b4bf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.102+0000 7fdd2620d700 1 -- 172.21.15.156:0/1977034875 >> 172.21.15.156:0/1977034875 conn(0x7fdd20072600 msgr2=0x7fdd200cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:13.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.102+0000 7fdd2620d700 1 -- 172.21.15.156:0/1977034875 shutdown_connections 2023-11-28T11:08:13.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.102+0000 7fdd2620d700 1 -- 172.21.15.156:0/1977034875 wait complete. 2023-11-28T11:08:13.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.103+0000 7fdd2620d700 1 Processor -- start 2023-11-28T11:08:13.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.103+0000 7fdd2620d700 1 -- start start 2023-11-28T11:08:13.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.104+0000 7fdd2620d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd200cd7c0 0x7fdd200c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.104+0000 7fdd2620d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd2015c9b0 0x7fdd2014a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.104+0000 7fdd2620d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd20161340 0x7fdd201e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.104+0000 7fdd2620d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd200c8690 con 0x7fdd20161340 2023-11-28T11:08:13.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.104+0000 7fdd2620d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd200c8970 con 0x7fdd200cd7c0 2023-11-28T11:08:13.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.104+0000 7fdd2620d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd200c8c50 con 0x7fdd2015c9b0 2023-11-28T11:08:13.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.104+0000 7fdd1ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd20161340 0x7fdd201e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.104+0000 7fdd1ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd20161340 0x7fdd201e6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59892/0 (socket says 172.21.15.156:59892) 2023-11-28T11:08:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.104+0000 7fdd1ffff700 1 -- 172.21.15.156:0/2585530802 learned_addr learned my addr 172.21.15.156:0/2585530802 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.104+0000 7fdd1effd700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd2015c9b0 0x7fdd2014a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.104+0000 7fdd1f7fe700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd200cd7c0 0x7fdd200c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.105+0000 7fdd1ffff700 1 -- 172.21.15.156:0/2585530802 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd2015c9b0 msgr2=0x7fdd2014a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.105+0000 7fdd1ffff700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd2015c9b0 0x7fdd2014a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.105+0000 7fdd1ffff700 1 -- 172.21.15.156:0/2585530802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd200cd7c0 msgr2=0x7fdd200c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.105+0000 7fdd1ffff700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd200cd7c0 0x7fdd200c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.105+0000 7fdd1ffff700 1 -- 172.21.15.156:0/2585530802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd10009550 con 0x7fdd20161340 2023-11-28T11:08:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.105+0000 7fdd1ffff700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd20161340 0x7fdd201e6440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800c7f0 tx=0x7fdd1800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.105+0000 7fdd17fff700 1 -- 172.21.15.156:0/2585530802 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd18011800 con 0x7fdd20161340 2023-11-28T11:08:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.105+0000 7fdd17fff700 1 -- 172.21.15.156:0/2585530802 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd18011980 con 0x7fdd20161340 2023-11-28T11:08:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.105+0000 7fdd2620d700 1 -- 172.21.15.156:0/2585530802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd200d1300 con 0x7fdd20161340 2023-11-28T11:08:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.105+0000 7fdd17fff700 1 -- 172.21.15.156:0/2585530802 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd18011d30 con 0x7fdd20161340 2023-11-28T11:08:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.105+0000 7fdd2620d700 1 -- 172.21.15.156:0/2585530802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd200b5520 con 0x7fdd20161340 2023-11-28T11:08:13.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.106+0000 7fdd2620d700 1 -- 172.21.15.156:0/2585530802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd200d15e0 con 0x7fdd20161340 2023-11-28T11:08:13.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.106+0000 7fdd17fff700 1 -- 172.21.15.156:0/2585530802 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdd18021070 con 0x7fdd20161340 2023-11-28T11:08:13.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.107+0000 7fdd17fff700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd000682f0 0x7fdd0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.107+0000 7fdd17fff700 1 -- 172.21.15.156:0/2585530802 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdd180856e0 con 0x7fdd20161340 2023-11-28T11:08:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.107+0000 7fdd1f7fe700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd000682f0 0x7fdd0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.107+0000 7fdd1f7fe700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd000682f0 0x7fdd0006a7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdd10014040 tx=0x7fdd10008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.111+0000 7fdd17fff700 1 -- 172.21.15.156:0/2585530802 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdd18052200 con 0x7fdd20161340 2023-11-28T11:08:13.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.335+0000 7fdd2620d700 1 -- 172.21.15.156:0/2585530802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd20066d60 con 0x7fdd20161340 2023-11-28T11:08:13.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.336+0000 7fdd17fff700 1 -- 172.21.15.156:0/2585530802 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7fdd1801f020 con 0x7fdd20161340 2023-11-28T11:08:13.337 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:13.337 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1221397150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1221397150},{"type":"v1","addr":"172.21.15.186:6834","nonce":1221397150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/22494485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":22494485},{"type":"v1","addr":"172.21.15.156:6835","nonce":22494485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3382922708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3382922708},{"type":"v1","addr":"172.21.15.156:6837","nonce":3382922708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4134790300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":4134790300},{"type":"v1","addr":"172.21.15.186:6835","nonce":4134790300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-28T11:08:13.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.338+0000 7fdd2620d700 1 -- 172.21.15.156:0/2585530802 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd000682f0 msgr2=0x7fdd0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.338+0000 7fdd2620d700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd000682f0 0x7fdd0006a7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdd10014040 tx=0x7fdd10008030 comp rx=0 tx=0).stop 2023-11-28T11:08:13.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.338+0000 7fdd2620d700 1 -- 172.21.15.156:0/2585530802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd20161340 msgr2=0x7fdd201e6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.338+0000 7fdd2620d700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd20161340 0x7fdd201e6440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800c7f0 tx=0x7fdd1800cb00 comp rx=0 tx=0).stop 2023-11-28T11:08:13.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.338+0000 7fdd2620d700 1 -- 172.21.15.156:0/2585530802 shutdown_connections 2023-11-28T11:08:13.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.338+0000 7fdd2620d700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd20161340 0x7fdd201e6440 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.338+0000 7fdd2620d700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd200cd7c0 0x7fdd200c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.338+0000 7fdd2620d700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd000682f0 0x7fdd0006a7a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.338+0000 7fdd2620d700 1 --2- 172.21.15.156:0/2585530802 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd2015c9b0 0x7fdd2014a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.338+0000 7fdd2620d700 1 -- 172.21.15.156:0/2585530802 >> 172.21.15.156:0/2585530802 conn(0x7fdd20072600 msgr2=0x7fdd200b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:13.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.338+0000 7fdd2620d700 1 -- 172.21.15.156:0/2585530802 shutdown_connections 2023-11-28T11:08:13.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.338+0000 7fdd2620d700 1 -- 172.21.15.156:0/2585530802 wait complete. 2023-11-28T11:08:13.341 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 3 2023-11-28T11:08:13.352 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:13.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.471+0000 7faf65dde700 1 Processor -- start 2023-11-28T11:08:13.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.471+0000 7faf65dde700 1 -- start start 2023-11-28T11:08:13.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.471+0000 7faf65dde700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7faf6006cdc0 0x7faf6006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.471+0000 7faf65dde700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf600cee00 con 0x7faf60071040 2023-11-28T11:08:13.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.471+0000 7faf65dde700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6006d9f0 con 0x7faf60161160 2023-11-28T11:08:13.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.471+0000 7faf65dde700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf600c38b0 con 0x7faf6006cdc0 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.471+0000 7faf5f7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7faf60071040 0x7faf60071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46542/0 (socket says 172.21.15.156:46542) 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5f7fe700 1 -- 172.21.15.156:0/2469567099 learned_addr learned my addr 172.21.15.156:0/2469567099 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538257920 0 0) 0x7faf600cee00 con 0x7faf60071040 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5ffff700 1 --2- 172.21.15.156:0/2469567099 >> v2:172.21.15.186:3300/0 conn(0x7faf6006cdc0 0x7faf6006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf480035b0 con 0x7faf60071040 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1500677205 0 0) 0x7faf480035b0 con 0x7faf60071040 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 >> v2:172.21.15.186:3300/0 conn(0x7faf6006cdc0 msgr2=0x7faf6006d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 --2- 172.21.15.156:0/2469567099 >> v2:172.21.15.186:3300/0 conn(0x7faf6006cdc0 0x7faf6006d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 >> v1:172.21.15.186:6789/0 conn(0x7faf60161160 legacy=0x7faf600cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600cf070 con 0x7faf60071040 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7faf50001670 con 0x7faf60071040 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 >> v1:172.21.15.156:6789/0 conn(0x7faf60071040 legacy=0x7faf60071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 --2- 172.21.15.156:0/2469567099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf48003a00 0x7faf48003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 --2- 172.21.15.156:0/2469567099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf48004850 0x7faf48004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 --2- 172.21.15.156:0/2469567099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf48005200 0x7faf4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf600c38b0 con 0x7faf48005200 2023-11-28T11:08:13.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4800c080 con 0x7faf48003a00 2023-11-28T11:08:13.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4800c240 con 0x7faf48004850 2023-11-28T11:08:13.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5ffff700 1 --2- 172.21.15.156:0/2469567099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf48005200 0x7faf4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5f7fe700 1 --2- 172.21.15.156:0/2469567099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf48003a00 0x7faf48003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5effd700 1 --2- 172.21.15.156:0/2469567099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf48004850 0x7faf48004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5ffff700 1 -- 172.21.15.156:0/2469567099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf48004850 msgr2=0x7faf48004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:13.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5ffff700 1 --2- 172.21.15.156:0/2469567099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf48004850 0x7faf48004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5ffff700 1 -- 172.21.15.156:0/2469567099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf48003a00 msgr2=0x7faf48003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:13.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5ffff700 1 --2- 172.21.15.156:0/2469567099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf48003a00 0x7faf48003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.472+0000 7faf5ffff700 1 -- 172.21.15.156:0/2469567099 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf480035b0 con 0x7faf48005200 2023-11-28T11:08:13.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.473+0000 7faf5ffff700 1 -- 172.21.15.156:0/2469567099 reap_dead start 2023-11-28T11:08:13.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.473+0000 7faf5ffff700 1 --2- 172.21.15.156:0/2469567099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf48005200 0x7faf4800ba10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf58009fc0 tx=0x7faf48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d622de8fbe14935 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:13.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.473+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58003a20 con 0x7faf48005200 2023-11-28T11:08:13.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.473+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5800f040 con 0x7faf48005200 2023-11-28T11:08:13.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.473+0000 7faf5e7fc700 1 -- 172.21.15.156:0/2469567099 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58003de0 con 0x7faf48005200 2023-11-28T11:08:13.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.473+0000 7faf65dde700 1 -- 172.21.15.156:0/2469567099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf48005200 msgr2=0x7faf4800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.473+0000 7faf65dde700 1 --2- 172.21.15.156:0/2469567099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf48005200 0x7faf4800ba10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf58009fc0 tx=0x7faf48005880 comp rx=0 tx=0).stop 2023-11-28T11:08:13.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.473+0000 7faf65dde700 1 -- 172.21.15.156:0/2469567099 shutdown_connections 2023-11-28T11:08:13.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.473+0000 7faf65dde700 1 --2- 172.21.15.156:0/2469567099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf48005200 0x7faf4800ba10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.473+0000 7faf65dde700 1 -- 172.21.15.156:0/2469567099 >> 172.21.15.156:0/2469567099 conn(0x7faf600d0000 msgr2=0x7faf600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:13.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.473+0000 7faf65dde700 1 -- 172.21.15.156:0/2469567099 shutdown_connections 2023-11-28T11:08:13.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.473+0000 7faf65dde700 1 -- 172.21.15.156:0/2469567099 wait complete. 2023-11-28T11:08:13.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.474+0000 7faf65dde700 1 Processor -- start 2023-11-28T11:08:13.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.475+0000 7faf65dde700 1 -- start start 2023-11-28T11:08:13.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.475+0000 7faf65dde700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf48005200 0x7faf60160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.475+0000 7faf65dde700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf600b4f50 0x7faf600c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.475+0000 7faf65dde700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf601533a0 0x7faf600acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.475+0000 7faf65dde700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf600b4680 con 0x7faf600b4f50 2023-11-28T11:08:13.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.475+0000 7faf65dde700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf600b4960 con 0x7faf601533a0 2023-11-28T11:08:13.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.475+0000 7faf65dde700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf600b4c40 con 0x7faf48005200 2023-11-28T11:08:13.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.475+0000 7faf5effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf600b4f50 0x7faf600c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf5effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf600b4f50 0x7faf600c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59908/0 (socket says 172.21.15.156:59908) 2023-11-28T11:08:13.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf5effd700 1 -- 172.21.15.156:0/2695806131 learned_addr learned my addr 172.21.15.156:0/2695806131 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:13.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf5f7fe700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf48005200 0x7faf60160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf5ffff700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf601533a0 0x7faf600acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf5effd700 1 -- 172.21.15.156:0/2695806131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf48005200 msgr2=0x7faf60160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf5effd700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf48005200 0x7faf60160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf5effd700 1 -- 172.21.15.156:0/2695806131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf601533a0 msgr2=0x7faf600acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf5effd700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf601533a0 0x7faf600acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf5effd700 1 -- 172.21.15.156:0/2695806131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6006dcd0 con 0x7faf600b4f50 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf5ffff700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf601533a0 0x7faf600acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf5effd700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf600b4f50 0x7faf600c42d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf5400f7f0 tx=0x7faf5400fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf47fff700 1 -- 172.21.15.156:0/2695806131 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540052e0 con 0x7faf600b4f50 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf47fff700 1 -- 172.21.15.156:0/2695806131 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf54005440 con 0x7faf600b4f50 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf65dde700 1 -- 172.21.15.156:0/2695806131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf600cf350 con 0x7faf600b4f50 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf47fff700 1 -- 172.21.15.156:0/2695806131 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540038a0 con 0x7faf600b4f50 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.476+0000 7faf65dde700 1 -- 172.21.15.156:0/2695806131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf60071410 con 0x7faf600b4f50 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.477+0000 7faf65dde700 1 -- 172.21.15.156:0/2695806131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf600c4120 con 0x7faf600b4f50 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.477+0000 7faf47fff700 1 -- 172.21.15.156:0/2695806131 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faf54003a00 con 0x7faf600b4f50 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.478+0000 7faf47fff700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faf4c0682f0 0x7faf4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.478+0000 7faf47fff700 1 -- 172.21.15.156:0/2695806131 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faf540858b0 con 0x7faf600b4f50 2023-11-28T11:08:13.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.478+0000 7faf5f7fe700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faf4c0682f0 0x7faf4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.478+0000 7faf5f7fe700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faf4c0682f0 0x7faf4c06a7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf50007cd0 tx=0x7faf50009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:13.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.482+0000 7faf47fff700 1 -- 172.21.15.156:0/2695806131 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faf54052460 con 0x7faf600b4f50 2023-11-28T11:08:13.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.667+0000 7faf65dde700 1 -- 172.21.15.156:0/2695806131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf600b5f80 con 0x7faf600b4f50 2023-11-28T11:08:13.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.668+0000 7faf47fff700 1 -- 172.21.15.156:0/2695806131 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14312 (secure 0 0 0) 0x7faf54055a80 con 0x7faf600b4f50 2023-11-28T11:08:13.669 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:13.670 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":19,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:12.943946+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.669+0000 7faf65dde700 1 -- 172.21.15.156:0/2695806131 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faf4c0682f0 msgr2=0x7faf4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.669+0000 7faf65dde700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faf4c0682f0 0x7faf4c06a7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf50007cd0 tx=0x7faf50009040 comp rx=0 tx=0).stop 2023-11-28T11:08:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.670+0000 7faf65dde700 1 -- 172.21.15.156:0/2695806131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf600b4f50 msgr2=0x7faf600c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.670+0000 7faf65dde700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf600b4f50 0x7faf600c42d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf5400f7f0 tx=0x7faf5400fbb0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.670+0000 7faf65dde700 1 -- 172.21.15.156:0/2695806131 shutdown_connections 2023-11-28T11:08:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.670+0000 7faf65dde700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf601533a0 0x7faf600acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.670+0000 7faf65dde700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faf4c0682f0 0x7faf4c06a7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.670+0000 7faf65dde700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf48005200 0x7faf60160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.670+0000 7faf65dde700 1 --2- 172.21.15.156:0/2695806131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf600b4f50 0x7faf600c42d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.670+0000 7faf65dde700 1 -- 172.21.15.156:0/2695806131 >> 172.21.15.156:0/2695806131 conn(0x7faf600d0000 msgr2=0x7faf600ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.670+0000 7faf65dde700 1 -- 172.21.15.156:0/2695806131 shutdown_connections 2023-11-28T11:08:13.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.670+0000 7faf65dde700 1 -- 172.21.15.156:0/2695806131 wait complete. 2023-11-28T11:08:13.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:13.684 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:13.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.806+0000 7f70fb6f7700 1 Processor -- start 2023-11-28T11:08:13.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.806+0000 7f70fb6f7700 1 -- start start 2023-11-28T11:08:13.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70fb6f7700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f70f40cd8c0 0x7f70f4140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70fb6f7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70f40cdc90 0x7f70f40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70fb6f7700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f413ff00 con 0x7f70f40b2e70 2023-11-28T11:08:13.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70fb6f7700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f41404c0 con 0x7f70f40cd8c0 2023-11-28T11:08:13.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70fb6f7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f415aa80 con 0x7f70f40cdc90 2023-11-28T11:08:13.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70f9c94700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f70f40b2e70 0x7f70f406f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46564/0 (socket says 172.21.15.156:46564) 2023-11-28T11:08:13.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70f9c94700 1 -- 172.21.15.156:0/3268741277 learned_addr learned my addr 172.21.15.156:0/3268741277 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:13.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70f9493700 1 --2- 172.21.15.156:0/3268741277 >> v2:172.21.15.186:3300/0 conn(0x7f70f40cd8c0 0x7f70f4140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70f8c92700 1 --2- 172.21.15.156:0/3268741277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70f40cdc90 0x7f70f40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70ebfff700 1 -- 172.21.15.156:0/3268741277 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348674787 0 0) 0x7f70f413ff00 con 0x7f70f40b2e70 2023-11-28T11:08:13.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70ebfff700 1 -- 172.21.15.156:0/3268741277 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70e00035b0 con 0x7f70f40b2e70 2023-11-28T11:08:13.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70f9493700 1 -- 172.21.15.156:0/3268741277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70f40cdc90 msgr2=0x7f70f40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:13.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70f9493700 1 --2- 172.21.15.156:0/3268741277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70f40cdc90 0x7f70f40b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70f9493700 1 -- 172.21.15.156:0/3268741277 >> v1:172.21.15.156:6789/0 conn(0x7f70f40b2e70 legacy=0x7f70f406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70f9493700 1 -- 172.21.15.156:0/3268741277 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f40b3310 con 0x7f70f40cd8c0 2023-11-28T11:08:13.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.807+0000 7f70f9493700 1 --2- 172.21.15.156:0/3268741277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f40cd8c0 0x7f70f4140950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009820 tx=0x7f70f0009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3486a9b517c5667f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:13.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.808+0000 7f70ebfff700 1 -- 172.21.15.156:0/3268741277 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0005e30 con 0x7f70f40cd8c0 2023-11-28T11:08:13.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.808+0000 7f70ebfff700 1 -- 172.21.15.156:0/3268741277 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f0006030 con 0x7f70f40cd8c0 2023-11-28T11:08:13.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.808+0000 7f70ebfff700 1 -- 172.21.15.156:0/3268741277 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f00063c0 con 0x7f70f40cd8c0 2023-11-28T11:08:13.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.808+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3268741277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f40cd8c0 msgr2=0x7f70f4140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.808+0000 7f70fb6f7700 1 --2- 172.21.15.156:0/3268741277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f40cd8c0 0x7f70f4140950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009820 tx=0x7f70f0009b30 comp rx=0 tx=0).stop 2023-11-28T11:08:13.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.808+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3268741277 shutdown_connections 2023-11-28T11:08:13.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.808+0000 7f70fb6f7700 1 --2- 172.21.15.156:0/3268741277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f40cd8c0 0x7f70f4140950 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.808+0000 7f70fb6f7700 1 --2- 172.21.15.156:0/3268741277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70f40cdc90 0x7f70f40b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.808+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3268741277 >> 172.21.15.156:0/3268741277 conn(0x7f70f406aed0 msgr2=0x7f70f406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:13.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.808+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3268741277 shutdown_connections 2023-11-28T11:08:13.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.808+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3268741277 wait complete. 2023-11-28T11:08:13.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.809+0000 7f70fb6f7700 1 Processor -- start 2023-11-28T11:08:13.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.809+0000 7f70fb6f7700 1 -- start start 2023-11-28T11:08:13.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.810+0000 7f70fb6f7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f40b2e70 0x7f70f40ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.810+0000 7f70fb6f7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40cd8c0 0x7f70f41e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.810+0000 7f70fb6f7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70f40cdc90 0x7f70f41e48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.810+0000 7f70fb6f7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f406cf10 con 0x7f70f40cd8c0 2023-11-28T11:08:13.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.810+0000 7f70fb6f7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f406d3a0 con 0x7f70f40b2e70 2023-11-28T11:08:13.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.810+0000 7f70fb6f7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40c73b0 con 0x7f70f40cdc90 2023-11-28T11:08:13.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.810+0000 7f70f8c92700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40cd8c0 0x7f70f41e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.810+0000 7f70f8c92700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40cd8c0 0x7f70f41e43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59924/0 (socket says 172.21.15.156:59924) 2023-11-28T11:08:13.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.810+0000 7f70f8c92700 1 -- 172.21.15.156:0/3152786908 learned_addr learned my addr 172.21.15.156:0/3152786908 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:13.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.810+0000 7f70f9c94700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70f40cdc90 0x7f70f41e48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.810+0000 7f70f9493700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f40b2e70 0x7f70f40ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70f8c92700 1 -- 172.21.15.156:0/3152786908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70f40cdc90 msgr2=0x7f70f41e48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70f8c92700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70f40cdc90 0x7f70f41e48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70f8c92700 1 -- 172.21.15.156:0/3152786908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f40b2e70 msgr2=0x7f70f40ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:13.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70f8c92700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f40b2e70 0x7f70f40ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:13.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70f8c92700 1 -- 172.21.15.156:0/3152786908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f0009430 con 0x7f70f40cd8c0 2023-11-28T11:08:13.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70f9c94700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70f40cdc90 0x7f70f41e48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:13.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70f9493700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f40b2e70 0x7f70f40ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:13.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70f8c92700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40cd8c0 0x7f70f41e43b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70e400e840 tx=0x7f70e400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:13.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70e9ffb700 1 -- 172.21.15.156:0/3152786908 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4009590 con 0x7f70f40cd8c0 2023-11-28T11:08:13.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70e9ffb700 1 -- 172.21.15.156:0/3152786908 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e4013680 con 0x7f70f40cd8c0 2023-11-28T11:08:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70e9ffb700 1 -- 172.21.15.156:0/3152786908 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4013a10 con 0x7f70f40cd8c0 2023-11-28T11:08:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3152786908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f4140150 con 0x7f70f40cd8c0 2023-11-28T11:08:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.811+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3152786908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f40b35f0 con 0x7f70f40cd8c0 2023-11-28T11:08:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.812+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3152786908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f40ce100 con 0x7f70f40cd8c0 2023-11-28T11:08:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.812+0000 7f70e9ffb700 1 -- 172.21.15.156:0/3152786908 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f70e4019030 con 0x7f70f40cd8c0 2023-11-28T11:08:13.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.813+0000 7f70e9ffb700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f70d80682f0 0x7f70d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:13.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.813+0000 7f70f9493700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f70d80682f0 0x7f70d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:13.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.813+0000 7f70e9ffb700 1 -- 172.21.15.156:0/3152786908 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f70e4085860 con 0x7f70f40cd8c0 2023-11-28T11:08:13.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.813+0000 7f70f9493700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f70d80682f0 0x7f70d806a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f70f0000c00 tx=0x7f70f0008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:13.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:13.817+0000 7f70e9ffb700 1 -- 172.21.15.156:0/3152786908 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f70e40522c0 con 0x7f70f40cd8c0 2023-11-28T11:08:14.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.001+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3152786908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f70f4068690 con 0x7f70f40cd8c0 2023-11-28T11:08:14.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.002+0000 7f70e9ffb700 1 -- 172.21.15.156:0/3152786908 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f70e40558e0 con 0x7f70f40cd8c0 2023-11-28T11:08:14.003 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:08:14.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.004+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3152786908 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f70d80682f0 msgr2=0x7f70d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.004+0000 7f70fb6f7700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f70d80682f0 0x7f70d806a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f70f0000c00 tx=0x7f70f0008070 comp rx=0 tx=0).stop 2023-11-28T11:08:14.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.004+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3152786908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40cd8c0 msgr2=0x7f70f41e43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.004+0000 7f70fb6f7700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40cd8c0 0x7f70f41e43b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70e400e840 tx=0x7f70e400ec00 comp rx=0 tx=0).stop 2023-11-28T11:08:14.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.004+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3152786908 shutdown_connections 2023-11-28T11:08:14.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.004+0000 7f70fb6f7700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f70d80682f0 0x7f70d806a7a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.004+0000 7f70fb6f7700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70f40cdc90 0x7f70f41e48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.004+0000 7f70fb6f7700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f40b2e70 0x7f70f40ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.004+0000 7f70fb6f7700 1 --2- 172.21.15.156:0/3152786908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40cd8c0 0x7f70f41e43b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.004+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3152786908 >> 172.21.15.156:0/3152786908 conn(0x7f70f406aed0 msgr2=0x7f70f40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:14.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.004+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3152786908 shutdown_connections 2023-11-28T11:08:14.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.004+0000 7f70fb6f7700 1 -- 172.21.15.156:0/3152786908 wait complete. 2023-11-28T11:08:14.017 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:14.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.141+0000 7f310338e700 1 Processor -- start 2023-11-28T11:08:14.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.141+0000 7f310338e700 1 -- start start 2023-11-28T11:08:14.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f310338e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f30fc06ba90 0x7f30fc0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f310338e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30fc0cbdf0 0x7f30fc0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f310338e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30fc0cc5f0 con 0x7f30fc15cef0 2023-11-28T11:08:14.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f310338e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc15d470 con 0x7f30fc06ba90 2023-11-28T11:08:14.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f310338e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc15d750 con 0x7f30fc0cbdf0 2023-11-28T11:08:14.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f3100929700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f30fc06ba90 0x7f30fc0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f310112a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f30fc15cef0 0x7f30fc06b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46576/0 (socket says 172.21.15.156:46576) 2023-11-28T11:08:14.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f310112a700 1 -- 172.21.15.156:0/3240783055 learned_addr learned my addr 172.21.15.156:0/3240783055 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:14.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f310192b700 1 --2- 172.21.15.156:0/3240783055 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30fc0cbdf0 0x7f30fc0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f3100929700 1 -- 172.21.15.156:0/3240783055 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30fc0cbdf0 msgr2=0x7f30fc0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:14.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f3100929700 1 --2- 172.21.15.156:0/3240783055 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30fc0cbdf0 0x7f30fc0ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f3100929700 1 -- 172.21.15.156:0/3240783055 >> v1:172.21.15.156:6789/0 conn(0x7f30fc15cef0 legacy=0x7f30fc06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f3100929700 1 -- 172.21.15.156:0/3240783055 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc15da30 con 0x7f30fc06ba90 2023-11-28T11:08:14.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f3100929700 1 --2- 172.21.15.156:0/3240783055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30fc06ba90 0x7f30fc0af400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009840 tx=0x7f30f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec75ad6c92135b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:14.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f30f3fff700 1 -- 172.21.15.156:0/3240783055 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f8011830 con 0x7f30fc06ba90 2023-11-28T11:08:14.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f30f3fff700 1 -- 172.21.15.156:0/3240783055 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f8011990 con 0x7f30fc06ba90 2023-11-28T11:08:14.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.142+0000 7f30f3fff700 1 -- 172.21.15.156:0/3240783055 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f8011d20 con 0x7f30fc06ba90 2023-11-28T11:08:14.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.143+0000 7f310338e700 1 -- 172.21.15.156:0/3240783055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30fc06ba90 msgr2=0x7f30fc0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.143+0000 7f310338e700 1 --2- 172.21.15.156:0/3240783055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30fc06ba90 0x7f30fc0af400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009840 tx=0x7f30f8009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:14.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.143+0000 7f310338e700 1 -- 172.21.15.156:0/3240783055 shutdown_connections 2023-11-28T11:08:14.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.143+0000 7f310338e700 1 --2- 172.21.15.156:0/3240783055 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30fc0cbdf0 0x7f30fc0ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.143+0000 7f310338e700 1 --2- 172.21.15.156:0/3240783055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30fc06ba90 0x7f30fc0af400 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.143+0000 7f310338e700 1 -- 172.21.15.156:0/3240783055 >> 172.21.15.156:0/3240783055 conn(0x7f30fc0d0000 msgr2=0x7f30fc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:14.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.143+0000 7f310338e700 1 -- 172.21.15.156:0/3240783055 shutdown_connections 2023-11-28T11:08:14.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.143+0000 7f310338e700 1 -- 172.21.15.156:0/3240783055 wait complete. 2023-11-28T11:08:14.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.144+0000 7f310338e700 1 Processor -- start 2023-11-28T11:08:14.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.144+0000 7f310338e700 1 -- start start 2023-11-28T11:08:14.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310338e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30fc06ba90 0x7f30fc153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310338e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30fc0cbdf0 0x7f30fc06a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310338e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30fc15cef0 0x7f30fc06a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310338e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0c71e0 con 0x7f30fc06ba90 2023-11-28T11:08:14.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310338e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0c74c0 con 0x7f30fc15cef0 2023-11-28T11:08:14.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310338e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0c77a0 con 0x7f30fc0cbdf0 2023-11-28T11:08:14.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310112a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30fc06ba90 0x7f30fc153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310112a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30fc06ba90 0x7f30fc153440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59936/0 (socket says 172.21.15.156:59936) 2023-11-28T11:08:14.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310112a700 1 -- 172.21.15.156:0/2417681210 learned_addr learned my addr 172.21.15.156:0/2417681210 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:14.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310192b700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30fc15cef0 0x7f30fc06a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310112a700 1 -- 172.21.15.156:0/2417681210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30fc0cbdf0 msgr2=0x7f30fc06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310112a700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30fc0cbdf0 0x7f30fc06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310112a700 1 -- 172.21.15.156:0/2417681210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30fc15cef0 msgr2=0x7f30fc06a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310112a700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30fc15cef0 0x7f30fc06a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310112a700 1 -- 172.21.15.156:0/2417681210 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f80094a0 con 0x7f30fc06ba90 2023-11-28T11:08:14.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.145+0000 7f310112a700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30fc06ba90 0x7f30fc153440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00c480 tx=0x7f30ec00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:14.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.146+0000 7f30f1ffb700 1 -- 172.21.15.156:0/2417681210 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec005d70 con 0x7f30fc06ba90 2023-11-28T11:08:14.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.146+0000 7f30f1ffb700 1 -- 172.21.15.156:0/2417681210 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ec0070d0 con 0x7f30fc06ba90 2023-11-28T11:08:14.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.146+0000 7f310338e700 1 -- 172.21.15.156:0/2417681210 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc15dff0 con 0x7f30fc06ba90 2023-11-28T11:08:14.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.146+0000 7f30f1ffb700 1 -- 172.21.15.156:0/2417681210 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec007440 con 0x7f30fc06ba90 2023-11-28T11:08:14.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.146+0000 7f310338e700 1 -- 172.21.15.156:0/2417681210 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc15dd10 con 0x7f30fc06ba90 2023-11-28T11:08:14.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.147+0000 7f310338e700 1 -- 172.21.15.156:0/2417681210 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc0cab90 con 0x7f30fc06ba90 2023-11-28T11:08:14.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.147+0000 7f30f1ffb700 1 -- 172.21.15.156:0/2417681210 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f30ec019030 con 0x7f30fc06ba90 2023-11-28T11:08:14.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.147+0000 7f30f1ffb700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f30e00682f0 0x7f30e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.147+0000 7f30f1ffb700 1 -- 172.21.15.156:0/2417681210 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f30ec084c10 con 0x7f30fc06ba90 2023-11-28T11:08:14.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.147+0000 7f3100929700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f30e00682f0 0x7f30e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.148+0000 7f3100929700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f30e00682f0 0x7f30e006a7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f30fc06afa0 tx=0x7f30f8010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:14.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.151+0000 7f30f1ffb700 1 -- 172.21.15.156:0/2417681210 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f30ec04b5e0 con 0x7f30fc06ba90 2023-11-28T11:08:14.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.332+0000 7f310338e700 1 -- 172.21.15.156:0/2417681210 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f30fc0c5630 con 0x7f30fc06ba90 2023-11-28T11:08:14.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.332+0000 7f30f1ffb700 1 -- 172.21.15.156:0/2417681210 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f30ec0170a0 con 0x7f30fc06ba90 2023-11-28T11:08:14.334 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:08:14.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.335+0000 7f310338e700 1 -- 172.21.15.156:0/2417681210 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f30e00682f0 msgr2=0x7f30e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.335+0000 7f310338e700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f30e00682f0 0x7f30e006a7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f30fc06afa0 tx=0x7f30f8010000 comp rx=0 tx=0).stop 2023-11-28T11:08:14.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.335+0000 7f310338e700 1 -- 172.21.15.156:0/2417681210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30fc06ba90 msgr2=0x7f30fc153440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.335+0000 7f310338e700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30fc06ba90 0x7f30fc153440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00c480 tx=0x7f30ec00c840 comp rx=0 tx=0).stop 2023-11-28T11:08:14.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.335+0000 7f310338e700 1 -- 172.21.15.156:0/2417681210 shutdown_connections 2023-11-28T11:08:14.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.335+0000 7f310338e700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30fc15cef0 0x7f30fc06a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.335+0000 7f310338e700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30fc06ba90 0x7f30fc153440 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.335+0000 7f310338e700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f30e00682f0 0x7f30e006a7a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.335+0000 7f310338e700 1 --2- 172.21.15.156:0/2417681210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30fc0cbdf0 0x7f30fc06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.335+0000 7f310338e700 1 -- 172.21.15.156:0/2417681210 >> 172.21.15.156:0/2417681210 conn(0x7f30fc0d0000 msgr2=0x7f30fc15c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:14.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.335+0000 7f310338e700 1 -- 172.21.15.156:0/2417681210 shutdown_connections 2023-11-28T11:08:14.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.335+0000 7f310338e700 1 -- 172.21.15.156:0/2417681210 wait complete. 2023-11-28T11:08:14.348 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:08:14.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.467+0000 7fa17d691700 1 Processor -- start 2023-11-28T11:08:14.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.467+0000 7fa17d691700 1 -- start start 2023-11-28T11:08:14.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.468+0000 7fa17d691700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1780cd8c0 0x7fa178140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.468+0000 7fa17d691700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa17806f850 0x7fa17806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.468+0000 7fa17d691700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17813ff00 con 0x7fa1780cdc90 2023-11-28T11:08:14.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.468+0000 7fa17d691700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1781404c0 con 0x7fa17806f850 2023-11-28T11:08:14.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.468+0000 7fa17d691700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17815aa80 con 0x7fa1780cd8c0 2023-11-28T11:08:14.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.468+0000 7fa1777fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa17806f850 0x7fa17806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.468+0000 7fa1767fc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa1780cdc90 0x7fa1780b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46590/0 (socket says 172.21.15.156:46590) 2023-11-28T11:08:14.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.468+0000 7fa1767fc700 1 -- 172.21.15.156:0/653618958 learned_addr learned my addr 172.21.15.156:0/653618958 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:14.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.469+0000 7fa176ffd700 1 --2- 172.21.15.156:0/653618958 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1780cd8c0 0x7fa178140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.469+0000 7fa1777fe700 1 -- 172.21.15.156:0/653618958 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1780cd8c0 msgr2=0x7fa178140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:14.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.469+0000 7fa1777fe700 1 --2- 172.21.15.156:0/653618958 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1780cd8c0 0x7fa178140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.469+0000 7fa1777fe700 1 -- 172.21.15.156:0/653618958 >> v1:172.21.15.156:6789/0 conn(0x7fa1780cdc90 legacy=0x7fa1780b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.469+0000 7fa1777fe700 1 -- 172.21.15.156:0/653618958 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1780b3040 con 0x7fa17806f850 2023-11-28T11:08:14.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.469+0000 7fa175ffb700 1 -- 172.21.15.156:0/653618958 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962658397 0 0) 0x7fa17813ff00 con 0x7fa1780cdc90 2023-11-28T11:08:14.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.469+0000 7fa1777fe700 1 --2- 172.21.15.156:0/653618958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17806f850 0x7fa17806fcc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa170009d90 tx=0x7fa17000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70eb7a3323d2cd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:14.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.469+0000 7fa175ffb700 1 -- 172.21.15.156:0/653618958 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17000e070 con 0x7fa17806f850 2023-11-28T11:08:14.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.469+0000 7fa175ffb700 1 -- 172.21.15.156:0/653618958 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17000bbd0 con 0x7fa17806f850 2023-11-28T11:08:14.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.469+0000 7fa175ffb700 1 -- 172.21.15.156:0/653618958 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170012430 con 0x7fa17806f850 2023-11-28T11:08:14.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.469+0000 7fa17d691700 1 -- 172.21.15.156:0/653618958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17806f850 msgr2=0x7fa17806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.469+0000 7fa17d691700 1 --2- 172.21.15.156:0/653618958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17806f850 0x7fa17806fcc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa170009d90 tx=0x7fa17000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:14.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.470+0000 7fa17d691700 1 -- 172.21.15.156:0/653618958 shutdown_connections 2023-11-28T11:08:14.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.470+0000 7fa17d691700 1 --2- 172.21.15.156:0/653618958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17806f850 0x7fa17806fcc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.470+0000 7fa17d691700 1 --2- 172.21.15.156:0/653618958 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1780cd8c0 0x7fa178140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.470+0000 7fa17d691700 1 -- 172.21.15.156:0/653618958 >> 172.21.15.156:0/653618958 conn(0x7fa17806aed0 msgr2=0x7fa17806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:14.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.470+0000 7fa17d691700 1 -- 172.21.15.156:0/653618958 shutdown_connections 2023-11-28T11:08:14.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.470+0000 7fa17d691700 1 -- 172.21.15.156:0/653618958 wait complete. 2023-11-28T11:08:14.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.471+0000 7fa17d691700 1 Processor -- start 2023-11-28T11:08:14.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.471+0000 7fa17d691700 1 -- start start 2023-11-28T11:08:14.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa17d691700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17806f850 0x7fa178154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa17d691700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1780cd8c0 0x7fa1780c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa17d691700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1780cdc90 0x7fa1780b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa17d691700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17806b8c0 con 0x7fa17806f850 2023-11-28T11:08:14.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa17d691700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17806c440 con 0x7fa1780cd8c0 2023-11-28T11:08:14.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa17d691700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa178155dd0 con 0x7fa1780cdc90 2023-11-28T11:08:14.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa176ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17806f850 0x7fa178154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa176ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17806f850 0x7fa178154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59950/0 (socket says 172.21.15.156:59950) 2023-11-28T11:08:14.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa176ffd700 1 -- 172.21.15.156:0/2843167610 learned_addr learned my addr 172.21.15.156:0/2843167610 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:14.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa1767fc700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1780cd8c0 0x7fa1780c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa1777fe700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1780cdc90 0x7fa1780b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa176ffd700 1 -- 172.21.15.156:0/2843167610 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1780cdc90 msgr2=0x7fa1780b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa176ffd700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1780cdc90 0x7fa1780b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa176ffd700 1 -- 172.21.15.156:0/2843167610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1780cd8c0 msgr2=0x7fa1780c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa176ffd700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1780cd8c0 0x7fa1780c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa176ffd700 1 -- 172.21.15.156:0/2843167610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa170009a70 con 0x7fa17806f850 2023-11-28T11:08:14.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa176ffd700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17806f850 0x7fa178154e20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00c820 tx=0x7fa16c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:14.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa15f7fe700 1 -- 172.21.15.156:0/2843167610 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c011800 con 0x7fa17806f850 2023-11-28T11:08:14.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa15f7fe700 1 -- 172.21.15.156:0/2843167610 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c011960 con 0x7fa17806f850 2023-11-28T11:08:14.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa17d691700 1 -- 172.21.15.156:0/2843167610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1780b35f0 con 0x7fa17806f850 2023-11-28T11:08:14.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa15f7fe700 1 -- 172.21.15.156:0/2843167610 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c011cf0 con 0x7fa17806f850 2023-11-28T11:08:14.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.472+0000 7fa17d691700 1 -- 172.21.15.156:0/2843167610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1780b3310 con 0x7fa17806f850 2023-11-28T11:08:14.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.473+0000 7fa17d691700 1 -- 172.21.15.156:0/2843167610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa178068690 con 0x7fa17806f850 2023-11-28T11:08:14.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.474+0000 7fa15f7fe700 1 -- 172.21.15.156:0/2843167610 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa16c021070 con 0x7fa17806f850 2023-11-28T11:08:14.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.474+0000 7fa15f7fe700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa1640682f0 0x7fa16406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.474+0000 7fa1767fc700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa1640682f0 0x7fa16406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.474+0000 7fa15f7fe700 1 -- 172.21.15.156:0/2843167610 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa16c0857a0 con 0x7fa17806f850 2023-11-28T11:08:14.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.475+0000 7fa1767fc700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa1640682f0 0x7fa16406a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa1781f5a90 tx=0x7fa168007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:14.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.478+0000 7fa15f7fe700 1 -- 172.21.15.156:0/2843167610 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa16c052350 con 0x7fa17806f850 2023-11-28T11:08:14.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.664+0000 7fa17d691700 1 -- 172.21.15.156:0/2843167610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa1781400d0 con 0x7fa17806f850 2023-11-28T11:08:14.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.664+0000 7fa15f7fe700 1 -- 172.21.15.156:0/2843167610 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa16c01f090 con 0x7fa17806f850 2023-11-28T11:08:14.666 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:14.666 INFO:teuthology.orchestra.run.smithi156.stdout:[ 2023-11-28T11:08:14.666 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:08:14.666 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 1, 2023-11-28T11:08:14.666 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": ".mgr" 2023-11-28T11:08:14.666 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:08:14.666 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:08:14.666 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 2, 2023-11-28T11:08:14.666 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": "rbd" 2023-11-28T11:08:14.666 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:08:14.666 INFO:teuthology.orchestra.run.smithi156.stdout:] 2023-11-28T11:08:14.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.666+0000 7fa17d691700 1 -- 172.21.15.156:0/2843167610 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa1640682f0 msgr2=0x7fa16406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.666+0000 7fa17d691700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa1640682f0 0x7fa16406a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa1781f5a90 tx=0x7fa168007c90 comp rx=0 tx=0).stop 2023-11-28T11:08:14.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.666+0000 7fa17d691700 1 -- 172.21.15.156:0/2843167610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17806f850 msgr2=0x7fa178154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.666+0000 7fa17d691700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17806f850 0x7fa178154e20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00c820 tx=0x7fa16c00cbe0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.666+0000 7fa17d691700 1 -- 172.21.15.156:0/2843167610 shutdown_connections 2023-11-28T11:08:14.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.666+0000 7fa17d691700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa1640682f0 0x7fa16406a7a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.666+0000 7fa17d691700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1780cdc90 0x7fa1780b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.666+0000 7fa17d691700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17806f850 0x7fa178154e20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.666+0000 7fa17d691700 1 --2- 172.21.15.156:0/2843167610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1780cd8c0 0x7fa1780c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.666+0000 7fa17d691700 1 -- 172.21.15.156:0/2843167610 >> 172.21.15.156:0/2843167610 conn(0x7fa17806aed0 msgr2=0x7fa1780cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:14.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.666+0000 7fa17d691700 1 -- 172.21.15.156:0/2843167610 shutdown_connections 2023-11-28T11:08:14.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.666+0000 7fa17d691700 1 -- 172.21.15.156:0/2843167610 wait complete. 2023-11-28T11:08:14.680 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:08:14.680 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:08:14.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.803+0000 7f3f3d438700 1 Processor -- start 2023-11-28T11:08:14.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.804+0000 7f3f3d438700 1 -- start start 2023-11-28T11:08:14.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.804+0000 7f3f3d438700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f380cc260 0x7f3f38071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.804+0000 7f3f3d438700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f380cc7b0 con 0x7f3f380d0470 2023-11-28T11:08:14.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.804+0000 7f3f3d438700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f380cd610 con 0x7f3f381596b0 2023-11-28T11:08:14.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.804+0000 7f3f3d438700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f38159a80 con 0x7f3f380cc260 2023-11-28T11:08:14.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.804+0000 7f3f367fc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3f380d0470 0x7f3f3806cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46604/0 (socket says 172.21.15.156:46604) 2023-11-28T11:08:14.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.804+0000 7f3f367fc700 1 -- 172.21.15.156:0/974892694 learned_addr learned my addr 172.21.15.156:0/974892694 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:14.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f377fe700 1 --2- 172.21.15.156:0/974892694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f380cc260 0x7f3f38071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f35ffb700 1 -- 172.21.15.156:0/974892694 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971578440 0 0) 0x7f3f380cc7b0 con 0x7f3f380d0470 2023-11-28T11:08:14.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f35ffb700 1 -- 172.21.15.156:0/974892694 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f200035b0 con 0x7f3f380d0470 2023-11-28T11:08:14.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f35ffb700 1 -- 172.21.15.156:0/974892694 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121553619 0 0) 0x7f3f380cd610 con 0x7f3f381596b0 2023-11-28T11:08:14.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f35ffb700 1 -- 172.21.15.156:0/974892694 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f380cc7b0 con 0x7f3f381596b0 2023-11-28T11:08:14.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f377fe700 1 -- 172.21.15.156:0/974892694 >> v1:172.21.15.186:6789/0 conn(0x7f3f381596b0 legacy=0x7f3f380cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f35ffb700 1 -- 172.21.15.156:0/974892694 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2352444036 0 0) 0x7f3f200035b0 con 0x7f3f380d0470 2023-11-28T11:08:14.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f377fe700 1 -- 172.21.15.156:0/974892694 >> v1:172.21.15.156:6789/0 conn(0x7f3f380d0470 legacy=0x7f3f3806cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f377fe700 1 -- 172.21.15.156:0/974892694 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f380cca00 con 0x7f3f380cc260 2023-11-28T11:08:14.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f377fe700 1 --2- 172.21.15.156:0/974892694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f380cc260 0x7f3f38071410 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009dc0 tx=0x7f3f3000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2329db20153b5e9e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:14.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f35ffb700 1 -- 172.21.15.156:0/974892694 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3000e070 con 0x7f3f380cc260 2023-11-28T11:08:14.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f35ffb700 1 -- 172.21.15.156:0/974892694 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f30003a20 con 0x7f3f380cc260 2023-11-28T11:08:14.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.805+0000 7f3f35ffb700 1 -- 172.21.15.156:0/974892694 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28001670 con 0x7f3f380cc260 2023-11-28T11:08:14.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.806+0000 7f3f3d438700 1 -- 172.21.15.156:0/974892694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f380cc260 msgr2=0x7f3f38071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.806+0000 7f3f3d438700 1 --2- 172.21.15.156:0/974892694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f380cc260 0x7f3f38071410 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009dc0 tx=0x7f3f3000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:14.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.806+0000 7f3f3d438700 1 -- 172.21.15.156:0/974892694 shutdown_connections 2023-11-28T11:08:14.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.806+0000 7f3f3d438700 1 --2- 172.21.15.156:0/974892694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f380cc260 0x7f3f38071410 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.806+0000 7f3f3d438700 1 -- 172.21.15.156:0/974892694 >> 172.21.15.156:0/974892694 conn(0x7f3f3806b380 msgr2=0x7f3f3806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:14.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.806+0000 7f3f3d438700 1 -- 172.21.15.156:0/974892694 shutdown_connections 2023-11-28T11:08:14.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.806+0000 7f3f3d438700 1 -- 172.21.15.156:0/974892694 wait complete. 2023-11-28T11:08:14.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.807+0000 7f3f3d438700 1 Processor -- start 2023-11-28T11:08:14.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.807+0000 7f3f3d438700 1 -- start start 2023-11-28T11:08:14.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f3d438700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f380cc260 0x7f3f381ee740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f3d438700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f380d0470 0x7f3f380b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f3d438700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f381596b0 0x7f3f380c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f3d438700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f380c7fa0 con 0x7f3f381596b0 2023-11-28T11:08:14.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f3d438700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f380c8ac0 con 0x7f3f380cc260 2023-11-28T11:08:14.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f3d438700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f380c81d0 con 0x7f3f380d0470 2023-11-28T11:08:14.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f377fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f381596b0 0x7f3f380c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f377fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f381596b0 0x7f3f380c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59966/0 (socket says 172.21.15.156:59966) 2023-11-28T11:08:14.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f377fe700 1 -- 172.21.15.156:0/3959520461 learned_addr learned my addr 172.21.15.156:0/3959520461 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:14.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f36ffd700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f380cc260 0x7f3f381ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f367fc700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f380d0470 0x7f3f380b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f377fe700 1 -- 172.21.15.156:0/3959520461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f380d0470 msgr2=0x7f3f380b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f377fe700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f380d0470 0x7f3f380b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f377fe700 1 -- 172.21.15.156:0/3959520461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f380cc260 msgr2=0x7f3f381ee740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:14.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f377fe700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f380cc260 0x7f3f381ee740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:14.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f377fe700 1 -- 172.21.15.156:0/3959520461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f30009a70 con 0x7f3f381596b0 2023-11-28T11:08:14.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.808+0000 7f3f367fc700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f380d0470 0x7f3f380b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:14.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.809+0000 7f3f377fe700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f381596b0 0x7f3f380c8f50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009d90 tx=0x7f3f30004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:14.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.809+0000 7f3f1f7fe700 1 -- 172.21.15.156:0/3959520461 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30004540 con 0x7f3f381596b0 2023-11-28T11:08:14.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.809+0000 7f3f1f7fe700 1 -- 172.21.15.156:0/3959520461 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f300046a0 con 0x7f3f381596b0 2023-11-28T11:08:14.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.809+0000 7f3f3d438700 1 -- 172.21.15.156:0/3959520461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f380ccfc0 con 0x7f3f381596b0 2023-11-28T11:08:14.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.809+0000 7f3f1f7fe700 1 -- 172.21.15.156:0/3959520461 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3001d470 con 0x7f3f381596b0 2023-11-28T11:08:14.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.809+0000 7f3f3d438700 1 -- 172.21.15.156:0/3959520461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f380ccce0 con 0x7f3f381596b0 2023-11-28T11:08:14.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.810+0000 7f3f3d438700 1 -- 172.21.15.156:0/3959520461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f380cd2a0 con 0x7f3f381596b0 2023-11-28T11:08:14.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.810+0000 7f3f1f7fe700 1 -- 172.21.15.156:0/3959520461 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f30023030 con 0x7f3f381596b0 2023-11-28T11:08:14.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.810+0000 7f3f1f7fe700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3f240682f0 0x7f3f2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:14.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.811+0000 7f3f1f7fe700 1 -- 172.21.15.156:0/3959520461 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3f30087570 con 0x7f3f381596b0 2023-11-28T11:08:14.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.811+0000 7f3f36ffd700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3f240682f0 0x7f3f2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:14.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.811+0000 7f3f36ffd700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3f240682f0 0x7f3f2406a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0011e0 tx=0x7f3f2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:14.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:14.815+0000 7f3f1f7fe700 1 -- 172.21.15.156:0/3959520461 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f30054040 con 0x7f3f381596b0 2023-11-28T11:08:15.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.002+0000 7f3f3d438700 1 -- 172.21.15.156:0/3959520461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3f38068690 con 0x7f3f381596b0 2023-11-28T11:08:15.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.957+0000 7f3f1f7fe700 1 -- 172.21.15.156:0/3959520461 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3f30057660 con 0x7f3f381596b0 2023-11-28T11:08:15.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.961+0000 7f3f3d438700 1 -- 172.21.15.156:0/3959520461 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3f240682f0 msgr2=0x7f3f2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:15.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.961+0000 7f3f3d438700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3f240682f0 0x7f3f2406a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0011e0 tx=0x7f3f2c009040 comp rx=0 tx=0).stop 2023-11-28T11:08:15.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.961+0000 7f3f3d438700 1 -- 172.21.15.156:0/3959520461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f381596b0 msgr2=0x7f3f380c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:15.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.961+0000 7f3f3d438700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f381596b0 0x7f3f380c8f50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009d90 tx=0x7f3f30004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:15.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.961+0000 7f3f3d438700 1 -- 172.21.15.156:0/3959520461 shutdown_connections 2023-11-28T11:08:15.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.961+0000 7f3f3d438700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f381596b0 0x7f3f380c8f50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:15.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.961+0000 7f3f3d438700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f380cc260 0x7f3f381ee740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:15.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.961+0000 7f3f3d438700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3f240682f0 0x7f3f2406a7a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:15.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.961+0000 7f3f3d438700 1 --2- 172.21.15.156:0/3959520461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f380d0470 0x7f3f380b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:15.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.961+0000 7f3f3d438700 1 -- 172.21.15.156:0/3959520461 >> 172.21.15.156:0/3959520461 conn(0x7f3f3806b380 msgr2=0x7f3f38159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:15.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.961+0000 7f3f3d438700 1 -- 172.21.15.156:0/3959520461 shutdown_connections 2023-11-28T11:08:15.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:15.962+0000 7f3f3d438700 1 -- 172.21.15.156:0/3959520461 wait complete. 2023-11-28T11:08:15.966 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' created 2023-11-28T11:08:15.980 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:08:16.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.113+0000 7fa0bb0a3700 1 Processor -- start 2023-11-28T11:08:16.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.113+0000 7fa0bb0a3700 1 -- start start 2023-11-28T11:08:16.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.114+0000 7fa0bb0a3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0b40cacb0 0x7fa0b40cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:16.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.114+0000 7fa0bb0a3700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b406c2f0 con 0x7fa0b40cde10 2023-11-28T11:08:16.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.114+0000 7fa0bb0a3700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b406b9c0 con 0x7fa0b40b0d80 2023-11-28T11:08:16.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.114+0000 7fa0bb0a3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b406bca0 con 0x7fa0b40cacb0 2023-11-28T11:08:16.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.114+0000 7fa0b3fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa0b40cde10 0x7fa0b40b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46618/0 (socket says 172.21.15.156:46618) 2023-11-28T11:08:16.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.114+0000 7fa0b3fff700 1 -- 172.21.15.156:0/3184843879 learned_addr learned my addr 172.21.15.156:0/3184843879 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:16.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.114+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145446997 0 0) 0x7fa0b406c2f0 con 0x7fa0b40cde10 2023-11-28T11:08:16.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.114+0000 7fa0b8e3f700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0b40cacb0 0x7fa0b40cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:16.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.119+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a4003a00 con 0x7fa0b40cde10 2023-11-28T11:08:16.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.119+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393248836 0 0) 0x7fa0b406b9c0 con 0x7fa0b40b0d80 2023-11-28T11:08:16.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.119+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b406c2f0 con 0x7fa0b40b0d80 2023-11-28T11:08:16.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.119+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3814837304 0 0) 0x7fa0a4003a00 con 0x7fa0b40cde10 2023-11-28T11:08:16.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0b40cacb0 msgr2=0x7fa0b40cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:16.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0b40cacb0 0x7fa0b40cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:16.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 >> v1:172.21.15.186:6789/0 conn(0x7fa0b40b0d80 legacy=0x7fa0b40cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:16.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b406bf80 con 0x7fa0b40cde10 2023-11-28T11:08:16.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fa0a0001670 con 0x7fa0b40cde10 2023-11-28T11:08:16.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 >> v1:172.21.15.156:6789/0 conn(0x7fa0b40cde10 legacy=0x7fa0b40b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:16.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4003e80 0x7fa0a4004270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:16.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a40047b0 0x7fa0a4006c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:16.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0b40cacb0 0x7fa0a4007330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:16.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b406bca0 con 0x7fa0a4003e80 2023-11-28T11:08:16.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4007a50 con 0x7fa0a40047b0 2023-11-28T11:08:16.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4007c10 con 0x7fa0b40cacb0 2023-11-28T11:08:16.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b8e3f700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4003e80 0x7fa0a4004270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:16.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b9640700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0b40cacb0 0x7fa0a4007330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b3fff700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a40047b0 0x7fa0a4006c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b8e3f700 1 -- 172.21.15.156:0/3184843879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0b40cacb0 msgr2=0x7fa0a4007330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b8e3f700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0b40cacb0 0x7fa0a4007330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b8e3f700 1 -- 172.21.15.156:0/3184843879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a40047b0 msgr2=0x7fa0a4006c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b8e3f700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a40047b0 0x7fa0a4006c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b8e3f700 1 -- 172.21.15.156:0/3184843879 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a4003a00 con 0x7fa0a4003e80 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b3fff700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a40047b0 0x7fa0a4006c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b8e3f700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4003e80 0x7fa0a4004270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800ca00 tx=0x7fa0a4003600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700e8dedd6d78bfc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8011040 con 0x7fa0a4003e80 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a800b7e0 con 0x7fa0a4003e80 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.120+0000 7fa0b37fe700 1 -- 172.21.15.156:0/3184843879 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a800bb50 con 0x7fa0a4003e80 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.121+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3184843879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4003e80 msgr2=0x7fa0a4004270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:16.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.121+0000 7fa0bb0a3700 1 --2- 172.21.15.156:0/3184843879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4003e80 0x7fa0a4004270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800ca00 tx=0x7fa0a4003600 comp rx=0 tx=0).stop 2023-11-28T11:08:16.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.121+0000 7fa0b9640700 1 -- 172.21.15.156:0/3184843879 reap_dead start 2023-11-28T11:08:16.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.121+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3184843879 shutdown_connections 2023-11-28T11:08:16.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.121+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3184843879 >> 172.21.15.156:0/3184843879 conn(0x7fa0b40cb670 msgr2=0x7fa0b40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:16.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.121+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3184843879 shutdown_connections 2023-11-28T11:08:16.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.121+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3184843879 wait complete. 2023-11-28T11:08:16.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.122+0000 7fa0bb0a3700 1 Processor -- start 2023-11-28T11:08:16.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.122+0000 7fa0bb0a3700 1 -- start start 2023-11-28T11:08:16.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.123+0000 7fa0bb0a3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4003e80 0x7fa0b40c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:16.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0bb0a3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0b4071390 0x7fa0b4167c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:16.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0bb0a3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b4071800 0x7fa0b4168140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:16.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0bb0a3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40c6c90 con 0x7fa0a4003e80 2023-11-28T11:08:16.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0bb0a3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40c76d0 con 0x7fa0b4071800 2023-11-28T11:08:16.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0bb0a3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40c79b0 con 0x7fa0b4071390 2023-11-28T11:08:16.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b8e3f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4003e80 0x7fa0b40c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:16.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b8e3f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4003e80 0x7fa0b40c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59982/0 (socket says 172.21.15.156:59982) 2023-11-28T11:08:16.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b8e3f700 1 -- 172.21.15.156:0/3767115618 learned_addr learned my addr 172.21.15.156:0/3767115618 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:16.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b8e3f700 1 -- 172.21.15.156:0/3767115618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0b4071390 msgr2=0x7fa0b4167c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:16.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b8e3f700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0b4071390 0x7fa0b4167c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:16.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b8e3f700 1 -- 172.21.15.156:0/3767115618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b4071800 msgr2=0x7fa0b4168140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:16.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b9640700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b4071800 0x7fa0b4168140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:16.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b8e3f700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b4071800 0x7fa0b4168140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:16.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b8e3f700 1 -- 172.21.15.156:0/3767115618 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b40cb080 con 0x7fa0a4003e80 2023-11-28T11:08:16.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b8e3f700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4003e80 0x7fa0b40c8490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800e010 tx=0x7fa0a8006070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:16.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b17fa700 1 -- 172.21.15.156:0/3767115618 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8011070 con 0x7fa0a4003e80 2023-11-28T11:08:16.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b17fa700 1 -- 172.21.15.156:0/3767115618 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a8005d50 con 0x7fa0a4003e80 2023-11-28T11:08:16.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0b17fa700 1 -- 172.21.15.156:0/3767115618 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8015a90 con 0x7fa0a4003e80 2023-11-28T11:08:16.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3767115618 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b406c780 con 0x7fa0a4003e80 2023-11-28T11:08:16.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.124+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3767115618 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b40cb210 con 0x7fa0a4003e80 2023-11-28T11:08:16.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.125+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3767115618 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b40c82e0 con 0x7fa0a4003e80 2023-11-28T11:08:16.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.126+0000 7fa0b17fa700 1 -- 172.21.15.156:0/3767115618 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa0a8015c10 con 0x7fa0a4003e80 2023-11-28T11:08:16.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.126+0000 7fa0b17fa700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa0980682f0 0x7fa09806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:16.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.126+0000 7fa0b17fa700 1 -- 172.21.15.156:0/3767115618 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fa0a8087250 con 0x7fa0a4003e80 2023-11-28T11:08:16.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.126+0000 7fa0b3fff700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa0980682f0 0x7fa09806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:16.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.127+0000 7fa0b3fff700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa0980682f0 0x7fa09806a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0001eb0 tx=0x7fa0a0001110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:16.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.131+0000 7fa0b17fa700 1 -- 172.21.15.156:0/3767115618 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa0a8053ba0 con 0x7fa0a4003e80 2023-11-28T11:08:16.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.314+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3767115618 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa0b40745c0 con 0x7fa0a4003e80 2023-11-28T11:08:16.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.959+0000 7fa0b17fa700 1 -- 172.21.15.156:0/3767115618 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa0a80571c0 con 0x7fa0a4003e80 2023-11-28T11:08:16.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.963+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3767115618 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa0980682f0 msgr2=0x7fa09806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:16.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.963+0000 7fa0bb0a3700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa0980682f0 0x7fa09806a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0001eb0 tx=0x7fa0a0001110 comp rx=0 tx=0).stop 2023-11-28T11:08:16.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.963+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3767115618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4003e80 msgr2=0x7fa0b40c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:16.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.963+0000 7fa0bb0a3700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4003e80 0x7fa0b40c8490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800e010 tx=0x7fa0a8006070 comp rx=0 tx=0).stop 2023-11-28T11:08:16.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.964+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3767115618 shutdown_connections 2023-11-28T11:08:16.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.964+0000 7fa0bb0a3700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b4071800 0x7fa0b4168140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:16.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.964+0000 7fa0bb0a3700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0a4003e80 0x7fa0b40c8490 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:16.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.964+0000 7fa0bb0a3700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa0980682f0 0x7fa09806a7a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:16.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.964+0000 7fa0bb0a3700 1 --2- 172.21.15.156:0/3767115618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0b4071390 0x7fa0b4167c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:16.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.964+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3767115618 >> 172.21.15.156:0/3767115618 conn(0x7fa0b40cb670 msgr2=0x7fa0b40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:16.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.965+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3767115618 shutdown_connections 2023-11-28T11:08:16.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:16.965+0000 7fa0bb0a3700 1 -- 172.21.15.156:0/3767115618 wait complete. 2023-11-28T11:08:16.972 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' created 2023-11-28T11:08:16.988 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:08:17.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.110+0000 7fb7a5d06700 1 Processor -- start 2023-11-28T11:08:17.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.110+0000 7fb7a5d06700 1 -- start start 2023-11-28T11:08:17.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb7a5d06700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7a0140950 0x7fb7a00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb7a5d06700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb7a00b2e70 0x7fb7a006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb7a5d06700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a013ff00 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb7a5d06700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a01404c0 con 0x7fb7a00b2e70 2023-11-28T11:08:17.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb7a5d06700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a0140dc0 con 0x7fb7a0140950 2023-11-28T11:08:17.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb79ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb7a00b2e70 0x7fb7a006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb79ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb7a00b2e70 0x7fb7a006f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59994/0 (socket says 172.21.15.156:59994) 2023-11-28T11:08:17.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb79ffff700 1 -- 172.21.15.156:0/3072712890 learned_addr learned my addr 172.21.15.156:0/3072712890 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:17.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb79effd700 1 --2- 172.21.15.156:0/3072712890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7a0140950 0x7fb7a00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb79ffff700 1 -- 172.21.15.156:0/3072712890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7a0140950 msgr2=0x7fb7a00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:17.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb79ffff700 1 --2- 172.21.15.156:0/3072712890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7a0140950 0x7fb7a00b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb79ffff700 1 -- 172.21.15.156:0/3072712890 >> v1:172.21.15.186:6789/0 conn(0x7fb7a00cd8c0 legacy=0x7fb7a00cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb79ffff700 1 -- 172.21.15.156:0/3072712890 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a015aa80 con 0x7fb7a00b2e70 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb79ffff700 1 --2- 172.21.15.156:0/3072712890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a00b2e70 0x7fb7a006f850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb798009e50 tx=0x7fb79800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ffdc5d95bad7e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb79e7fc700 1 -- 172.21.15.156:0/3072712890 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79800e070 con 0x7fb7a00b2e70 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb79e7fc700 1 -- 172.21.15.156:0/3072712890 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79800b910 con 0x7fb7a00b2e70 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.111+0000 7fb79e7fc700 1 -- 172.21.15.156:0/3072712890 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79800bcc0 con 0x7fb7a00b2e70 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.112+0000 7fb7a5d06700 1 -- 172.21.15.156:0/3072712890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a00b2e70 msgr2=0x7fb7a006f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.112+0000 7fb7a5d06700 1 --2- 172.21.15.156:0/3072712890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a00b2e70 0x7fb7a006f850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb798009e50 tx=0x7fb79800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.112+0000 7fb7a5d06700 1 -- 172.21.15.156:0/3072712890 shutdown_connections 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.112+0000 7fb7a5d06700 1 --2- 172.21.15.156:0/3072712890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a00b2e70 0x7fb7a006f850 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.112+0000 7fb7a5d06700 1 --2- 172.21.15.156:0/3072712890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7a0140950 0x7fb7a00b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.112+0000 7fb7a5d06700 1 -- 172.21.15.156:0/3072712890 >> 172.21.15.156:0/3072712890 conn(0x7fb7a006aed0 msgr2=0x7fb7a006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:17.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.112+0000 7fb7a5d06700 1 -- 172.21.15.156:0/3072712890 shutdown_connections 2023-11-28T11:08:17.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.112+0000 7fb7a5d06700 1 -- 172.21.15.156:0/3072712890 wait complete. 2023-11-28T11:08:17.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.114+0000 7fb7a5d06700 1 Processor -- start 2023-11-28T11:08:17.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.114+0000 7fb7a5d06700 1 -- start start 2023-11-28T11:08:17.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb7a5d06700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7a00b2e70 0x7fb7a01f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb7a5d06700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a00cd8c0 0x7fb7a00b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb7a5d06700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a0140950 0x7fb7a00b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb7a5d06700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a00b66c0 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb7a5d06700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a006ce20 con 0x7fb7a0140950 2023-11-28T11:08:17.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb7a5d06700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a01419b0 con 0x7fb7a00b2e70 2023-11-28T11:08:17.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a00cd8c0 0x7fb7a00b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a00cd8c0 0x7fb7a00b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59998/0 (socket says 172.21.15.156:59998) 2023-11-28T11:08:17.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79effd700 1 -- 172.21.15.156:0/1466317264 learned_addr learned my addr 172.21.15.156:0/1466317264 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:17.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79ffff700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a0140950 0x7fb7a00b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79f7fe700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7a00b2e70 0x7fb7a01f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79effd700 1 -- 172.21.15.156:0/1466317264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7a00b2e70 msgr2=0x7fb7a01f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79effd700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7a00b2e70 0x7fb7a01f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79effd700 1 -- 172.21.15.156:0/1466317264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a0140950 msgr2=0x7fb7a00b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79effd700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a0140950 0x7fb7a00b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79effd700 1 -- 172.21.15.156:0/1466317264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798009b30 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79f7fe700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7a00b2e70 0x7fb7a01f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:17.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79ffff700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a0140950 0x7fb7a00b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:17.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb79effd700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a00cd8c0 0x7fb7a00b4250 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb79000e840 tx=0x7fb79000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:17.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb787fff700 1 -- 172.21.15.156:0/1466317264 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790009590 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb787fff700 1 -- 172.21.15.156:0/1466317264 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb790013680 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb787fff700 1 -- 172.21.15.156:0/1466317264 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790013a10 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb7a5d06700 1 -- 172.21.15.156:0/1466317264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a00b35f0 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.115+0000 7fb7a5d06700 1 -- 172.21.15.156:0/1466317264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a00b3310 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.116+0000 7fb7a5d06700 1 -- 172.21.15.156:0/1466317264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a0140150 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.117+0000 7fb787fff700 1 -- 172.21.15.156:0/1466317264 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb790019030 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.117+0000 7fb787fff700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb78c0682f0 0x7fb78c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.117+0000 7fb787fff700 1 -- 172.21.15.156:0/1466317264 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fb790085ba0 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.117+0000 7fb79f7fe700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb78c0682f0 0x7fb78c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.118+0000 7fb79f7fe700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb78c0682f0 0x7fb78c06a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb794000fb0 tx=0x7fb794009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:17.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.122+0000 7fb787fff700 1 -- 172.21.15.156:0/1466317264 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb790052310 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.350+0000 7fb7a5d06700 1 -- 172.21.15.156:0/1466317264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb7a0142580 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.355+0000 7fb787fff700 1 -- 172.21.15.156:0/1466317264 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb790055930 con 0x7fb7a00cd8c0 2023-11-28T11:08:17.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.359+0000 7fb7a5d06700 1 -- 172.21.15.156:0/1466317264 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb78c0682f0 msgr2=0x7fb78c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.359+0000 7fb7a5d06700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb78c0682f0 0x7fb78c06a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb794000fb0 tx=0x7fb794009040 comp rx=0 tx=0).stop 2023-11-28T11:08:17.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.359+0000 7fb7a5d06700 1 -- 172.21.15.156:0/1466317264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a00cd8c0 msgr2=0x7fb7a00b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.359+0000 7fb7a5d06700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a00cd8c0 0x7fb7a00b4250 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb79000e840 tx=0x7fb79000ec00 comp rx=0 tx=0).stop 2023-11-28T11:08:17.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.360+0000 7fb7a5d06700 1 -- 172.21.15.156:0/1466317264 shutdown_connections 2023-11-28T11:08:17.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.360+0000 7fb7a5d06700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a0140950 0x7fb7a00b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.360+0000 7fb7a5d06700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb78c0682f0 0x7fb78c06a7a0 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb794000fb0 tx=0x7fb794009040 comp rx=0 tx=0).stop 2023-11-28T11:08:17.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.360+0000 7fb7a5d06700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7a00b2e70 0x7fb7a01f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.360+0000 7fb7a5d06700 1 --2- 172.21.15.156:0/1466317264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a00cd8c0 0x7fb7a00b4250 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.360+0000 7fb7a5d06700 1 -- 172.21.15.156:0/1466317264 >> 172.21.15.156:0/1466317264 conn(0x7fb7a006aed0 msgr2=0x7fb7a013e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:17.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.367+0000 7fb7a5d06700 1 -- 172.21.15.156:0/1466317264 shutdown_connections 2023-11-28T11:08:17.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.368+0000 7fb7a5d06700 1 -- 172.21.15.156:0/1466317264 wait complete. 2023-11-28T11:08:17.379 INFO:teuthology.orchestra.run.smithi156.stderr:new fs with metadata pool 3 and data pool 4 2023-11-28T11:08:17.388 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:17.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.506+0000 7f5c12098700 1 Processor -- start 2023-11-28T11:08:17.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.506+0000 7f5c12098700 1 -- start start 2023-11-28T11:08:17.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c12098700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5c0c06dee0 0x7f5c0c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c12098700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5c0c160540 0x7f5c0c160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c12098700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0c15d060 con 0x7f5c0c071040 2023-11-28T11:08:17.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c12098700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c15d340 con 0x7f5c0c160540 2023-11-28T11:08:17.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c12098700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c15d620 con 0x7f5c0c06dee0 2023-11-28T11:08:17.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c0b7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5c0c071040 0x7f5c0c071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46650/0 (socket says 172.21.15.156:46650) 2023-11-28T11:08:17.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c0b7fe700 1 -- 172.21.15.156:0/1561529744 learned_addr learned my addr 172.21.15.156:0/1561529744 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:17.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c0bfff700 1 --2- 172.21.15.156:0/1561529744 >> v2:172.21.15.156:3300/0 conn(0x7f5c0c160540 0x7f5c0c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c0affd700 1 --2- 172.21.15.156:0/1561529744 >> v2:172.21.15.186:3300/0 conn(0x7f5c0c06dee0 0x7f5c0c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c0a7fc700 1 -- 172.21.15.156:0/1561529744 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544065030 0 0) 0x7f5c0c15d060 con 0x7f5c0c071040 2023-11-28T11:08:17.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c0a7fc700 1 -- 172.21.15.156:0/1561529744 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bf40035b0 con 0x7f5c0c071040 2023-11-28T11:08:17.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c0bfff700 1 -- 172.21.15.156:0/1561529744 >> v2:172.21.15.186:3300/0 conn(0x7f5c0c06dee0 msgr2=0x7f5c0c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:17.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c0bfff700 1 --2- 172.21.15.156:0/1561529744 >> v2:172.21.15.186:3300/0 conn(0x7f5c0c06dee0 0x7f5c0c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c0bfff700 1 -- 172.21.15.156:0/1561529744 >> v1:172.21.15.156:6789/0 conn(0x7f5c0c071040 legacy=0x7f5c0c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c0bfff700 1 -- 172.21.15.156:0/1561529744 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c15d900 con 0x7f5c0c160540 2023-11-28T11:08:17.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c0bfff700 1 --2- 172.21.15.156:0/1561529744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c160540 0x7f5c0c160980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c06fb00 tx=0x7f5c0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e50b88071da127 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:17.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.507+0000 7f5c0a7fc700 1 -- 172.21.15.156:0/1561529744 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400e070 con 0x7f5c0c160540 2023-11-28T11:08:17.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.508+0000 7f5c0a7fc700 1 -- 172.21.15.156:0/1561529744 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0400bb20 con 0x7f5c0c160540 2023-11-28T11:08:17.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.508+0000 7f5c0a7fc700 1 -- 172.21.15.156:0/1561529744 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04012430 con 0x7f5c0c160540 2023-11-28T11:08:17.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.508+0000 7f5c12098700 1 -- 172.21.15.156:0/1561529744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c160540 msgr2=0x7f5c0c160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.508+0000 7f5c12098700 1 --2- 172.21.15.156:0/1561529744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c160540 0x7f5c0c160980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c06fb00 tx=0x7f5c0400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:17.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.508+0000 7f5c12098700 1 -- 172.21.15.156:0/1561529744 shutdown_connections 2023-11-28T11:08:17.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.508+0000 7f5c12098700 1 --2- 172.21.15.156:0/1561529744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c160540 0x7f5c0c160980 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.508+0000 7f5c12098700 1 --2- 172.21.15.156:0/1561529744 >> v2:172.21.15.186:3300/0 conn(0x7f5c0c06dee0 0x7f5c0c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.508+0000 7f5c12098700 1 -- 172.21.15.156:0/1561529744 >> 172.21.15.156:0/1561529744 conn(0x7f5c0c0d0000 msgr2=0x7f5c0c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:17.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.508+0000 7f5c12098700 1 -- 172.21.15.156:0/1561529744 shutdown_connections 2023-11-28T11:08:17.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.508+0000 7f5c12098700 1 -- 172.21.15.156:0/1561529744 wait complete. 2023-11-28T11:08:17.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.509+0000 7f5c12098700 1 Processor -- start 2023-11-28T11:08:17.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.509+0000 7f5c12098700 1 -- start start 2023-11-28T11:08:17.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.510+0000 7f5c12098700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c06dee0 0x7f5c0c153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.510+0000 7f5c12098700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c0c071040 0x7f5c0c0afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.510+0000 7f5c12098700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c160540 0x7f5c0c0aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.510+0000 7f5c12098700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c1539a0 con 0x7f5c0c160540 2023-11-28T11:08:17.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.510+0000 7f5c12098700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c1f34d0 con 0x7f5c0c071040 2023-11-28T11:08:17.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.510+0000 7f5c12098700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0c4d90 con 0x7f5c0c06dee0 2023-11-28T11:08:17.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.510+0000 7f5c0bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c160540 0x7f5c0c0aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.510+0000 7f5c0bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c160540 0x7f5c0c0aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60014/0 (socket says 172.21.15.156:60014) 2023-11-28T11:08:17.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.510+0000 7f5c0bfff700 1 -- 172.21.15.156:0/140513706 learned_addr learned my addr 172.21.15.156:0/140513706 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:17.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.510+0000 7f5c0affd700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c0c071040 0x7f5c0c0afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.510+0000 7f5c0b7fe700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c06dee0 0x7f5c0c153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.511+0000 7f5c0bfff700 1 -- 172.21.15.156:0/140513706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c06dee0 msgr2=0x7f5c0c153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.511+0000 7f5c0bfff700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c06dee0 0x7f5c0c153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.511+0000 7f5c0bfff700 1 -- 172.21.15.156:0/140513706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c0c071040 msgr2=0x7f5c0c0afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.511+0000 7f5c0bfff700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c0c071040 0x7f5c0c0afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.511+0000 7f5c0bfff700 1 -- 172.21.15.156:0/140513706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c04009a70 con 0x7f5c0c160540 2023-11-28T11:08:17.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.511+0000 7f5c0affd700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c0c071040 0x7f5c0c0afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:17.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.511+0000 7f5c0bfff700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c160540 0x7f5c0c0aff40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5c04014040 tx=0x7f5c04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:17.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.511+0000 7f5bf3fff700 1 -- 172.21.15.156:0/140513706 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04004960 con 0x7f5c0c160540 2023-11-28T11:08:17.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.511+0000 7f5bf3fff700 1 -- 172.21.15.156:0/140513706 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c04020070 con 0x7f5c0c160540 2023-11-28T11:08:17.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.511+0000 7f5bf3fff700 1 -- 172.21.15.156:0/140513706 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040276a0 con 0x7f5c0c160540 2023-11-28T11:08:17.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.511+0000 7f5c12098700 1 -- 172.21.15.156:0/140513706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0c15ccd0 con 0x7f5c0c160540 2023-11-28T11:08:17.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.511+0000 7f5c12098700 1 -- 172.21.15.156:0/140513706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c06e870 con 0x7f5c0c160540 2023-11-28T11:08:17.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.512+0000 7f5c12098700 1 -- 172.21.15.156:0/140513706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c06a270 con 0x7f5c0c160540 2023-11-28T11:08:17.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.512+0000 7f5bf3fff700 1 -- 172.21.15.156:0/140513706 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5c0402e070 con 0x7f5c0c160540 2023-11-28T11:08:17.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.513+0000 7f5bf3fff700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5bf80682f0 0x7f5bf806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.513+0000 7f5c0b7fe700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5bf80682f0 0x7f5bf806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.513+0000 7f5bf3fff700 1 -- 172.21.15.156:0/140513706 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c04022070 con 0x7f5c0c160540 2023-11-28T11:08:17.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.513+0000 7f5c0b7fe700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5bf80682f0 0x7f5bf806a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c000011e0 tx=0x7f5c00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:17.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.517+0000 7f5bf3fff700 1 -- 172.21.15.156:0/140513706 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5c04054180 con 0x7f5c0c160540 2023-11-28T11:08:17.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.699+0000 7f5c12098700 1 -- 172.21.15.156:0/140513706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c0c0c4bd0 con 0x7f5c0c160540 2023-11-28T11:08:17.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.700+0000 7f5bf3fff700 1 -- 172.21.15.156:0/140513706 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17968 (secure 0 0 0) 0x7f5c0402c090 con 0x7f5c0c160540 2023-11-28T11:08:17.702 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:17.702 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":22,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:17.352029+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:17.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.702+0000 7f5c12098700 1 -- 172.21.15.156:0/140513706 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5bf80682f0 msgr2=0x7f5bf806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.702+0000 7f5c12098700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5bf80682f0 0x7f5bf806a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c000011e0 tx=0x7f5c00009040 comp rx=0 tx=0).stop 2023-11-28T11:08:17.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.702+0000 7f5c12098700 1 -- 172.21.15.156:0/140513706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c160540 msgr2=0x7f5c0c0aff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.702+0000 7f5c12098700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c160540 0x7f5c0c0aff40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5c04014040 tx=0x7f5c04004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.702+0000 7f5c12098700 1 -- 172.21.15.156:0/140513706 shutdown_connections 2023-11-28T11:08:17.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.702+0000 7f5c12098700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c160540 0x7f5c0c0aff40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.702+0000 7f5c12098700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5bf80682f0 0x7f5bf806a7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.702+0000 7f5c12098700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c06dee0 0x7f5c0c153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.702+0000 7f5c12098700 1 --2- 172.21.15.156:0/140513706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c0c071040 0x7f5c0c0afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.702+0000 7f5c12098700 1 -- 172.21.15.156:0/140513706 >> 172.21.15.156:0/140513706 conn(0x7f5c0c0d0000 msgr2=0x7f5c0c15e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:17.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.702+0000 7f5c12098700 1 -- 172.21.15.156:0/140513706 shutdown_connections 2023-11-28T11:08:17.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.702+0000 7f5c12098700 1 -- 172.21.15.156:0/140513706 wait complete. 2023-11-28T11:08:17.716 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:17.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.834+0000 7f5e64175700 1 Processor -- start 2023-11-28T11:08:17.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.835+0000 7f5e64175700 1 -- start start 2023-11-28T11:08:17.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.835+0000 7f5e64175700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5c160740 0x7f5e5c0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.835+0000 7f5e64175700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5e5c06b060 0x7f5e5c06b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e64175700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5c0cc030 con 0x7f5e5c160b10 2023-11-28T11:08:17.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e64175700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c0ca720 con 0x7f5e5c06b060 2023-11-28T11:08:17.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e64175700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c0c9e10 con 0x7f5e5c160740 2023-11-28T11:08:17.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e61710700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5e5c160b10 0x7f5e5c0cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46664/0 (socket says 172.21.15.156:46664) 2023-11-28T11:08:17.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e61710700 1 -- 172.21.15.156:0/1808704537 learned_addr learned my addr 172.21.15.156:0/1808704537 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:17.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e62712700 1 --2- 172.21.15.156:0/1808704537 >> v2:172.21.15.186:3300/0 conn(0x7f5e5c06b060 0x7f5e5c06b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e61f11700 1 --2- 172.21.15.156:0/1808704537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5c160740 0x7f5e5c0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e60f0f700 1 -- 172.21.15.156:0/1808704537 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320522719 0 0) 0x7f5e5c0cc030 con 0x7f5e5c160b10 2023-11-28T11:08:17.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e60f0f700 1 -- 172.21.15.156:0/1808704537 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e480035b0 con 0x7f5e5c160b10 2023-11-28T11:08:17.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e62712700 1 -- 172.21.15.156:0/1808704537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5c160740 msgr2=0x7f5e5c0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:17.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e62712700 1 --2- 172.21.15.156:0/1808704537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5c160740 0x7f5e5c0cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e62712700 1 -- 172.21.15.156:0/1808704537 >> v1:172.21.15.156:6789/0 conn(0x7f5e5c160b10 legacy=0x7f5e5c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e60f0f700 1 -- 172.21.15.156:0/1808704537 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3761315852 0 0) 0x7f5e480035b0 con 0x7f5e5c160b10 2023-11-28T11:08:17.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e62712700 1 -- 172.21.15.156:0/1808704537 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c06efc0 con 0x7f5e5c06b060 2023-11-28T11:08:17.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e61f11700 1 --2- 172.21.15.156:0/1808704537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5c160740 0x7f5e5c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:17.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.836+0000 7f5e62712700 1 --2- 172.21.15.156:0/1808704537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e5c06b060 0x7f5e5c06b4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009d90 tx=0x7f5e5400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf09da8243acc25f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:17.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.837+0000 7f5e60f0f700 1 -- 172.21.15.156:0/1808704537 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5400e070 con 0x7f5e5c06b060 2023-11-28T11:08:17.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.837+0000 7f5e60f0f700 1 -- 172.21.15.156:0/1808704537 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e54003a20 con 0x7f5e5c06b060 2023-11-28T11:08:17.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.837+0000 7f5e60f0f700 1 -- 172.21.15.156:0/1808704537 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58001670 con 0x7f5e5c06b060 2023-11-28T11:08:17.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.837+0000 7f5e64175700 1 -- 172.21.15.156:0/1808704537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e5c06b060 msgr2=0x7f5e5c06b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.837+0000 7f5e64175700 1 --2- 172.21.15.156:0/1808704537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e5c06b060 0x7f5e5c06b4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009d90 tx=0x7f5e5400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:17.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.837+0000 7f5e64175700 1 -- 172.21.15.156:0/1808704537 shutdown_connections 2023-11-28T11:08:17.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.837+0000 7f5e64175700 1 --2- 172.21.15.156:0/1808704537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e5c06b060 0x7f5e5c06b4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.837+0000 7f5e64175700 1 --2- 172.21.15.156:0/1808704537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5c160740 0x7f5e5c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.837+0000 7f5e64175700 1 -- 172.21.15.156:0/1808704537 >> 172.21.15.156:0/1808704537 conn(0x7f5e5c0b12b0 msgr2=0x7f5e5c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:17.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.837+0000 7f5e64175700 1 -- 172.21.15.156:0/1808704537 shutdown_connections 2023-11-28T11:08:17.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.837+0000 7f5e64175700 1 -- 172.21.15.156:0/1808704537 wait complete. 2023-11-28T11:08:17.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.838+0000 7f5e64175700 1 Processor -- start 2023-11-28T11:08:17.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.838+0000 7f5e64175700 1 -- start start 2023-11-28T11:08:17.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e64175700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e5c06b060 0x7f5e5c146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e64175700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e5c160740 0x7f5e5c0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e64175700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5c160b10 0x7f5e5c1f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e64175700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c06e9f0 con 0x7f5e5c06b060 2023-11-28T11:08:17.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e64175700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c1473a0 con 0x7f5e5c160740 2023-11-28T11:08:17.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e64175700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c157d20 con 0x7f5e5c160b10 2023-11-28T11:08:17.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e61f11700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e5c06b060 0x7f5e5c146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e61f11700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e5c06b060 0x7f5e5c146e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60024/0 (socket says 172.21.15.156:60024) 2023-11-28T11:08:17.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e61f11700 1 -- 172.21.15.156:0/4036088531 learned_addr learned my addr 172.21.15.156:0/4036088531 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:17.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e61710700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e5c160740 0x7f5e5c0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e62712700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5c160b10 0x7f5e5c1f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e61f11700 1 -- 172.21.15.156:0/4036088531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5c160b10 msgr2=0x7f5e5c1f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e61f11700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5c160b10 0x7f5e5c1f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e61f11700 1 -- 172.21.15.156:0/4036088531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e5c160740 msgr2=0x7f5e5c0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:17.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e61f11700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e5c160740 0x7f5e5c0afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:17.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e61f11700 1 -- 172.21.15.156:0/4036088531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c0094a0 con 0x7f5e5c06b060 2023-11-28T11:08:17.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.839+0000 7f5e61710700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e5c160740 0x7f5e5c0afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:17.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.840+0000 7f5e61f11700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e5c06b060 0x7f5e5c146e60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00ebe0 tx=0x7f5e4c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:17.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.840+0000 7f5e527fc700 1 -- 172.21.15.156:0/4036088531 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00c910 con 0x7f5e5c06b060 2023-11-28T11:08:17.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.840+0000 7f5e527fc700 1 -- 172.21.15.156:0/4036088531 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c00ca70 con 0x7f5e5c06b060 2023-11-28T11:08:17.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.840+0000 7f5e64175700 1 -- 172.21.15.156:0/4036088531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e54009a70 con 0x7f5e5c06b060 2023-11-28T11:08:17.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.840+0000 7f5e527fc700 1 -- 172.21.15.156:0/4036088531 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00cde0 con 0x7f5e5c06b060 2023-11-28T11:08:17.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.840+0000 7f5e64175700 1 -- 172.21.15.156:0/4036088531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5c0ca3b0 con 0x7f5e5c06b060 2023-11-28T11:08:17.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.841+0000 7f5e64175700 1 -- 172.21.15.156:0/4036088531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5c0ca0d0 con 0x7f5e5c06b060 2023-11-28T11:08:17.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.841+0000 7f5e527fc700 1 -- 172.21.15.156:0/4036088531 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5e4c019030 con 0x7f5e5c06b060 2023-11-28T11:08:17.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.842+0000 7f5e527fc700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5e400682f0 0x7f5e4006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:17.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.842+0000 7f5e527fc700 1 -- 172.21.15.156:0/4036088531 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5e4c085d40 con 0x7f5e5c06b060 2023-11-28T11:08:17.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.842+0000 7f5e61710700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5e400682f0 0x7f5e4006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:17.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.842+0000 7f5e61710700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5e400682f0 0x7f5e4006a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c160450 tx=0x7f5e58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:17.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:17.846+0000 7f5e527fc700 1 -- 172.21.15.156:0/4036088531 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5e4c0525a0 con 0x7f5e5c06b060 2023-11-28T11:08:18.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.034+0000 7f5e64175700 1 -- 172.21.15.156:0/4036088531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e5c06e840 con 0x7f5e5c06b060 2023-11-28T11:08:18.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.035+0000 7f5e527fc700 1 -- 172.21.15.156:0/4036088531 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17968 (secure 0 0 0) 0x7f5e4c017090 con 0x7f5e5c06b060 2023-11-28T11:08:18.036 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:18.037 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":22,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:17.352029+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:18.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.037+0000 7f5e64175700 1 -- 172.21.15.156:0/4036088531 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5e400682f0 msgr2=0x7f5e4006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.037+0000 7f5e64175700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5e400682f0 0x7f5e4006a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c160450 tx=0x7f5e58009040 comp rx=0 tx=0).stop 2023-11-28T11:08:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.037+0000 7f5e64175700 1 -- 172.21.15.156:0/4036088531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e5c06b060 msgr2=0x7f5e5c146e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.037+0000 7f5e64175700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e5c06b060 0x7f5e5c146e60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00ebe0 tx=0x7f5e4c009ec0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.037+0000 7f5e64175700 1 -- 172.21.15.156:0/4036088531 shutdown_connections 2023-11-28T11:08:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.037+0000 7f5e64175700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5e400682f0 0x7f5e4006a7a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.037+0000 7f5e64175700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5c160b10 0x7f5e5c1f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.037+0000 7f5e64175700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e5c06b060 0x7f5e5c146e60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.037+0000 7f5e64175700 1 --2- 172.21.15.156:0/4036088531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e5c160740 0x7f5e5c0afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.037+0000 7f5e64175700 1 -- 172.21.15.156:0/4036088531 >> 172.21.15.156:0/4036088531 conn(0x7f5e5c0b12b0 msgr2=0x7f5e5c0ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.037+0000 7f5e64175700 1 -- 172.21.15.156:0/4036088531 shutdown_connections 2023-11-28T11:08:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.037+0000 7f5e64175700 1 -- 172.21.15.156:0/4036088531 wait complete. 2023-11-28T11:08:18.051 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:08:18.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.173+0000 7fea6f4ef700 1 Processor -- start 2023-11-28T11:08:18.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.173+0000 7fea6f4ef700 1 -- start start 2023-11-28T11:08:18.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.174+0000 7fea6f4ef700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fea6815cef0 0x7fea680af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.174+0000 7fea6f4ef700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea680cc5f0 con 0x7fea680cbdf0 2023-11-28T11:08:18.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.174+0000 7fea6f4ef700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6815d470 con 0x7fea6806ba90 2023-11-28T11:08:18.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.174+0000 7fea6f4ef700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6815d750 con 0x7fea6815cef0 2023-11-28T11:08:18.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.174+0000 7fea6d28b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fea6815cef0 0x7fea680af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.174+0000 7fea6ca8a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fea680cbdf0 0x7fea6806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46678/0 (socket says 172.21.15.156:46678) 2023-11-28T11:08:18.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.174+0000 7fea6ca8a700 1 -- 172.21.15.156:0/1467632014 learned_addr learned my addr 172.21.15.156:0/1467632014 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:18.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.175+0000 7fea6d28b700 1 -- 172.21.15.156:0/1467632014 >> v1:172.21.15.186:6789/0 conn(0x7fea6806ba90 legacy=0x7fea680ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:18.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.175+0000 7fea6d28b700 1 -- 172.21.15.156:0/1467632014 >> v1:172.21.15.156:6789/0 conn(0x7fea680cbdf0 legacy=0x7fea6806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.175+0000 7fea6d28b700 1 -- 172.21.15.156:0/1467632014 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6815da30 con 0x7fea6815cef0 2023-11-28T11:08:18.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.175+0000 7fea6d28b700 1 --2- 172.21.15.156:0/1467632014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea6815cef0 0x7fea680af400 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fea58009870 tx=0x7fea58009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc2be88b5cd4bf6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:18.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.175+0000 7fea5ffff700 1 -- 172.21.15.156:0/1467632014 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58004230 con 0x7fea6815cef0 2023-11-28T11:08:18.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.175+0000 7fea5ffff700 1 -- 172.21.15.156:0/1467632014 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea58004390 con 0x7fea6815cef0 2023-11-28T11:08:18.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.175+0000 7fea5ffff700 1 -- 172.21.15.156:0/1467632014 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea580036a0 con 0x7fea6815cef0 2023-11-28T11:08:18.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.175+0000 7fea6f4ef700 1 -- 172.21.15.156:0/1467632014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea6815cef0 msgr2=0x7fea680af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.175+0000 7fea6f4ef700 1 --2- 172.21.15.156:0/1467632014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea6815cef0 0x7fea680af400 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fea58009870 tx=0x7fea58009b80 comp rx=0 tx=0).stop 2023-11-28T11:08:18.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.175+0000 7fea6f4ef700 1 -- 172.21.15.156:0/1467632014 shutdown_connections 2023-11-28T11:08:18.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.175+0000 7fea6f4ef700 1 --2- 172.21.15.156:0/1467632014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea6815cef0 0x7fea680af400 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.175+0000 7fea6f4ef700 1 -- 172.21.15.156:0/1467632014 >> 172.21.15.156:0/1467632014 conn(0x7fea680d0000 msgr2=0x7fea680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:18.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.176+0000 7fea6f4ef700 1 -- 172.21.15.156:0/1467632014 shutdown_connections 2023-11-28T11:08:18.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.176+0000 7fea6f4ef700 1 -- 172.21.15.156:0/1467632014 wait complete. 2023-11-28T11:08:18.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.177+0000 7fea6f4ef700 1 Processor -- start 2023-11-28T11:08:18.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.177+0000 7fea6f4ef700 1 -- start start 2023-11-28T11:08:18.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.177+0000 7fea6f4ef700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea6806ba90 0x7fea68158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.177+0000 7fea6f4ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea680cbdf0 0x7fea680b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.177+0000 7fea6f4ef700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea6815cef0 0x7fea6815a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.177+0000 7fea6f4ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea680c7470 con 0x7fea680cbdf0 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.177+0000 7fea6f4ef700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea680ca090 con 0x7fea6806ba90 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.177+0000 7fea6f4ef700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea680b5c50 con 0x7fea6815cef0 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea6ca8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea680cbdf0 0x7fea680b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea6ca8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea680cbdf0 0x7fea680b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60040/0 (socket says 172.21.15.156:60040) 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea6ca8a700 1 -- 172.21.15.156:0/3297245430 learned_addr learned my addr 172.21.15.156:0/3297245430 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea6d28b700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea6806ba90 0x7fea68158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea6ca8a700 1 -- 172.21.15.156:0/3297245430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea6815cef0 msgr2=0x7fea6815a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea6ca8a700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea6815cef0 0x7fea6815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea6ca8a700 1 -- 172.21.15.156:0/3297245430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea6806ba90 msgr2=0x7fea68158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea6ca8a700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea6806ba90 0x7fea68158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea6ca8a700 1 -- 172.21.15.156:0/3297245430 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea580094a0 con 0x7fea680cbdf0 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea6ca8a700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea680cbdf0 0x7fea680b5710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea6400c520 tx=0x7fea6400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:18.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea5dffb700 1 -- 172.21.15.156:0/3297245430 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64005d70 con 0x7fea680cbdf0 2023-11-28T11:08:18.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea5dffb700 1 -- 172.21.15.156:0/3297245430 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea640070d0 con 0x7fea680cbdf0 2023-11-28T11:08:18.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea6f4ef700 1 -- 172.21.15.156:0/3297245430 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6815dff0 con 0x7fea680cbdf0 2023-11-28T11:08:18.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea5dffb700 1 -- 172.21.15.156:0/3297245430 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64007440 con 0x7fea680cbdf0 2023-11-28T11:08:18.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.178+0000 7fea6f4ef700 1 -- 172.21.15.156:0/3297245430 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6815dd10 con 0x7fea680cbdf0 2023-11-28T11:08:18.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.179+0000 7fea6f4ef700 1 -- 172.21.15.156:0/3297245430 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea680cad60 con 0x7fea680cbdf0 2023-11-28T11:08:18.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.179+0000 7fea5dffb700 1 -- 172.21.15.156:0/3297245430 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea64019030 con 0x7fea680cbdf0 2023-11-28T11:08:18.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.180+0000 7fea5dffb700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea4c0682f0 0x7fea4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:18.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.180+0000 7fea5dffb700 1 -- 172.21.15.156:0/3297245430 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea64084f30 con 0x7fea680cbdf0 2023-11-28T11:08:18.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.180+0000 7fea6d28b700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea4c0682f0 0x7fea4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:18.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.180+0000 7fea6d28b700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea4c0682f0 0x7fea4c06a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fea680b0f40 tx=0x7fea58013040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:18.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.184+0000 7fea5dffb700 1 -- 172.21.15.156:0/3297245430 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fea6404b5e0 con 0x7fea680cbdf0 2023-11-28T11:08:18.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:18.413+0000 7fea6f4ef700 1 -- 172.21.15.156:0/3297245430 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fea680d0e60 con 0x7fea680cbdf0 2023-11-28T11:08:19.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.371+0000 7fea5dffb700 1 -- 172.21.15.156:0/3297245430 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fea640170b0 con 0x7fea680cbdf0 2023-11-28T11:08:19.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.374+0000 7fea6f4ef700 1 -- 172.21.15.156:0/3297245430 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea4c0682f0 msgr2=0x7fea4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.374+0000 7fea6f4ef700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea4c0682f0 0x7fea4c06a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fea680b0f40 tx=0x7fea58013040 comp rx=0 tx=0).stop 2023-11-28T11:08:19.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.374+0000 7fea6f4ef700 1 -- 172.21.15.156:0/3297245430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea680cbdf0 msgr2=0x7fea680b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.374+0000 7fea6f4ef700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea680cbdf0 0x7fea680b5710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea6400c520 tx=0x7fea6400c8e0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.374+0000 7fea6f4ef700 1 -- 172.21.15.156:0/3297245430 shutdown_connections 2023-11-28T11:08:19.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.374+0000 7fea6f4ef700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea4c0682f0 0x7fea4c06a7a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.374+0000 7fea6f4ef700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea6815cef0 0x7fea6815a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.374+0000 7fea6f4ef700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea6806ba90 0x7fea68158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.374+0000 7fea6f4ef700 1 --2- 172.21.15.156:0/3297245430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea680cbdf0 0x7fea680b5710 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.374+0000 7fea6f4ef700 1 -- 172.21.15.156:0/3297245430 >> 172.21.15.156:0/3297245430 conn(0x7fea680d0000 msgr2=0x7fea680b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.374+0000 7fea6f4ef700 1 -- 172.21.15.156:0/3297245430 shutdown_connections 2023-11-28T11:08:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.374+0000 7fea6f4ef700 1 -- 172.21.15.156:0/3297245430 wait complete. 2023-11-28T11:08:19.390 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:08:19.390 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:08:19.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.520+0000 7ff0e8900700 1 Processor -- start 2023-11-28T11:08:19.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.520+0000 7ff0e8900700 1 -- start start 2023-11-28T11:08:19.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e8900700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0e40ce040 0x7ff0e415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e8900700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0e40b2930 0x7ff0e40b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e8900700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e406b420 con 0x7ff0e40d0a00 2023-11-28T11:08:19.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e8900700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e40b32f0 con 0x7ff0e40b2930 2023-11-28T11:08:19.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e8900700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e406ff60 con 0x7ff0e40ce040 2023-11-28T11:08:19.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e2d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0e40b2930 0x7ff0e40b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e2d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0e40b2930 0x7ff0e40b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60050/0 (socket says 172.21.15.156:60050) 2023-11-28T11:08:19.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e2d9d700 1 -- 172.21.15.156:0/1682858240 learned_addr learned my addr 172.21.15.156:0/1682858240 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:19.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e259c700 1 --2- 172.21.15.156:0/1682858240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0e40ce040 0x7ff0e415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e2d9d700 1 -- 172.21.15.156:0/1682858240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0e40ce040 msgr2=0x7ff0e415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:19.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e2d9d700 1 --2- 172.21.15.156:0/1682858240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0e40ce040 0x7ff0e415df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e2d9d700 1 -- 172.21.15.156:0/1682858240 >> v1:172.21.15.186:6789/0 conn(0x7ff0e40d0a00 legacy=0x7ff0e406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:19.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e2d9d700 1 -- 172.21.15.156:0/1682858240 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e406b700 con 0x7ff0e40b2930 2023-11-28T11:08:19.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e2d9d700 1 --2- 172.21.15.156:0/1682858240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e40b2930 0x7ff0e40b2db0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009e50 tx=0x7ff0dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e764b570ebf90527 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:19.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e159a700 1 -- 172.21.15.156:0/1682858240 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00e070 con 0x7ff0e40b2930 2023-11-28T11:08:19.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e159a700 1 -- 172.21.15.156:0/1682858240 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc003a20 con 0x7ff0e40b2930 2023-11-28T11:08:19.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.521+0000 7ff0e159a700 1 -- 172.21.15.156:0/1682858240 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc003dd0 con 0x7ff0e40b2930 2023-11-28T11:08:19.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.522+0000 7ff0e8900700 1 -- 172.21.15.156:0/1682858240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e40b2930 msgr2=0x7ff0e40b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.522+0000 7ff0e8900700 1 --2- 172.21.15.156:0/1682858240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e40b2930 0x7ff0e40b2db0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009e50 tx=0x7ff0dc00b330 comp rx=0 tx=0).stop 2023-11-28T11:08:19.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.522+0000 7ff0e8900700 1 -- 172.21.15.156:0/1682858240 shutdown_connections 2023-11-28T11:08:19.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.522+0000 7ff0e8900700 1 --2- 172.21.15.156:0/1682858240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e40b2930 0x7ff0e40b2db0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.522+0000 7ff0e8900700 1 --2- 172.21.15.156:0/1682858240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0e40ce040 0x7ff0e415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.522+0000 7ff0e8900700 1 -- 172.21.15.156:0/1682858240 >> 172.21.15.156:0/1682858240 conn(0x7ff0e406aed0 msgr2=0x7ff0e406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:19.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.522+0000 7ff0e8900700 1 -- 172.21.15.156:0/1682858240 shutdown_connections 2023-11-28T11:08:19.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.522+0000 7ff0e8900700 1 -- 172.21.15.156:0/1682858240 wait complete. 2023-11-28T11:08:19.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.523+0000 7ff0e8900700 1 Processor -- start 2023-11-28T11:08:19.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.523+0000 7ff0e8900700 1 -- start start 2023-11-28T11:08:19.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e8900700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0e40b2930 0x7ff0e4151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e8900700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e40ce040 0x7ff0e41543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e8900700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e40d0a00 0x7ff0e41548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e8900700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e413ed90 con 0x7ff0e40d0a00 2023-11-28T11:08:19.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e8900700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4155c20 con 0x7ff0e40ce040 2023-11-28T11:08:19.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e8900700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4155660 con 0x7ff0e40b2930 2023-11-28T11:08:19.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e40d0a00 0x7ff0e41548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e40d0a00 0x7ff0e41548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60056/0 (socket says 172.21.15.156:60056) 2023-11-28T11:08:19.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e2d9d700 1 -- 172.21.15.156:0/236512309 learned_addr learned my addr 172.21.15.156:0/236512309 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:19.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e1d9b700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e40ce040 0x7ff0e41543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e259c700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0e40b2930 0x7ff0e4151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e2d9d700 1 -- 172.21.15.156:0/236512309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0e40b2930 msgr2=0x7ff0e4151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e2d9d700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0e40b2930 0x7ff0e4151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e2d9d700 1 -- 172.21.15.156:0/236512309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e40ce040 msgr2=0x7ff0e41543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.524+0000 7ff0e2d9d700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e40ce040 0x7ff0e41543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.525+0000 7ff0e2d9d700 1 -- 172.21.15.156:0/236512309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc009b30 con 0x7ff0e40d0a00 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.525+0000 7ff0e2d9d700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e40d0a00 0x7ff0e41548f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40b3aa0 tx=0x7ff0dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.525+0000 7ff0d2ffd700 1 -- 172.21.15.156:0/236512309 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc004440 con 0x7ff0e40d0a00 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.525+0000 7ff0d2ffd700 1 -- 172.21.15.156:0/236512309 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc0045a0 con 0x7ff0e40d0a00 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.525+0000 7ff0e8900700 1 -- 172.21.15.156:0/236512309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e406bcc0 con 0x7ff0e40d0a00 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.525+0000 7ff0d2ffd700 1 -- 172.21.15.156:0/236512309 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc004950 con 0x7ff0e40d0a00 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.525+0000 7ff0e8900700 1 -- 172.21.15.156:0/236512309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e406b9e0 con 0x7ff0e40d0a00 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.526+0000 7ff0e8900700 1 -- 172.21.15.156:0/236512309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e406bfa0 con 0x7ff0e40d0a00 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.526+0000 7ff0d2ffd700 1 -- 172.21.15.156:0/236512309 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff0dc022030 con 0x7ff0e40d0a00 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.527+0000 7ff0d2ffd700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0c40682f0 0x7ff0c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:19.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.527+0000 7ff0e259c700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0c40682f0 0x7ff0c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:19.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.527+0000 7ff0d2ffd700 1 -- 172.21.15.156:0/236512309 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff0dc087120 con 0x7ff0e40d0a00 2023-11-28T11:08:19.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.527+0000 7ff0e259c700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0c40682f0 0x7ff0c406a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40cd600 tx=0x7ff0d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:19.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.531+0000 7ff0d2ffd700 1 -- 172.21.15.156:0/236512309 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff0dc053900 con 0x7ff0e40d0a00 2023-11-28T11:08:19.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:19.755+0000 7ff0e8900700 1 -- 172.21.15.156:0/236512309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff0e4068690 con 0x7ff0e40d0a00 2023-11-28T11:08:20.437 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:08:20.444 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:08:20.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.755+0000 7ff0d2ffd700 1 -- 172.21.15.156:0/236512309 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff0dc02c300 con 0x7ff0e40d0a00 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.757+0000 7ff0e8900700 1 -- 172.21.15.156:0/236512309 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0c40682f0 msgr2=0x7ff0c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.757+0000 7ff0e8900700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0c40682f0 0x7ff0c406a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40cd600 tx=0x7ff0d4008040 comp rx=0 tx=0).stop 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.757+0000 7ff0e8900700 1 -- 172.21.15.156:0/236512309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e40d0a00 msgr2=0x7ff0e41548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.757+0000 7ff0e8900700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e40d0a00 0x7ff0e41548f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40b3aa0 tx=0x7ff0dc004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.757+0000 7ff0e8900700 1 -- 172.21.15.156:0/236512309 shutdown_connections 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.757+0000 7ff0e8900700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e40d0a00 0x7ff0e41548f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.757+0000 7ff0e8900700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0c40682f0 0x7ff0c406a7a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.757+0000 7ff0e8900700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0e40b2930 0x7ff0e4151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.757+0000 7ff0e8900700 1 --2- 172.21.15.156:0/236512309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e40ce040 0x7ff0e41543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.757+0000 7ff0e8900700 1 -- 172.21.15.156:0/236512309 >> 172.21.15.156:0/236512309 conn(0x7ff0e406aed0 msgr2=0x7ff0e40d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:20.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.757+0000 7ff0e8900700 1 -- 172.21.15.156:0/236512309 shutdown_connections 2023-11-28T11:08:20.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.757+0000 7ff0e8900700 1 -- 172.21.15.156:0/236512309 wait complete. 2023-11-28T11:08:20.773 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:08:20.773 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:08:20.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.900+0000 7f659ad8f700 1 Processor -- start 2023-11-28T11:08:20.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.900+0000 7f659ad8f700 1 -- start start 2023-11-28T11:08:20.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.900+0000 7f659ad8f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6594071040 0x7f6594071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.900+0000 7f659ad8f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6594160540 0x7f65941609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.900+0000 7f659ad8f700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6594071950 con 0x7f659406dee0 2023-11-28T11:08:20.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.900+0000 7f659ad8f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659415d060 con 0x7f6594071040 2023-11-28T11:08:20.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.900+0000 7f659ad8f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659415d340 con 0x7f6594160540 2023-11-28T11:08:20.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f6598b2b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6594071040 0x7f6594071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f6593fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f659406dee0 0x7f659406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46710/0 (socket says 172.21.15.156:46710) 2023-11-28T11:08:20.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f6593fff700 1 -- 172.21.15.156:0/1517453772 learned_addr learned my addr 172.21.15.156:0/1517453772 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:20.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f659932c700 1 --2- 172.21.15.156:0/1517453772 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6594160540 0x7f65941609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f6598b2b700 1 -- 172.21.15.156:0/1517453772 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6594160540 msgr2=0x7f65941609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:20.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f6598b2b700 1 --2- 172.21.15.156:0/1517453772 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6594160540 0x7f65941609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f65937fe700 1 -- 172.21.15.156:0/1517453772 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2489878024 0 0) 0x7f6594071950 con 0x7f659406dee0 2023-11-28T11:08:20.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f6598b2b700 1 -- 172.21.15.156:0/1517453772 >> v1:172.21.15.156:6789/0 conn(0x7f659406dee0 legacy=0x7f659406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f6598b2b700 1 -- 172.21.15.156:0/1517453772 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659415d620 con 0x7f6594071040 2023-11-28T11:08:20.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f659932c700 1 --2- 172.21.15.156:0/1517453772 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6594160540 0x7f65941609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:20.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f6598b2b700 1 --2- 172.21.15.156:0/1517453772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6594071040 0x7f6594071410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6588009840 tx=0x7f6588009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cc92ca2a4177ed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:20.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f65937fe700 1 -- 172.21.15.156:0/1517453772 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588011830 con 0x7f6594071040 2023-11-28T11:08:20.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f65937fe700 1 -- 172.21.15.156:0/1517453772 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6588011990 con 0x7f6594071040 2023-11-28T11:08:20.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.901+0000 7f65937fe700 1 -- 172.21.15.156:0/1517453772 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588011d20 con 0x7f6594071040 2023-11-28T11:08:20.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.902+0000 7f659ad8f700 1 -- 172.21.15.156:0/1517453772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6594071040 msgr2=0x7f6594071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.902+0000 7f659ad8f700 1 --2- 172.21.15.156:0/1517453772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6594071040 0x7f6594071410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6588009840 tx=0x7f6588009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:20.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.902+0000 7f659ad8f700 1 -- 172.21.15.156:0/1517453772 shutdown_connections 2023-11-28T11:08:20.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.902+0000 7f659ad8f700 1 --2- 172.21.15.156:0/1517453772 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6594160540 0x7f65941609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.902+0000 7f659ad8f700 1 --2- 172.21.15.156:0/1517453772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6594071040 0x7f6594071410 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.902+0000 7f659ad8f700 1 -- 172.21.15.156:0/1517453772 >> 172.21.15.156:0/1517453772 conn(0x7f65940d0000 msgr2=0x7f65940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:20.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.902+0000 7f659ad8f700 1 -- 172.21.15.156:0/1517453772 shutdown_connections 2023-11-28T11:08:20.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.902+0000 7f659ad8f700 1 -- 172.21.15.156:0/1517453772 wait complete. 2023-11-28T11:08:20.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.903+0000 7f659ad8f700 1 Processor -- start 2023-11-28T11:08:20.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.903+0000 7f659ad8f700 1 -- start start 2023-11-28T11:08:20.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f659ad8f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f659406dee0 0x7f6594153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f659ad8f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6594071040 0x7f65940c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f659ad8f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6594160540 0x7f6594152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f659ad8f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65940c6960 con 0x7f659406dee0 2023-11-28T11:08:20.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f659ad8f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65940b3eb0 con 0x7f6594071040 2023-11-28T11:08:20.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f659ad8f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65940b4190 con 0x7f6594160540 2023-11-28T11:08:20.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f6598b2b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f659406dee0 0x7f6594153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f6598b2b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f659406dee0 0x7f6594153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60066/0 (socket says 172.21.15.156:60066) 2023-11-28T11:08:20.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f6598b2b700 1 -- 172.21.15.156:0/3610872059 learned_addr learned my addr 172.21.15.156:0/3610872059 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:20.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f6598b2b700 1 -- 172.21.15.156:0/3610872059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6594160540 msgr2=0x7f6594152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f659932c700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6594160540 0x7f6594152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f6593fff700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6594071040 0x7f65940c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f6598b2b700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6594160540 0x7f6594152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f6598b2b700 1 -- 172.21.15.156:0/3610872059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6594071040 msgr2=0x7f65940c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:20.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f6598b2b700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6594071040 0x7f65940c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:20.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f6598b2b700 1 -- 172.21.15.156:0/3610872059 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65880094a0 con 0x7f659406dee0 2023-11-28T11:08:20.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.904+0000 7f6598b2b700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f659406dee0 0x7f6594153c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6588014040 tx=0x7f6588004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:20.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.905+0000 7f65917fa700 1 -- 172.21.15.156:0/3610872059 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65880036a0 con 0x7f659406dee0 2023-11-28T11:08:20.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.905+0000 7f65917fa700 1 -- 172.21.15.156:0/3610872059 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658801e070 con 0x7f659406dee0 2023-11-28T11:08:20.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.905+0000 7f65917fa700 1 -- 172.21.15.156:0/3610872059 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588003a20 con 0x7f659406dee0 2023-11-28T11:08:20.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.905+0000 7f659ad8f700 1 -- 172.21.15.156:0/3610872059 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659415ccd0 con 0x7f659406dee0 2023-11-28T11:08:20.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.905+0000 7f659ad8f700 1 -- 172.21.15.156:0/3610872059 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659415d900 con 0x7f659406dee0 2023-11-28T11:08:20.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.906+0000 7f659ad8f700 1 -- 172.21.15.156:0/3610872059 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659406a270 con 0x7f659406dee0 2023-11-28T11:08:20.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.906+0000 7f65917fa700 1 -- 172.21.15.156:0/3610872059 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f658800f050 con 0x7f659406dee0 2023-11-28T11:08:20.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.906+0000 7f65917fa700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65780682f0 0x7f657806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:20.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.906+0000 7f65917fa700 1 -- 172.21.15.156:0/3610872059 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6588086c90 con 0x7f659406dee0 2023-11-28T11:08:20.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.906+0000 7f6593fff700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65780682f0 0x7f657806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:20.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.907+0000 7f6593fff700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65780682f0 0x7f657806a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f65940b1ae0 tx=0x7f6580009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:20.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:20.911+0000 7f65917fa700 1 -- 172.21.15.156:0/3610872059 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f65880534f0 con 0x7f659406dee0 2023-11-28T11:08:21.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.136+0000 7f659ad8f700 1 -- 172.21.15.156:0/3610872059 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65940720a0 con 0x7f659406dee0 2023-11-28T11:08:21.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.137+0000 7f65917fa700 1 -- 172.21.15.156:0/3610872059 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f6588015020 con 0x7f659406dee0 2023-11-28T11:08:21.139 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:21.139 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1221397150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1221397150},{"type":"v1","addr":"172.21.15.186:6834","nonce":1221397150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/22494485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":22494485},{"type":"v1","addr":"172.21.15.156:6835","nonce":22494485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3382922708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3382922708},{"type":"v1","addr":"172.21.15.156:6837","nonce":3382922708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:17.352335+0000","modified":"2023-11-28T11:08:20.752191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/4134790300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":4134790300},{"type":"v1","addr":"172.21.15.186:6835","nonce":4134790300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:08:21.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.139+0000 7f659ad8f700 1 -- 172.21.15.156:0/3610872059 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65780682f0 msgr2=0x7f657806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.139+0000 7f659ad8f700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65780682f0 0x7f657806a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f65940b1ae0 tx=0x7f6580009040 comp rx=0 tx=0).stop 2023-11-28T11:08:21.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.139+0000 7f659ad8f700 1 -- 172.21.15.156:0/3610872059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f659406dee0 msgr2=0x7f6594153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.139+0000 7f659ad8f700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f659406dee0 0x7f6594153c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6588014040 tx=0x7f6588004750 comp rx=0 tx=0).stop 2023-11-28T11:08:21.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.139+0000 7f659ad8f700 1 -- 172.21.15.156:0/3610872059 shutdown_connections 2023-11-28T11:08:21.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.139+0000 7f659ad8f700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65780682f0 0x7f657806a7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.139+0000 7f659ad8f700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6594160540 0x7f6594152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.139+0000 7f659ad8f700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f659406dee0 0x7f6594153c80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.139+0000 7f659ad8f700 1 --2- 172.21.15.156:0/3610872059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6594071040 0x7f65940c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.139+0000 7f659ad8f700 1 -- 172.21.15.156:0/3610872059 >> 172.21.15.156:0/3610872059 conn(0x7f65940d0000 msgr2=0x7f659415e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:21.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.139+0000 7f659ad8f700 1 -- 172.21.15.156:0/3610872059 shutdown_connections 2023-11-28T11:08:21.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.139+0000 7f659ad8f700 1 -- 172.21.15.156:0/3610872059 wait complete. 2023-11-28T11:08:21.142 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 8 2023-11-28T11:08:21.152 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:21.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.270+0000 7f122cb8a700 1 Processor -- start 2023-11-28T11:08:21.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.270+0000 7f122cb8a700 1 -- start start 2023-11-28T11:08:21.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.271+0000 7f122cb8a700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f122815cef0 0x7f12280af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.271+0000 7f122cb8a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f12280cbdf0 0x7f122806b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.271+0000 7f122cb8a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12280cc5f0 con 0x7f122806b8c0 2023-11-28T11:08:21.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.271+0000 7f122cb8a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122806bd40 con 0x7f12280cbdf0 2023-11-28T11:08:21.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.271+0000 7f122cb8a700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122815d470 con 0x7f122815cef0 2023-11-28T11:08:21.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f1225d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f12280cbdf0 0x7f122806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f1226d9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f122806b8c0 0x7f1228071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46724/0 (socket says 172.21.15.156:46724) 2023-11-28T11:08:21.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f1225d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f12280cbdf0 0x7f122806b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60074/0 (socket says 172.21.15.156:60074) 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f1226d9d700 1 -- 172.21.15.156:0/1820116492 learned_addr learned my addr 172.21.15.156:0/1820116492 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f122659c700 1 --2- 172.21.15.156:0/1820116492 >> v2:172.21.15.186:3300/0 conn(0x7f122815cef0 0x7f12280af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f1225d9b700 1 -- 172.21.15.156:0/1820116492 >> v2:172.21.15.186:3300/0 conn(0x7f122815cef0 msgr2=0x7f12280af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f1225d9b700 1 --2- 172.21.15.156:0/1820116492 >> v2:172.21.15.186:3300/0 conn(0x7f122815cef0 0x7f12280af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f1225d9b700 1 -- 172.21.15.156:0/1820116492 >> v1:172.21.15.156:6789/0 conn(0x7f122806b8c0 legacy=0x7f1228071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f1225d9b700 1 -- 172.21.15.156:0/1820116492 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122815d750 con 0x7f12280cbdf0 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f122559a700 1 -- 172.21.15.156:0/1820116492 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071876532 0 0) 0x7f12280cc5f0 con 0x7f122806b8c0 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f1225d9b700 1 --2- 172.21.15.156:0/1820116492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12280cbdf0 0x7f122806b380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f121800bae0 tx=0x7f121800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3c0dbaa46470bc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f122559a700 1 -- 172.21.15.156:0/1820116492 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121800dc30 con 0x7f12280cbdf0 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f122559a700 1 -- 172.21.15.156:0/1820116492 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121800dd90 con 0x7f12280cbdf0 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.272+0000 7f122559a700 1 -- 172.21.15.156:0/1820116492 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218010690 con 0x7f12280cbdf0 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.273+0000 7f122cb8a700 1 -- 172.21.15.156:0/1820116492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12280cbdf0 msgr2=0x7f122806b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.273+0000 7f122cb8a700 1 --2- 172.21.15.156:0/1820116492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12280cbdf0 0x7f122806b380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f121800bae0 tx=0x7f121800bdf0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.273+0000 7f122cb8a700 1 -- 172.21.15.156:0/1820116492 shutdown_connections 2023-11-28T11:08:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.273+0000 7f122cb8a700 1 --2- 172.21.15.156:0/1820116492 >> v2:172.21.15.186:3300/0 conn(0x7f122815cef0 0x7f12280af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.273+0000 7f122cb8a700 1 --2- 172.21.15.156:0/1820116492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12280cbdf0 0x7f122806b380 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.273+0000 7f122cb8a700 1 -- 172.21.15.156:0/1820116492 >> 172.21.15.156:0/1820116492 conn(0x7f12280d0000 msgr2=0x7f12280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.273+0000 7f122cb8a700 1 -- 172.21.15.156:0/1820116492 shutdown_connections 2023-11-28T11:08:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.273+0000 7f122cb8a700 1 -- 172.21.15.156:0/1820116492 wait complete. 2023-11-28T11:08:21.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.274+0000 7f122cb8a700 1 Processor -- start 2023-11-28T11:08:21.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.274+0000 7f122cb8a700 1 -- start start 2023-11-28T11:08:21.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f122cb8a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f122806b8c0 0x7f12280b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f122cb8a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12280cbdf0 0x7f12280c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f122cb8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122815cef0 0x7f12280cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f122cb8a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12280b0d40 con 0x7f122815cef0 2023-11-28T11:08:21.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f122cb8a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12280b0450 con 0x7f122806b8c0 2023-11-28T11:08:21.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f122cb8a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12280b06f0 con 0x7f12280cbdf0 2023-11-28T11:08:21.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f1226d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122815cef0 0x7f12280cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f1226d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122815cef0 0x7f12280cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60084/0 (socket says 172.21.15.156:60084) 2023-11-28T11:08:21.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f1226d9d700 1 -- 172.21.15.156:0/173766574 learned_addr learned my addr 172.21.15.156:0/173766574 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f122659c700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f122806b8c0 0x7f12280b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f1225d9b700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12280cbdf0 0x7f12280c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f1226d9d700 1 -- 172.21.15.156:0/173766574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12280cbdf0 msgr2=0x7f12280c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f1226d9d700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12280cbdf0 0x7f12280c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f1226d9d700 1 -- 172.21.15.156:0/173766574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f122806b8c0 msgr2=0x7f12280b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f1226d9d700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f122806b8c0 0x7f12280b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f1226d9d700 1 -- 172.21.15.156:0/173766574 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121800b740 con 0x7f122815cef0 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f1225d9b700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12280cbdf0 0x7f12280c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.275+0000 7f1226d9d700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122815cef0 0x7f12280cca10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f122000cb60 tx=0x7f122000ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.276+0000 7f1216ffd700 1 -- 172.21.15.156:0/173766574 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122000aaa0 con 0x7f122815cef0 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.276+0000 7f1216ffd700 1 -- 172.21.15.156:0/173766574 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122000ac00 con 0x7f122815cef0 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.276+0000 7f122cb8a700 1 -- 172.21.15.156:0/173766574 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122815dd10 con 0x7f122815cef0 2023-11-28T11:08:21.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.276+0000 7f1216ffd700 1 -- 172.21.15.156:0/173766574 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12200070d0 con 0x7f122815cef0 2023-11-28T11:08:21.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.276+0000 7f122cb8a700 1 -- 172.21.15.156:0/173766574 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122815da30 con 0x7f122815cef0 2023-11-28T11:08:21.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.276+0000 7f122cb8a700 1 -- 172.21.15.156:0/173766574 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122815dff0 con 0x7f122815cef0 2023-11-28T11:08:21.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.277+0000 7f1216ffd700 1 -- 172.21.15.156:0/173766574 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1220010070 con 0x7f122815cef0 2023-11-28T11:08:21.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.277+0000 7f1216ffd700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f12080682f0 0x7f120806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.277+0000 7f1216ffd700 1 -- 172.21.15.156:0/173766574 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1220085730 con 0x7f122815cef0 2023-11-28T11:08:21.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.277+0000 7f122659c700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f12080682f0 0x7f120806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.278+0000 7f122659c700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f12080682f0 0x7f120806a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f121c009be0 tx=0x7f121c009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:21.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.282+0000 7f1216ffd700 1 -- 172.21.15.156:0/173766574 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1220051ea0 con 0x7f122815cef0 2023-11-28T11:08:21.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.466+0000 7f122cb8a700 1 -- 172.21.15.156:0/173766574 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12280b0e80 con 0x7f122815cef0 2023-11-28T11:08:21.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.467+0000 7f1216ffd700 1 -- 172.21.15.156:0/173766574 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f1220018090 con 0x7f122815cef0 2023-11-28T11:08:21.471 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:18.354009+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:21.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.469+0000 7f122cb8a700 1 -- 172.21.15.156:0/173766574 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f12080682f0 msgr2=0x7f120806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.469+0000 7f122cb8a700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f12080682f0 0x7f120806a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f121c009be0 tx=0x7f121c009210 comp rx=0 tx=0).stop 2023-11-28T11:08:21.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.469+0000 7f122cb8a700 1 -- 172.21.15.156:0/173766574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122815cef0 msgr2=0x7f12280cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.469+0000 7f122cb8a700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122815cef0 0x7f12280cca10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f122000cb60 tx=0x7f122000ce70 comp rx=0 tx=0).stop 2023-11-28T11:08:21.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.469+0000 7f122cb8a700 1 -- 172.21.15.156:0/173766574 shutdown_connections 2023-11-28T11:08:21.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.469+0000 7f122cb8a700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122815cef0 0x7f12280cca10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.469+0000 7f122cb8a700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f122806b8c0 0x7f12280b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.469+0000 7f122cb8a700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f12080682f0 0x7f120806a7a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.469+0000 7f122cb8a700 1 --2- 172.21.15.156:0/173766574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12280cbdf0 0x7f12280c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.469+0000 7f122cb8a700 1 -- 172.21.15.156:0/173766574 >> 172.21.15.156:0/173766574 conn(0x7f12280d0000 msgr2=0x7f122806ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:21.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.469+0000 7f122cb8a700 1 -- 172.21.15.156:0/173766574 shutdown_connections 2023-11-28T11:08:21.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.469+0000 7f122cb8a700 1 -- 172.21.15.156:0/173766574 wait complete. 2023-11-28T11:08:21.483 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:08:21.484 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:08:21.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.604+0000 7f5230cee700 1 Processor -- start 2023-11-28T11:08:21.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.604+0000 7f5230cee700 1 -- start start 2023-11-28T11:08:21.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.605+0000 7f5230cee700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f522c071040 0x7f522c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.605+0000 7f5230cee700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f522c160540 0x7f522c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.605+0000 7f5230cee700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522c071950 con 0x7f522c06dee0 2023-11-28T11:08:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.605+0000 7f5230cee700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522c15d060 con 0x7f522c071040 2023-11-28T11:08:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.605+0000 7f5230cee700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c15d340 con 0x7f522c160540 2023-11-28T11:08:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.605+0000 7f5229d9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f522c06dee0 0x7f522c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46734/0 (socket says 172.21.15.156:46734) 2023-11-28T11:08:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.605+0000 7f5229d9b700 1 -- 172.21.15.156:0/3378946961 learned_addr learned my addr 172.21.15.156:0/3378946961 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.605+0000 7f522a59c700 1 --2- 172.21.15.156:0/3378946961 >> v2:172.21.15.186:3300/0 conn(0x7f522c071040 0x7f522c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.605+0000 7f522ad9d700 1 --2- 172.21.15.156:0/3378946961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f522c160540 0x7f522c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.606+0000 7f522959a700 1 -- 172.21.15.156:0/3378946961 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207154336 0 0) 0x7f522c071950 con 0x7f522c06dee0 2023-11-28T11:08:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.606+0000 7f522959a700 1 -- 172.21.15.156:0/3378946961 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52140035b0 con 0x7f522c06dee0 2023-11-28T11:08:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.606+0000 7f522ad9d700 1 -- 172.21.15.156:0/3378946961 >> v2:172.21.15.186:3300/0 conn(0x7f522c071040 msgr2=0x7f522c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.606+0000 7f522ad9d700 1 --2- 172.21.15.156:0/3378946961 >> v2:172.21.15.186:3300/0 conn(0x7f522c071040 0x7f522c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.606+0000 7f522ad9d700 1 -- 172.21.15.156:0/3378946961 >> v1:172.21.15.156:6789/0 conn(0x7f522c06dee0 legacy=0x7f522c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.606+0000 7f522ad9d700 1 -- 172.21.15.156:0/3378946961 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c15d620 con 0x7f522c160540 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.606+0000 7f522a59c700 1 --2- 172.21.15.156:0/3378946961 >> v2:172.21.15.186:3300/0 conn(0x7f522c071040 0x7f522c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.606+0000 7f522ad9d700 1 --2- 172.21.15.156:0/3378946961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f522c160540 0x7f522c1609c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5224009d90 tx=0x7f522400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b023fbfd54807a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.606+0000 7f522959a700 1 -- 172.21.15.156:0/3378946961 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522400e070 con 0x7f522c160540 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.606+0000 7f522959a700 1 -- 172.21.15.156:0/3378946961 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522400bb20 con 0x7f522c160540 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.606+0000 7f522959a700 1 -- 172.21.15.156:0/3378946961 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224012430 con 0x7f522c160540 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.607+0000 7f5230cee700 1 -- 172.21.15.156:0/3378946961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f522c160540 msgr2=0x7f522c1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.607+0000 7f5230cee700 1 --2- 172.21.15.156:0/3378946961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f522c160540 0x7f522c1609c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5224009d90 tx=0x7f522400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.607+0000 7f5230cee700 1 -- 172.21.15.156:0/3378946961 shutdown_connections 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.607+0000 7f5230cee700 1 --2- 172.21.15.156:0/3378946961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f522c160540 0x7f522c1609c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.607+0000 7f5230cee700 1 --2- 172.21.15.156:0/3378946961 >> v2:172.21.15.186:3300/0 conn(0x7f522c071040 0x7f522c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.607+0000 7f5230cee700 1 -- 172.21.15.156:0/3378946961 >> 172.21.15.156:0/3378946961 conn(0x7f522c0d0000 msgr2=0x7f522c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:21.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.607+0000 7f5230cee700 1 -- 172.21.15.156:0/3378946961 shutdown_connections 2023-11-28T11:08:21.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.607+0000 7f5230cee700 1 -- 172.21.15.156:0/3378946961 wait complete. 2023-11-28T11:08:21.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.608+0000 7f5230cee700 1 Processor -- start 2023-11-28T11:08:21.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.608+0000 7f5230cee700 1 -- start start 2023-11-28T11:08:21.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5230cee700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522c06dee0 0x7f522c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5230cee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f522c071040 0x7f522c0c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5230cee700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f522c160540 0x7f522c0c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5230cee700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c1f2750 con 0x7f522c071040 2023-11-28T11:08:21.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5230cee700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0c6920 con 0x7f522c06dee0 2023-11-28T11:08:21.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5230cee700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0b3690 con 0x7f522c160540 2023-11-28T11:08:21.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5229d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f522c071040 0x7f522c0c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5229d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f522c071040 0x7f522c0c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60094/0 (socket says 172.21.15.156:60094) 2023-11-28T11:08:21.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5229d9b700 1 -- 172.21.15.156:0/1888749253 learned_addr learned my addr 172.21.15.156:0/1888749253 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:21.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f522a59c700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522c06dee0 0x7f522c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f522ad9d700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f522c160540 0x7f522c0c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5229d9b700 1 -- 172.21.15.156:0/1888749253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f522c160540 msgr2=0x7f522c0c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5229d9b700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f522c160540 0x7f522c0c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5229d9b700 1 -- 172.21.15.156:0/1888749253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522c06dee0 msgr2=0x7f522c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5229d9b700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522c06dee0 0x7f522c0b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f5229d9b700 1 -- 172.21.15.156:0/1888749253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52200094a0 con 0x7f522c071040 2023-11-28T11:08:21.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.609+0000 7f522ad9d700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f522c160540 0x7f522c0c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:21.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.610+0000 7f5229d9b700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f522c071040 0x7f522c0c63e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f521c00c4d0 tx=0x7f521c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:21.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.610+0000 7f521affd700 1 -- 172.21.15.156:0/1888749253 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c005d70 con 0x7f522c071040 2023-11-28T11:08:21.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.610+0000 7f521affd700 1 -- 172.21.15.156:0/1888749253 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c0070d0 con 0x7f522c071040 2023-11-28T11:08:21.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.610+0000 7f5230cee700 1 -- 172.21.15.156:0/1888749253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5224009a70 con 0x7f522c071040 2023-11-28T11:08:21.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.610+0000 7f521affd700 1 -- 172.21.15.156:0/1888749253 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c007460 con 0x7f522c071040 2023-11-28T11:08:21.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.610+0000 7f5230cee700 1 -- 172.21.15.156:0/1888749253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c15ccd0 con 0x7f522c071040 2023-11-28T11:08:21.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.611+0000 7f5230cee700 1 -- 172.21.15.156:0/1888749253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522c15d900 con 0x7f522c071040 2023-11-28T11:08:21.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.611+0000 7f521affd700 1 -- 172.21.15.156:0/1888749253 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f521c019030 con 0x7f522c071040 2023-11-28T11:08:21.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.611+0000 7f521affd700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f520c0682f0 0x7f520c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.612+0000 7f522a59c700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f520c0682f0 0x7f520c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.612+0000 7f521affd700 1 -- 172.21.15.156:0/1888749253 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f521c084fe0 con 0x7f522c071040 2023-11-28T11:08:21.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.612+0000 7f522a59c700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f520c0682f0 0x7f520c06a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5220009cd0 tx=0x7f5220009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:21.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.616+0000 7f521affd700 1 -- 172.21.15.156:0/1888749253 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f521c051840 con 0x7f522c071040 2023-11-28T11:08:21.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.795+0000 7f5230cee700 1 -- 172.21.15.156:0/1888749253 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f522c06a270 con 0x7f520c0682f0 2023-11-28T11:08:21.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.803+0000 7f521affd700 1 -- 172.21.15.156:0/1888749253 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275778 (secure 0 0 0) 0x7f522c06a270 con 0x7f520c0682f0 2023-11-28T11:08:21.807 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:21.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.806+0000 7f5230cee700 1 -- 172.21.15.156:0/1888749253 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f520c0682f0 msgr2=0x7f520c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.806+0000 7f5230cee700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f520c0682f0 0x7f520c06a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5220009cd0 tx=0x7f5220009210 comp rx=0 tx=0).stop 2023-11-28T11:08:21.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.806+0000 7f5230cee700 1 -- 172.21.15.156:0/1888749253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f522c071040 msgr2=0x7f522c0c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.806+0000 7f5230cee700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f522c071040 0x7f522c0c63e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f521c00c4d0 tx=0x7f521c00c7e0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.806+0000 7f5230cee700 1 -- 172.21.15.156:0/1888749253 shutdown_connections 2023-11-28T11:08:21.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.806+0000 7f5230cee700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f520c0682f0 0x7f520c06a7a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.806+0000 7f5230cee700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f522c160540 0x7f522c0c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.806+0000 7f5230cee700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522c06dee0 0x7f522c0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.806+0000 7f5230cee700 1 --2- 172.21.15.156:0/1888749253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f522c071040 0x7f522c0c63e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.806+0000 7f5230cee700 1 -- 172.21.15.156:0/1888749253 >> 172.21.15.156:0/1888749253 conn(0x7f522c0d0000 msgr2=0x7f522c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:21.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.806+0000 7f5230cee700 1 -- 172.21.15.156:0/1888749253 shutdown_connections 2023-11-28T11:08:21.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.806+0000 7f5230cee700 1 -- 172.21.15.156:0/1888749253 wait complete. 2023-11-28T11:08:21.811 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:08:21.819 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-28T11:08:20.525511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":64704,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":61952,"kb_avail":754910016,"statfs":{"total":773094113280,"available":773027856384,"internally_reserved":0,"allocated":2752512,"data_stored":1697300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":63438848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.570133"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.963187+0000","last_change":"2023-11-28T11:08:15.963187+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.963187+0000","last_undegraded":"2023-11-28T11:08:15.963187+0000","last_fullsized":"2023-11-28T11:08:15.963187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.963573+0000","last_change":"2023-11-28T11:08:15.963573+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.963573+0000","last_undegraded":"2023-11-28T11:08:15.963573+0000","last_fullsized":"2023-11-28T11:08:15.963573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960116+0000","last_change":"2023-11-28T11:08:15.960116+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960116+0000","last_undegraded":"2023-11-28T11:08:15.960116+0000","last_fullsized":"2023-11-28T11:08:15.960116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.963633+0000","last_change":"2023-11-28T11:08:15.963633+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.963633+0000","last_undegraded":"2023-11-28T11:08:15.963633+0000","last_fullsized":"2023-11-28T11:08:15.963633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.957618+0000","last_change":"2023-11-28T11:08:15.957618+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.957618+0000","last_undegraded":"2023-11-28T11:08:15.957618+0000","last_fullsized":"2023-11-28T11:08:15.957618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960907+0000","last_change":"2023-11-28T11:08:15.960907+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960907+0000","last_undegraded":"2023-11-28T11:08:15.960907+0000","last_fullsized":"2023-11-28T11:08:15.960907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.962223+0000","last_change":"2023-11-28T11:08:15.962223+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.962223+0000","last_undegraded":"2023-11-28T11:08:15.962223+0000","last_fullsized":"2023-11-28T11:08:15.962223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:13.313856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960185+0000","last_change":"2023-11-28T11:08:15.960185+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960185+0000","last_undegraded":"2023-11-28T11:08:15.960185+0000","last_fullsized":"2023-11-28T11:08:15.960185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:36.981686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.964049+0000","last_change":"2023-11-28T11:08:15.964049+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.964049+0000","last_undegraded":"2023-11-28T11:08:15.964049+0000","last_fullsized":"2023-11-28T11:08:15.964049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960333+0000","last_change":"2023-11-28T11:08:15.960333+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960333+0000","last_undegraded":"2023-11-28T11:08:15.960333+0000","last_fullsized":"2023-11-28T11:08:15.960333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:55.975785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.964125+0000","last_change":"2023-11-28T11:08:15.964125+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.964125+0000","last_undegraded":"2023-11-28T11:08:15.964125+0000","last_fullsized":"2023-11-28T11:08:15.964125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960216+0000","last_change":"2023-11-28T11:08:15.960216+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960216+0000","last_undegraded":"2023-11-28T11:08:15.960216+0000","last_fullsized":"2023-11-28T11:08:15.960216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:59.917012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960301+0000","last_change":"2023-11-28T11:08:15.960301+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960301+0000","last_undegraded":"2023-11-28T11:08:15.960301+0000","last_fullsized":"2023-11-28T11:08:15.960301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:33.709483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960054+0000","last_change":"2023-11-28T11:08:15.960054+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960054+0000","last_undegraded":"2023-11-28T11:08:15.960054+0000","last_fullsized":"2023-11-28T11:08:15.960054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.957500+0000","last_change":"2023-11-28T11:08:15.957500+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.957500+0000","last_undegraded":"2023-11-28T11:08:15.957500+0000","last_fullsized":"2023-11-28T11:08:15.957500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.959539+0000","last_change":"2023-11-28T11:08:15.959539+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.959539+0000","last_undegraded":"2023-11-28T11:08:15.959539+0000","last_fullsized":"2023-11-28T11:08:15.959539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.964790+0000","last_change":"2023-11-28T11:08:15.964790+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.964790+0000","last_undegraded":"2023-11-28T11:08:15.964790+0000","last_fullsized":"2023-11-28T11:08:15.964790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.962160+0000","last_change":"2023-11-28T11:08:15.962160+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.962160+0000","last_undegraded":"2023-11-28T11:08:15.962160+0000","last_fullsized":"2023-11-28T11:08:15.962160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:43.575239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.959350+0000","last_change":"2023-11-28T11:08:15.959350+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.959350+0000","last_undegraded":"2023-11-28T11:08:15.959350+0000","last_fullsized":"2023-11-28T11:08:15.959350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-28T11:08:12.950564+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:08:12.950564+0000","last_peered":"2023-11-28T11:08:12.950564+0000","last_clean":"2023-11-28T11:08:12.950564+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:08:12.950564+0000","last_undegraded":"2023-11-28T11:08:12.950564+0000","last_fullsized":"2023-11-28T11:08:12.950564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.957572+0000","last_change":"2023-11-28T11:08:15.957572+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.957572+0000","last_undegraded":"2023-11-28T11:08:15.957572+0000","last_fullsized":"2023-11-28T11:08:15.957572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:08:15.959349+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:08:15.959349+0000","last_peered":"2023-11-28T11:08:15.959349+0000","last_clean":"2023-11-28T11:08:15.959349+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:08:15.959349+0000","last_undegraded":"2023-11-28T11:08:15.959349+0000","last_fullsized":"2023-11-28T11:08:15.959349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.962206+0000","last_change":"2023-11-28T11:08:15.962206+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.962206+0000","last_undegraded":"2023-11-28T11:08:15.962206+0000","last_fullsized":"2023-11-28T11:08:15.962206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:08:15.957132+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:08:15.957132+0000","last_peered":"2023-11-28T11:08:15.957132+0000","last_clean":"2023-11-28T11:08:15.957132+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:08:15.957132+0000","last_undegraded":"2023-11-28T11:08:15.957132+0000","last_fullsized":"2023-11-28T11:08:15.957132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.963450+0000","last_change":"2023-11-28T11:08:15.963450+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.963450+0000","last_undegraded":"2023-11-28T11:08:15.963450+0000","last_fullsized":"2023-11-28T11:08:15.963450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:08:15.959372+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:08:15.959372+0000","last_peered":"2023-11-28T11:08:15.959372+0000","last_clean":"2023-11-28T11:08:15.959372+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:08:15.959372+0000","last_undegraded":"2023-11-28T11:08:15.959372+0000","last_fullsized":"2023-11-28T11:08:15.959372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-28T11:08:15.564201+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:08:15.564201+0000","last_peered":"2023-11-28T11:08:15.564201+0000","last_clean":"2023-11-28T11:08:15.564201+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:08:15.564201+0000","last_undegraded":"2023-11-28T11:08:15.564201+0000","last_fullsized":"2023-11-28T11:08:15.564201+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.959320+0000","last_change":"2023-11-28T11:08:15.959320+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.959320+0000","last_undegraded":"2023-11-28T11:08:15.959320+0000","last_fullsized":"2023-11-28T11:08:15.959320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-28T11:08:15.564180+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:08:15.564180+0000","last_peered":"2023-11-28T11:08:15.564180+0000","last_clean":"2023-11-28T11:08:15.564180+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:08:15.564180+0000","last_undegraded":"2023-11-28T11:08:15.564180+0000","last_fullsized":"2023-11-28T11:08:15.564180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.964598+0000","last_change":"2023-11-28T11:08:15.964598+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.964598+0000","last_undegraded":"2023-11-28T11:08:15.964598+0000","last_fullsized":"2023-11-28T11:08:15.964598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-28T11:08:15.564216+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:08:15.564216+0000","last_peered":"2023-11-28T11:08:15.564216+0000","last_clean":"2023-11-28T11:08:15.564216+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:08:15.564216+0000","last_undegraded":"2023-11-28T11:08:15.564216+0000","last_fullsized":"2023-11-28T11:08:15.564216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.964343+0000","last_change":"2023-11-28T11:08:15.964343+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.964343+0000","last_undegraded":"2023-11-28T11:08:15.964343+0000","last_fullsized":"2023-11-28T11:08:15.964343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.959321+0000","last_change":"2023-11-28T11:08:15.959321+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.959321+0000","last_undegraded":"2023-11-28T11:08:15.959321+0000","last_fullsized":"2023-11-28T11:08:15.959321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-28T11:08:12.950607+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:08:12.950607+0000","last_peered":"2023-11-28T11:08:12.950607+0000","last_clean":"2023-11-28T11:08:12.950607+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:08:12.950607+0000","last_undegraded":"2023-11-28T11:08:12.950607+0000","last_fullsized":"2023-11-28T11:08:12.950607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-28T11:08:14.424683+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:08:14.424683+0000","last_peered":"2023-11-28T11:08:14.424683+0000","last_clean":"2023-11-28T11:08:14.424683+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:08:14.424683+0000","last_undegraded":"2023-11-28T11:08:14.424683+0000","last_fullsized":"2023-11-28T11:08:14.424683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.963612+0000","last_change":"2023-11-28T11:08:15.963612+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.963612+0000","last_undegraded":"2023-11-28T11:08:15.963612+0000","last_fullsized":"2023-11-28T11:08:15.963612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.963817+0000","last_change":"2023-11-28T11:08:15.963817+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.963817+0000","last_undegraded":"2023-11-28T11:08:15.963817+0000","last_fullsized":"2023-11-28T11:08:15.963817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:11.421587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.958096+0000","last_change":"2023-11-28T11:08:15.958096+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.958096+0000","last_undegraded":"2023-11-28T11:08:15.958096+0000","last_fullsized":"2023-11-28T11:08:15.958096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.958807+0000","last_change":"2023-11-28T11:08:15.958807+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.958807+0000","last_undegraded":"2023-11-28T11:08:15.958807+0000","last_fullsized":"2023-11-28T11:08:15.958807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960723+0000","last_change":"2023-11-28T11:08:15.960723+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960723+0000","last_undegraded":"2023-11-28T11:08:15.960723+0000","last_fullsized":"2023-11-28T11:08:15.960723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.964624+0000","last_change":"2023-11-28T11:08:15.964624+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.964624+0000","last_undegraded":"2023-11-28T11:08:15.964624+0000","last_fullsized":"2023-11-28T11:08:15.964624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8916,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94362924,"statfs":{"total":96636764160,"available":96627634176,"internally_reserved":0,"allocated":798720,"data_stored":659566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7812,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364028,"statfs":{"total":96636764160,"available":96628764672,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7812,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364028,"statfs":{"total":96636764160,"available":96628764672,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8916,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94362924,"statfs":{"total":96636764160,"available":96627634176,"internally_reserved":0,"allocated":798720,"data_stored":659566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7812,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364028,"statfs":{"total":96636764160,"available":96628764672,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7812,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364028,"statfs":{"total":96636764160,"available":96628764672,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7812,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364028,"statfs":{"total":96636764160,"available":96628764672,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7812,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364028,"statfs":{"total":96636764160,"available":96628764672,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:08:21.824 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:08:21.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.979+0000 7f5f2d70d700 1 Processor -- start 2023-11-28T11:08:21.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.979+0000 7f5f2d70d700 1 -- start start 2023-11-28T11:08:21.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.979+0000 7f5f2d70d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5f2815c9b0 0x7f5f280b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.979+0000 7f5f2d70d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f28161340 0x7f5f280d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.979+0000 7f5f2d70d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5f280d2320 0x7f5f280cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.979+0000 7f5f2d70d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f280b5800 con 0x7f5f2815c9b0 2023-11-28T11:08:21.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.979+0000 7f5f2d70d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f280d1950 con 0x7f5f280d2320 2023-11-28T11:08:21.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.979+0000 7f5f2d70d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f280b5520 con 0x7f5f28161340 2023-11-28T11:08:21.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f26ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5f2815c9b0 0x7f5f280b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f26ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5f2815c9b0 0x7f5f280b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60102/0 (socket says 172.21.15.156:60102) 2023-11-28T11:08:21.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f26ffd700 1 -- 172.21.15.156:0/3028378736 learned_addr learned my addr 172.21.15.156:0/3028378736 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f277fe700 1 --2- 172.21.15.156:0/3028378736 >> v2:172.21.15.186:3300/0 conn(0x7f5f280d2320 0x7f5f280cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f26ffd700 1 -- 172.21.15.156:0/3028378736 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f28161340 msgr2=0x7f5f280d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f267fc700 1 --2- 172.21.15.156:0/3028378736 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f28161340 0x7f5f280d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f26ffd700 1 --2- 172.21.15.156:0/3028378736 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f28161340 0x7f5f280d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f26ffd700 1 -- 172.21.15.156:0/3028378736 >> v2:172.21.15.186:3300/0 conn(0x7f5f280d2320 msgr2=0x7f5f280cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f26ffd700 1 --2- 172.21.15.156:0/3028378736 >> v2:172.21.15.186:3300/0 conn(0x7f5f280d2320 0x7f5f280cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f26ffd700 1 -- 172.21.15.156:0/3028378736 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f280d1300 con 0x7f5f2815c9b0 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f277fe700 1 --2- 172.21.15.156:0/3028378736 >> v2:172.21.15.186:3300/0 conn(0x7f5f280d2320 0x7f5f280cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f26ffd700 1 --2- 172.21.15.156:0/3028378736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f2815c9b0 0x7f5f280b4bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5f2808c4d0 tx=0x7f5f18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da571585ad4c5cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f25ffb700 1 -- 172.21.15.156:0/3028378736 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800b870 con 0x7f5f2815c9b0 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f25ffb700 1 -- 172.21.15.156:0/3028378736 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1800b9d0 con 0x7f5f2815c9b0 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.980+0000 7f5f25ffb700 1 -- 172.21.15.156:0/3028378736 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800bd60 con 0x7f5f2815c9b0 2023-11-28T11:08:21.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.981+0000 7f5f2d70d700 1 -- 172.21.15.156:0/3028378736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f2815c9b0 msgr2=0x7f5f280b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.981+0000 7f5f2d70d700 1 --2- 172.21.15.156:0/3028378736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f2815c9b0 0x7f5f280b4bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5f2808c4d0 tx=0x7f5f18009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.981+0000 7f5f2d70d700 1 -- 172.21.15.156:0/3028378736 shutdown_connections 2023-11-28T11:08:21.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.981+0000 7f5f2d70d700 1 --2- 172.21.15.156:0/3028378736 >> v2:172.21.15.186:3300/0 conn(0x7f5f280d2320 0x7f5f280cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.981+0000 7f5f2d70d700 1 --2- 172.21.15.156:0/3028378736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f2815c9b0 0x7f5f280b4bf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.981+0000 7f5f2d70d700 1 --2- 172.21.15.156:0/3028378736 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f28161340 0x7f5f280d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.981+0000 7f5f2d70d700 1 -- 172.21.15.156:0/3028378736 >> 172.21.15.156:0/3028378736 conn(0x7f5f28072600 msgr2=0x7f5f280cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:21.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.981+0000 7f5f2d70d700 1 -- 172.21.15.156:0/3028378736 shutdown_connections 2023-11-28T11:08:21.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.981+0000 7f5f2d70d700 1 -- 172.21.15.156:0/3028378736 wait complete. 2023-11-28T11:08:21.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.982+0000 7f5f2d70d700 1 Processor -- start 2023-11-28T11:08:21.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.983+0000 7f5f2d70d700 1 -- start start 2023-11-28T11:08:21.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.983+0000 7f5f2d70d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f280d2320 0x7f5f280c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.983+0000 7f5f2d70d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f2815c9b0 0x7f5f2814fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.983+0000 7f5f2d70d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f28161340 0x7f5f28150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.983+0000 7f5f2d70d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f280c6ca0 con 0x7f5f28161340 2023-11-28T11:08:21.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.983+0000 7f5f2d70d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f281e80a0 con 0x7f5f2815c9b0 2023-11-28T11:08:21.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.983+0000 7f5f2d70d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f281e8380 con 0x7f5f280d2320 2023-11-28T11:08:21.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f28161340 0x7f5f28150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f28161340 0x7f5f28150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60112/0 (socket says 172.21.15.156:60112) 2023-11-28T11:08:21.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f277fe700 1 -- 172.21.15.156:0/616556037 learned_addr learned my addr 172.21.15.156:0/616556037 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:21.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f267fc700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f2815c9b0 0x7f5f2814fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f26ffd700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f280d2320 0x7f5f280c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f277fe700 1 -- 172.21.15.156:0/616556037 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f280d2320 msgr2=0x7f5f280c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f277fe700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f280d2320 0x7f5f280c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f277fe700 1 -- 172.21.15.156:0/616556037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f2815c9b0 msgr2=0x7f5f2814fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:21.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f277fe700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f2815c9b0 0x7f5f2814fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:21.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f277fe700 1 -- 172.21.15.156:0/616556037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f18009550 con 0x7f5f28161340 2023-11-28T11:08:21.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f26ffd700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f280d2320 0x7f5f280c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:21.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f277fe700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f28161340 0x7f5f28150300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000cd50 tx=0x7f5f2000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:21.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f0f7fe700 1 -- 172.21.15.156:0/616556037 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000cea0 con 0x7f5f28161340 2023-11-28T11:08:21.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f0f7fe700 1 -- 172.21.15.156:0/616556037 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2000f480 con 0x7f5f28161340 2023-11-28T11:08:21.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f2d70d700 1 -- 172.21.15.156:0/616556037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f280b5130 con 0x7f5f28161340 2023-11-28T11:08:21.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f0f7fe700 1 -- 172.21.15.156:0/616556037 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000f810 con 0x7f5f28161340 2023-11-28T11:08:21.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.984+0000 7f5f2d70d700 1 -- 172.21.15.156:0/616556037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f280d15e0 con 0x7f5f28161340 2023-11-28T11:08:21.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.985+0000 7f5f2d70d700 1 -- 172.21.15.156:0/616556037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f280b52c0 con 0x7f5f28161340 2023-11-28T11:08:21.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.986+0000 7f5f0f7fe700 1 -- 172.21.15.156:0/616556037 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5f20009030 con 0x7f5f28161340 2023-11-28T11:08:21.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.986+0000 7f5f0f7fe700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5f140682f0 0x7f5f1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:21.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.986+0000 7f5f0f7fe700 1 -- 172.21.15.156:0/616556037 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5f20086590 con 0x7f5f28161340 2023-11-28T11:08:21.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.986+0000 7f5f26ffd700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5f140682f0 0x7f5f1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:21.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.986+0000 7f5f26ffd700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5f140682f0 0x7f5f1406a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f18014040 tx=0x7f5f18008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:21.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:21.990+0000 7f5f0f7fe700 1 -- 172.21.15.156:0/616556037 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5f20052df0 con 0x7f5f28161340 2023-11-28T11:08:22.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.170+0000 7f5f2d70d700 1 -- 172.21.15.156:0/616556037 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f2814eac0 con 0x7f5f140682f0 2023-11-28T11:08:22.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.177+0000 7f5f0f7fe700 1 -- 172.21.15.156:0/616556037 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275778 (secure 0 0 0) 0x7f5f2814eac0 con 0x7f5f140682f0 2023-11-28T11:08:22.185 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:22.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.180+0000 7f5f2d70d700 1 -- 172.21.15.156:0/616556037 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5f140682f0 msgr2=0x7f5f1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:22.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.180+0000 7f5f2d70d700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5f140682f0 0x7f5f1406a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f18014040 tx=0x7f5f18008030 comp rx=0 tx=0).stop 2023-11-28T11:08:22.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.180+0000 7f5f2d70d700 1 -- 172.21.15.156:0/616556037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f28161340 msgr2=0x7f5f28150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:22.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.180+0000 7f5f2d70d700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f28161340 0x7f5f28150300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000cd50 tx=0x7f5f2000a560 comp rx=0 tx=0).stop 2023-11-28T11:08:22.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.180+0000 7f5f2d70d700 1 -- 172.21.15.156:0/616556037 shutdown_connections 2023-11-28T11:08:22.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.180+0000 7f5f2d70d700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f28161340 0x7f5f28150300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.180+0000 7f5f2d70d700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5f140682f0 0x7f5f1406a7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.180+0000 7f5f2d70d700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f280d2320 0x7f5f280c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.180+0000 7f5f2d70d700 1 --2- 172.21.15.156:0/616556037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f2815c9b0 0x7f5f2814fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.180+0000 7f5f2d70d700 1 -- 172.21.15.156:0/616556037 >> 172.21.15.156:0/616556037 conn(0x7f5f28072600 msgr2=0x7f5f2815cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:22.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.180+0000 7f5f2d70d700 1 -- 172.21.15.156:0/616556037 shutdown_connections 2023-11-28T11:08:22.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.180+0000 7f5f2d70d700 1 -- 172.21.15.156:0/616556037 wait complete. 2023-11-28T11:08:22.192 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:08:22.194 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-28T11:08:20.525511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":64704,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":61952,"kb_avail":754910016,"statfs":{"total":773094113280,"available":773027856384,"internally_reserved":0,"allocated":2752512,"data_stored":1697300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":63438848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.570133"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.963187+0000","last_change":"2023-11-28T11:08:15.963187+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.963187+0000","last_undegraded":"2023-11-28T11:08:15.963187+0000","last_fullsized":"2023-11-28T11:08:15.963187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.963573+0000","last_change":"2023-11-28T11:08:15.963573+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.963573+0000","last_undegraded":"2023-11-28T11:08:15.963573+0000","last_fullsized":"2023-11-28T11:08:15.963573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960116+0000","last_change":"2023-11-28T11:08:15.960116+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960116+0000","last_undegraded":"2023-11-28T11:08:15.960116+0000","last_fullsized":"2023-11-28T11:08:15.960116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.963633+0000","last_change":"2023-11-28T11:08:15.963633+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.963633+0000","last_undegraded":"2023-11-28T11:08:15.963633+0000","last_fullsized":"2023-11-28T11:08:15.963633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.957618+0000","last_change":"2023-11-28T11:08:15.957618+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.957618+0000","last_undegraded":"2023-11-28T11:08:15.957618+0000","last_fullsized":"2023-11-28T11:08:15.957618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960907+0000","last_change":"2023-11-28T11:08:15.960907+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960907+0000","last_undegraded":"2023-11-28T11:08:15.960907+0000","last_fullsized":"2023-11-28T11:08:15.960907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.962223+0000","last_change":"2023-11-28T11:08:15.962223+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.962223+0000","last_undegraded":"2023-11-28T11:08:15.962223+0000","last_fullsized":"2023-11-28T11:08:15.962223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:13.313856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960185+0000","last_change":"2023-11-28T11:08:15.960185+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960185+0000","last_undegraded":"2023-11-28T11:08:15.960185+0000","last_fullsized":"2023-11-28T11:08:15.960185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:36.981686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.964049+0000","last_change":"2023-11-28T11:08:15.964049+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.964049+0000","last_undegraded":"2023-11-28T11:08:15.964049+0000","last_fullsized":"2023-11-28T11:08:15.964049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960333+0000","last_change":"2023-11-28T11:08:15.960333+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960333+0000","last_undegraded":"2023-11-28T11:08:15.960333+0000","last_fullsized":"2023-11-28T11:08:15.960333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:55.975785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.964125+0000","last_change":"2023-11-28T11:08:15.964125+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.964125+0000","last_undegraded":"2023-11-28T11:08:15.964125+0000","last_fullsized":"2023-11-28T11:08:15.964125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960216+0000","last_change":"2023-11-28T11:08:15.960216+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960216+0000","last_undegraded":"2023-11-28T11:08:15.960216+0000","last_fullsized":"2023-11-28T11:08:15.960216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:59.917012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960301+0000","last_change":"2023-11-28T11:08:15.960301+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960301+0000","last_undegraded":"2023-11-28T11:08:15.960301+0000","last_fullsized":"2023-11-28T11:08:15.960301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:33.709483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960054+0000","last_change":"2023-11-28T11:08:15.960054+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960054+0000","last_undegraded":"2023-11-28T11:08:15.960054+0000","last_fullsized":"2023-11-28T11:08:15.960054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.957500+0000","last_change":"2023-11-28T11:08:15.957500+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.957500+0000","last_undegraded":"2023-11-28T11:08:15.957500+0000","last_fullsized":"2023-11-28T11:08:15.957500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.959539+0000","last_change":"2023-11-28T11:08:15.959539+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.959539+0000","last_undegraded":"2023-11-28T11:08:15.959539+0000","last_fullsized":"2023-11-28T11:08:15.959539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.964790+0000","last_change":"2023-11-28T11:08:15.964790+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.964790+0000","last_undegraded":"2023-11-28T11:08:15.964790+0000","last_fullsized":"2023-11-28T11:08:15.964790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.962160+0000","last_change":"2023-11-28T11:08:15.962160+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.962160+0000","last_undegraded":"2023-11-28T11:08:15.962160+0000","last_fullsized":"2023-11-28T11:08:15.962160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:43.575239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.959350+0000","last_change":"2023-11-28T11:08:15.959350+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.959350+0000","last_undegraded":"2023-11-28T11:08:15.959350+0000","last_fullsized":"2023-11-28T11:08:15.959350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-28T11:08:12.950564+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:08:12.950564+0000","last_peered":"2023-11-28T11:08:12.950564+0000","last_clean":"2023-11-28T11:08:12.950564+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:08:12.950564+0000","last_undegraded":"2023-11-28T11:08:12.950564+0000","last_fullsized":"2023-11-28T11:08:12.950564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.957572+0000","last_change":"2023-11-28T11:08:15.957572+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.957572+0000","last_undegraded":"2023-11-28T11:08:15.957572+0000","last_fullsized":"2023-11-28T11:08:15.957572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:08:15.959349+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:08:15.959349+0000","last_peered":"2023-11-28T11:08:15.959349+0000","last_clean":"2023-11-28T11:08:15.959349+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:08:15.959349+0000","last_undegraded":"2023-11-28T11:08:15.959349+0000","last_fullsized":"2023-11-28T11:08:15.959349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.962206+0000","last_change":"2023-11-28T11:08:15.962206+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.962206+0000","last_undegraded":"2023-11-28T11:08:15.962206+0000","last_fullsized":"2023-11-28T11:08:15.962206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:08:15.957132+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:08:15.957132+0000","last_peered":"2023-11-28T11:08:15.957132+0000","last_clean":"2023-11-28T11:08:15.957132+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:08:15.957132+0000","last_undegraded":"2023-11-28T11:08:15.957132+0000","last_fullsized":"2023-11-28T11:08:15.957132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.963450+0000","last_change":"2023-11-28T11:08:15.963450+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.963450+0000","last_undegraded":"2023-11-28T11:08:15.963450+0000","last_fullsized":"2023-11-28T11:08:15.963450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T11:08:15.959372+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:08:15.959372+0000","last_peered":"2023-11-28T11:08:15.959372+0000","last_clean":"2023-11-28T11:08:15.959372+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:08:15.959372+0000","last_undegraded":"2023-11-28T11:08:15.959372+0000","last_fullsized":"2023-11-28T11:08:15.959372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-28T11:08:15.564201+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:08:15.564201+0000","last_peered":"2023-11-28T11:08:15.564201+0000","last_clean":"2023-11-28T11:08:15.564201+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:08:15.564201+0000","last_undegraded":"2023-11-28T11:08:15.564201+0000","last_fullsized":"2023-11-28T11:08:15.564201+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.959320+0000","last_change":"2023-11-28T11:08:15.959320+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.959320+0000","last_undegraded":"2023-11-28T11:08:15.959320+0000","last_fullsized":"2023-11-28T11:08:15.959320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-28T11:08:15.564180+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:08:15.564180+0000","last_peered":"2023-11-28T11:08:15.564180+0000","last_clean":"2023-11-28T11:08:15.564180+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:08:15.564180+0000","last_undegraded":"2023-11-28T11:08:15.564180+0000","last_fullsized":"2023-11-28T11:08:15.564180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.964598+0000","last_change":"2023-11-28T11:08:15.964598+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.964598+0000","last_undegraded":"2023-11-28T11:08:15.964598+0000","last_fullsized":"2023-11-28T11:08:15.964598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-28T11:08:15.564216+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:08:15.564216+0000","last_peered":"2023-11-28T11:08:15.564216+0000","last_clean":"2023-11-28T11:08:15.564216+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:08:15.564216+0000","last_undegraded":"2023-11-28T11:08:15.564216+0000","last_fullsized":"2023-11-28T11:08:15.564216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.964343+0000","last_change":"2023-11-28T11:08:15.964343+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.964343+0000","last_undegraded":"2023-11-28T11:08:15.964343+0000","last_fullsized":"2023-11-28T11:08:15.964343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.959321+0000","last_change":"2023-11-28T11:08:15.959321+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.959321+0000","last_undegraded":"2023-11-28T11:08:15.959321+0000","last_fullsized":"2023-11-28T11:08:15.959321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-28T11:08:12.950607+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:08:12.950607+0000","last_peered":"2023-11-28T11:08:12.950607+0000","last_clean":"2023-11-28T11:08:12.950607+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:08:12.950607+0000","last_undegraded":"2023-11-28T11:08:12.950607+0000","last_fullsized":"2023-11-28T11:08:12.950607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-28T11:08:14.424683+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:08:14.424683+0000","last_peered":"2023-11-28T11:08:14.424683+0000","last_clean":"2023-11-28T11:08:14.424683+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:08:14.424683+0000","last_undegraded":"2023-11-28T11:08:14.424683+0000","last_fullsized":"2023-11-28T11:08:14.424683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.963612+0000","last_change":"2023-11-28T11:08:15.963612+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.963612+0000","last_undegraded":"2023-11-28T11:08:15.963612+0000","last_fullsized":"2023-11-28T11:08:15.963612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.963817+0000","last_change":"2023-11-28T11:08:15.963817+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.963817+0000","last_undegraded":"2023-11-28T11:08:15.963817+0000","last_fullsized":"2023-11-28T11:08:15.963817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:11.421587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.958096+0000","last_change":"2023-11-28T11:08:15.958096+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.958096+0000","last_undegraded":"2023-11-28T11:08:15.958096+0000","last_fullsized":"2023-11-28T11:08:15.958096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.958807+0000","last_change":"2023-11-28T11:08:15.958807+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.958807+0000","last_undegraded":"2023-11-28T11:08:15.958807+0000","last_fullsized":"2023-11-28T11:08:15.958807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.960723+0000","last_change":"2023-11-28T11:08:15.960723+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.960723+0000","last_undegraded":"2023-11-28T11:08:15.960723+0000","last_fullsized":"2023-11-28T11:08:15.960723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2023-11-28T11:08:15.964624+0000","last_change":"2023-11-28T11:08:15.964624+0000","last_active":"2023-11-28T11:08:15.953025+0000","last_peered":"2023-11-28T11:08:15.953025+0000","last_clean":"2023-11-28T11:08:15.953025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:15.964624+0000","last_undegraded":"2023-11-28T11:08:15.964624+0000","last_fullsized":"2023-11-28T11:08:15.964624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:08:16.954910+0000","last_change":"2023-11-28T11:08:16.954910+0000","last_active":"2023-11-28T11:08:16.954910+0000","last_peered":"2023-11-28T11:08:16.954910+0000","last_clean":"2023-11-28T11:08:16.954910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:08:16.954910+0000","last_undegraded":"2023-11-28T11:08:16.954910+0000","last_fullsized":"2023-11-28T11:08:16.954910+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8916,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94362924,"statfs":{"total":96636764160,"available":96627634176,"internally_reserved":0,"allocated":798720,"data_stored":659566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7812,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364028,"statfs":{"total":96636764160,"available":96628764672,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7812,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364028,"statfs":{"total":96636764160,"available":96628764672,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8916,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94362924,"statfs":{"total":96636764160,"available":96627634176,"internally_reserved":0,"allocated":798720,"data_stored":659566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7812,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364028,"statfs":{"total":96636764160,"available":96628764672,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7812,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364028,"statfs":{"total":96636764160,"available":96628764672,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7812,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364028,"statfs":{"total":96636764160,"available":96628764672,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7812,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364028,"statfs":{"total":96636764160,"available":96628764672,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:08:22.199 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:08:22.355 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.354+0000 7f0c2acc1700 1 Processor -- start 2023-11-28T11:08:22.355 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.354+0000 7f0c2acc1700 1 -- start start 2023-11-28T11:08:22.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.354+0000 7f0c2acc1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0c240cd8c0 0x7f0c24140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.354+0000 7f0c2acc1700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c240cdc90 0x7f0c240b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.354+0000 7f0c2acc1700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2413ff00 con 0x7f0c240b2e70 2023-11-28T11:08:22.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.354+0000 7f0c2acc1700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c241404c0 con 0x7f0c240cd8c0 2023-11-28T11:08:22.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.354+0000 7f0c2acc1700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2415aa80 con 0x7f0c240cdc90 2023-11-28T11:08:22.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.355+0000 7f0c28a5d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0c240cd8c0 0x7f0c24140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.355+0000 7f0c28a5d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0c240cd8c0 0x7f0c24140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60116/0 (socket says 172.21.15.156:60116) 2023-11-28T11:08:22.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.355+0000 7f0c28a5d700 1 -- 172.21.15.156:0/1374737283 learned_addr learned my addr 172.21.15.156:0/1374737283 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:22.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.355+0000 7f0c23fff700 1 --2- 172.21.15.156:0/1374737283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c240cdc90 0x7f0c240b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.355+0000 7f0c28a5d700 1 -- 172.21.15.156:0/1374737283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c240cdc90 msgr2=0x7f0c240b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:22.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.355+0000 7f0c28a5d700 1 --2- 172.21.15.156:0/1374737283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c240cdc90 0x7f0c240b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.355+0000 7f0c28a5d700 1 -- 172.21.15.156:0/1374737283 >> v1:172.21.15.186:6789/0 conn(0x7f0c240b2e70 legacy=0x7f0c2406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:22.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.355+0000 7f0c28a5d700 1 -- 172.21.15.156:0/1374737283 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c240b3310 con 0x7f0c240cd8c0 2023-11-28T11:08:22.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.355+0000 7f0c28a5d700 1 --2- 172.21.15.156:0/1374737283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c240cd8c0 0x7f0c24140950 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0c180098e0 tx=0x7f0c18009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77fea029fbc4959 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:22.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.355+0000 7f0c237fe700 1 -- 172.21.15.156:0/1374737283 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18011830 con 0x7f0c240cd8c0 2023-11-28T11:08:22.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.355+0000 7f0c237fe700 1 -- 172.21.15.156:0/1374737283 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c18011990 con 0x7f0c240cd8c0 2023-11-28T11:08:22.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.355+0000 7f0c237fe700 1 -- 172.21.15.156:0/1374737283 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18011d40 con 0x7f0c240cd8c0 2023-11-28T11:08:22.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.356+0000 7f0c2acc1700 1 -- 172.21.15.156:0/1374737283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c240cd8c0 msgr2=0x7f0c24140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:22.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.356+0000 7f0c2acc1700 1 --2- 172.21.15.156:0/1374737283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c240cd8c0 0x7f0c24140950 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0c180098e0 tx=0x7f0c18009bf0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.356+0000 7f0c2acc1700 1 -- 172.21.15.156:0/1374737283 shutdown_connections 2023-11-28T11:08:22.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.356+0000 7f0c2acc1700 1 --2- 172.21.15.156:0/1374737283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c240cd8c0 0x7f0c24140950 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.356+0000 7f0c2acc1700 1 --2- 172.21.15.156:0/1374737283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c240cdc90 0x7f0c240b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.356+0000 7f0c2acc1700 1 -- 172.21.15.156:0/1374737283 >> 172.21.15.156:0/1374737283 conn(0x7f0c2406aed0 msgr2=0x7f0c2406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:22.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.356+0000 7f0c2acc1700 1 -- 172.21.15.156:0/1374737283 shutdown_connections 2023-11-28T11:08:22.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.356+0000 7f0c2acc1700 1 -- 172.21.15.156:0/1374737283 wait complete. 2023-11-28T11:08:22.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.357+0000 7f0c2acc1700 1 Processor -- start 2023-11-28T11:08:22.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.357+0000 7f0c2acc1700 1 -- start start 2023-11-28T11:08:22.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c2acc1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c240b2e70 0x7f0c240c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c2acc1700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c240cd8c0 0x7f0c24143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c2acc1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c240cdc90 0x7f0c24143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c2acc1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2414b830 con 0x7f0c240b2e70 2023-11-28T11:08:22.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c2acc1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c24141060 con 0x7f0c240cdc90 2023-11-28T11:08:22.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c2acc1700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240c5ad0 con 0x7f0c240cd8c0 2023-11-28T11:08:22.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c28a5d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c240b2e70 0x7f0c240c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c28a5d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c240b2e70 0x7f0c240c53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60122/0 (socket says 172.21.15.156:60122) 2023-11-28T11:08:22.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c28a5d700 1 -- 172.21.15.156:0/2407560333 learned_addr learned my addr 172.21.15.156:0/2407560333 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:22.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c2925e700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c240cdc90 0x7f0c24143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c23fff700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c240cd8c0 0x7f0c24143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c28a5d700 1 -- 172.21.15.156:0/2407560333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c240cd8c0 msgr2=0x7f0c24143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:22.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c28a5d700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c240cd8c0 0x7f0c24143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c28a5d700 1 -- 172.21.15.156:0/2407560333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c240cdc90 msgr2=0x7f0c24143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:22.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c28a5d700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c240cdc90 0x7f0c24143ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c28a5d700 1 -- 172.21.15.156:0/2407560333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c180094f0 con 0x7f0c240b2e70 2023-11-28T11:08:22.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.358+0000 7f0c28a5d700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c240b2e70 0x7f0c240c53f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0c18000c00 tx=0x7f0c18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:22.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.359+0000 7f0c217fa700 1 -- 172.21.15.156:0/2407560333 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1801a840 con 0x7f0c240b2e70 2023-11-28T11:08:22.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.359+0000 7f0c217fa700 1 -- 172.21.15.156:0/2407560333 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1801f070 con 0x7f0c240b2e70 2023-11-28T11:08:22.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.359+0000 7f0c2acc1700 1 -- 172.21.15.156:0/2407560333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c24140150 con 0x7f0c240b2e70 2023-11-28T11:08:22.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.359+0000 7f0c217fa700 1 -- 172.21.15.156:0/2407560333 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1801ac00 con 0x7f0c240b2e70 2023-11-28T11:08:22.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.359+0000 7f0c2acc1700 1 -- 172.21.15.156:0/2407560333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c240b35f0 con 0x7f0c240b2e70 2023-11-28T11:08:22.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.360+0000 7f0c2acc1700 1 -- 172.21.15.156:0/2407560333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c240ce100 con 0x7f0c240b2e70 2023-11-28T11:08:22.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.360+0000 7f0c217fa700 1 -- 172.21.15.156:0/2407560333 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c1800f050 con 0x7f0c240b2e70 2023-11-28T11:08:22.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.360+0000 7f0c217fa700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c080682f0 0x7f0c0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.360+0000 7f0c217fa700 1 -- 172.21.15.156:0/2407560333 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0c18021070 con 0x7f0c240b2e70 2023-11-28T11:08:22.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.360+0000 7f0c23fff700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c080682f0 0x7f0c0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.361+0000 7f0c23fff700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c080682f0 0x7f0c0806a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0c240d1ae0 tx=0x7f0c10009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:22.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.365+0000 7f0c217fa700 1 -- 172.21.15.156:0/2407560333 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0c18053ca0 con 0x7f0c240b2e70 2023-11-28T11:08:22.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.541+0000 7f0c2acc1700 1 -- 172.21.15.156:0/2407560333 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c24068690 con 0x7f0c080682f0 2023-11-28T11:08:22.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.549+0000 7f0c217fa700 1 -- 172.21.15.156:0/2407560333 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286716 (secure 0 0 0) 0x7f0c24068690 con 0x7f0c080682f0 2023-11-28T11:08:22.554 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:22.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.552+0000 7f0c2acc1700 1 -- 172.21.15.156:0/2407560333 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c080682f0 msgr2=0x7f0c0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:22.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.552+0000 7f0c2acc1700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c080682f0 0x7f0c0806a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0c240d1ae0 tx=0x7f0c10009210 comp rx=0 tx=0).stop 2023-11-28T11:08:22.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.552+0000 7f0c2acc1700 1 -- 172.21.15.156:0/2407560333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c240b2e70 msgr2=0x7f0c240c53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:22.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.552+0000 7f0c2acc1700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c240b2e70 0x7f0c240c53f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0c18000c00 tx=0x7f0c18004750 comp rx=0 tx=0).stop 2023-11-28T11:08:22.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.552+0000 7f0c2acc1700 1 -- 172.21.15.156:0/2407560333 shutdown_connections 2023-11-28T11:08:22.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.552+0000 7f0c2acc1700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c240cdc90 0x7f0c24143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.552+0000 7f0c2acc1700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c240b2e70 0x7f0c240c53f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.552+0000 7f0c2acc1700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c080682f0 0x7f0c0806a7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.552+0000 7f0c2acc1700 1 --2- 172.21.15.156:0/2407560333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c240cd8c0 0x7f0c24143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.552+0000 7f0c2acc1700 1 -- 172.21.15.156:0/2407560333 >> 172.21.15.156:0/2407560333 conn(0x7f0c2406aed0 msgr2=0x7f0c2413ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:22.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.552+0000 7f0c2acc1700 1 -- 172.21.15.156:0/2407560333 shutdown_connections 2023-11-28T11:08:22.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.552+0000 7f0c2acc1700 1 -- 172.21.15.156:0/2407560333 wait complete. 2023-11-28T11:08:22.557 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:08:22.565 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-28T11:08:22.526031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":198,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66216,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":754908504,"statfs":{"total":773094113280,"available":773026308096,"internally_reserved":0,"allocated":3186688,"data_stored":1902188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001301"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072799+0000","last_change":"2023-11-28T11:08:17.367751+0000","last_active":"2023-11-28T11:08:19.072799+0000","last_peered":"2023-11-28T11:08:19.072799+0000","last_clean":"2023-11-28T11:08:19.072799+0000","last_became_active":"2023-11-28T11:08:17.367481+0000","last_became_peered":"2023-11-28T11:08:17.367481+0000","last_unstale":"2023-11-28T11:08:19.072799+0000","last_undegraded":"2023-11-28T11:08:19.072799+0000","last_fullsized":"2023-11-28T11:08:19.072799+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:09.760690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073854+0000","last_change":"2023-11-28T11:08:17.369749+0000","last_active":"2023-11-28T11:08:19.073854+0000","last_peered":"2023-11-28T11:08:19.073854+0000","last_clean":"2023-11-28T11:08:19.073854+0000","last_became_active":"2023-11-28T11:08:17.369479+0000","last_became_peered":"2023-11-28T11:08:17.369479+0000","last_unstale":"2023-11-28T11:08:19.073854+0000","last_undegraded":"2023-11-28T11:08:19.073854+0000","last_fullsized":"2023-11-28T11:08:19.073854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369159+0000","last_change":"2023-11-28T11:08:17.370726+0000","last_active":"2023-11-28T11:08:18.369159+0000","last_peered":"2023-11-28T11:08:18.369159+0000","last_clean":"2023-11-28T11:08:18.369159+0000","last_became_active":"2023-11-28T11:08:17.367062+0000","last_became_peered":"2023-11-28T11:08:17.367062+0000","last_unstale":"2023-11-28T11:08:18.369159+0000","last_undegraded":"2023-11-28T11:08:18.369159+0000","last_fullsized":"2023-11-28T11:08:18.369159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:27.219209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073872+0000","last_change":"2023-11-28T11:08:17.383989+0000","last_active":"2023-11-28T11:08:19.073872+0000","last_peered":"2023-11-28T11:08:19.073872+0000","last_clean":"2023-11-28T11:08:19.073872+0000","last_became_active":"2023-11-28T11:08:17.381618+0000","last_became_peered":"2023-11-28T11:08:17.381618+0000","last_unstale":"2023-11-28T11:08:19.073872+0000","last_undegraded":"2023-11-28T11:08:19.073872+0000","last_fullsized":"2023-11-28T11:08:19.073872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:09:00.918897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673454+0000","last_change":"2023-11-28T11:08:17.377110+0000","last_active":"2023-11-28T11:08:18.673454+0000","last_peered":"2023-11-28T11:08:18.673454+0000","last_clean":"2023-11-28T11:08:18.673454+0000","last_became_active":"2023-11-28T11:08:17.373583+0000","last_became_peered":"2023-11-28T11:08:17.373583+0000","last_unstale":"2023-11-28T11:08:18.673454+0000","last_undegraded":"2023-11-28T11:08:18.673454+0000","last_fullsized":"2023-11-28T11:08:18.673454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:24:59.277184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368780+0000","last_change":"2023-11-28T11:08:17.370700+0000","last_active":"2023-11-28T11:08:18.368780+0000","last_peered":"2023-11-28T11:08:18.368780+0000","last_clean":"2023-11-28T11:08:18.368780+0000","last_became_active":"2023-11-28T11:08:17.367222+0000","last_became_peered":"2023-11-28T11:08:17.367222+0000","last_unstale":"2023-11-28T11:08:18.368780+0000","last_undegraded":"2023-11-28T11:08:18.368780+0000","last_fullsized":"2023-11-28T11:08:18.368780+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:22.140101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073497+0000","last_change":"2023-11-28T11:08:17.374848+0000","last_active":"2023-11-28T11:08:19.073497+0000","last_peered":"2023-11-28T11:08:19.073497+0000","last_clean":"2023-11-28T11:08:19.073497+0000","last_became_active":"2023-11-28T11:08:17.374536+0000","last_became_peered":"2023-11-28T11:08:17.374536+0000","last_unstale":"2023-11-28T11:08:19.073497+0000","last_undegraded":"2023-11-28T11:08:19.073497+0000","last_fullsized":"2023-11-28T11:08:19.073497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074252+0000","last_change":"2023-11-28T11:08:17.366207+0000","last_active":"2023-11-28T11:08:19.074252+0000","last_peered":"2023-11-28T11:08:19.074252+0000","last_clean":"2023-11-28T11:08:19.074252+0000","last_became_active":"2023-11-28T11:08:17.365918+0000","last_became_peered":"2023-11-28T11:08:17.365918+0000","last_unstale":"2023-11-28T11:08:19.074252+0000","last_undegraded":"2023-11-28T11:08:19.074252+0000","last_fullsized":"2023-11-28T11:08:19.074252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673623+0000","last_change":"2023-11-28T11:08:17.376435+0000","last_active":"2023-11-28T11:08:18.673623+0000","last_peered":"2023-11-28T11:08:18.673623+0000","last_clean":"2023-11-28T11:08:18.673623+0000","last_became_active":"2023-11-28T11:08:17.373543+0000","last_became_peered":"2023-11-28T11:08:17.373543+0000","last_unstale":"2023-11-28T11:08:18.673623+0000","last_undegraded":"2023-11-28T11:08:18.673623+0000","last_fullsized":"2023-11-28T11:08:18.673623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074030+0000","last_change":"2023-11-28T11:08:17.367284+0000","last_active":"2023-11-28T11:08:19.074030+0000","last_peered":"2023-11-28T11:08:19.074030+0000","last_clean":"2023-11-28T11:08:19.074030+0000","last_became_active":"2023-11-28T11:08:17.366992+0000","last_became_peered":"2023-11-28T11:08:17.366992+0000","last_unstale":"2023-11-28T11:08:19.074030+0000","last_undegraded":"2023-11-28T11:08:19.074030+0000","last_fullsized":"2023-11-28T11:08:19.074030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369258+0000","last_change":"2023-11-28T11:08:17.376061+0000","last_active":"2023-11-28T11:08:18.369258+0000","last_peered":"2023-11-28T11:08:18.369258+0000","last_clean":"2023-11-28T11:08:18.369258+0000","last_became_active":"2023-11-28T11:08:17.375789+0000","last_became_peered":"2023-11-28T11:08:17.375789+0000","last_unstale":"2023-11-28T11:08:18.369258+0000","last_undegraded":"2023-11-28T11:08:18.369258+0000","last_fullsized":"2023-11-28T11:08:18.369258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:10.040305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072686+0000","last_change":"2023-11-28T11:08:17.372872+0000","last_active":"2023-11-28T11:08:19.072686+0000","last_peered":"2023-11-28T11:08:19.072686+0000","last_clean":"2023-11-28T11:08:19.072686+0000","last_became_active":"2023-11-28T11:08:17.372624+0000","last_became_peered":"2023-11-28T11:08:17.372624+0000","last_unstale":"2023-11-28T11:08:19.072686+0000","last_undegraded":"2023-11-28T11:08:19.072686+0000","last_fullsized":"2023-11-28T11:08:19.072686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:34.711368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072664+0000","last_change":"2023-11-28T11:08:17.374949+0000","last_active":"2023-11-28T11:08:19.072664+0000","last_peered":"2023-11-28T11:08:19.072664+0000","last_clean":"2023-11-28T11:08:19.072664+0000","last_became_active":"2023-11-28T11:08:17.374690+0000","last_became_peered":"2023-11-28T11:08:17.374690+0000","last_unstale":"2023-11-28T11:08:19.072664+0000","last_undegraded":"2023-11-28T11:08:19.072664+0000","last_fullsized":"2023-11-28T11:08:19.072664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:52.301973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073186+0000","last_change":"2023-11-28T11:08:17.361341+0000","last_active":"2023-11-28T11:08:19.073186+0000","last_peered":"2023-11-28T11:08:19.073186+0000","last_clean":"2023-11-28T11:08:19.073186+0000","last_became_active":"2023-11-28T11:08:17.361053+0000","last_became_peered":"2023-11-28T11:08:17.361053+0000","last_unstale":"2023-11-28T11:08:19.073186+0000","last_undegraded":"2023-11-28T11:08:19.073186+0000","last_fullsized":"2023-11-28T11:08:19.073186+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073689+0000","last_change":"2023-11-28T11:08:17.377341+0000","last_active":"2023-11-28T11:08:19.073689+0000","last_peered":"2023-11-28T11:08:19.073689+0000","last_clean":"2023-11-28T11:08:19.073689+0000","last_became_active":"2023-11-28T11:08:17.376840+0000","last_became_peered":"2023-11-28T11:08:17.376840+0000","last_unstale":"2023-11-28T11:08:19.073689+0000","last_undegraded":"2023-11-28T11:08:19.073689+0000","last_fullsized":"2023-11-28T11:08:19.073689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368722+0000","last_change":"2023-11-28T11:08:17.370526+0000","last_active":"2023-11-28T11:08:18.368722+0000","last_peered":"2023-11-28T11:08:18.368722+0000","last_clean":"2023-11-28T11:08:18.368722+0000","last_became_active":"2023-11-28T11:08:17.367599+0000","last_became_peered":"2023-11-28T11:08:17.367599+0000","last_unstale":"2023-11-28T11:08:18.368722+0000","last_undegraded":"2023-11-28T11:08:18.368722+0000","last_fullsized":"2023-11-28T11:08:18.368722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:21.226857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673664+0000","last_change":"2023-11-28T11:08:17.377283+0000","last_active":"2023-11-28T11:08:18.673664+0000","last_peered":"2023-11-28T11:08:18.673664+0000","last_clean":"2023-11-28T11:08:18.673664+0000","last_became_active":"2023-11-28T11:08:17.373226+0000","last_became_peered":"2023-11-28T11:08:17.373226+0000","last_unstale":"2023-11-28T11:08:18.673664+0000","last_undegraded":"2023-11-28T11:08:18.673664+0000","last_fullsized":"2023-11-28T11:08:18.673664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369217+0000","last_change":"2023-11-28T11:08:17.373771+0000","last_active":"2023-11-28T11:08:18.369217+0000","last_peered":"2023-11-28T11:08:18.369217+0000","last_clean":"2023-11-28T11:08:18.369217+0000","last_became_active":"2023-11-28T11:08:17.373510+0000","last_became_peered":"2023-11-28T11:08:17.373510+0000","last_unstale":"2023-11-28T11:08:18.369217+0000","last_undegraded":"2023-11-28T11:08:18.369217+0000","last_fullsized":"2023-11-28T11:08:18.369217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:14.696335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072793+0000","last_change":"2023-11-28T11:08:17.381345+0000","last_active":"2023-11-28T11:08:19.072793+0000","last_peered":"2023-11-28T11:08:19.072793+0000","last_clean":"2023-11-28T11:08:19.072793+0000","last_became_active":"2023-11-28T11:08:17.381044+0000","last_became_peered":"2023-11-28T11:08:17.381044+0000","last_unstale":"2023-11-28T11:08:19.072793+0000","last_undegraded":"2023-11-28T11:08:19.072793+0000","last_fullsized":"2023-11-28T11:08:19.072793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:09.877040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369092+0000","last_change":"2023-11-28T11:08:17.387391+0000","last_active":"2023-11-28T11:08:18.369092+0000","last_peered":"2023-11-28T11:08:18.369092+0000","last_clean":"2023-11-28T11:08:18.369092+0000","last_became_active":"2023-11-28T11:08:17.387127+0000","last_became_peered":"2023-11-28T11:08:17.387127+0000","last_unstale":"2023-11-28T11:08:18.369092+0000","last_undegraded":"2023-11-28T11:08:18.369092+0000","last_fullsized":"2023-11-28T11:08:18.369092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074429+0000","last_change":"2023-11-28T11:08:17.380537+0000","last_active":"2023-11-28T11:08:19.074429+0000","last_peered":"2023-11-28T11:08:19.074429+0000","last_clean":"2023-11-28T11:08:19.074429+0000","last_became_active":"2023-11-28T11:08:17.380267+0000","last_became_peered":"2023-11-28T11:08:17.380267+0000","last_unstale":"2023-11-28T11:08:19.074429+0000","last_undegraded":"2023-11-28T11:08:19.074429+0000","last_fullsized":"2023-11-28T11:08:19.074429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673493+0000","last_change":"2023-11-28T11:08:17.371837+0000","last_active":"2023-11-28T11:08:18.673493+0000","last_peered":"2023-11-28T11:08:18.673493+0000","last_clean":"2023-11-28T11:08:18.673493+0000","last_became_active":"2023-11-28T11:08:17.371543+0000","last_became_peered":"2023-11-28T11:08:17.371543+0000","last_unstale":"2023-11-28T11:08:18.673493+0000","last_undegraded":"2023-11-28T11:08:18.673493+0000","last_fullsized":"2023-11-28T11:08:18.673493+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365642+0000","last_change":"2023-11-28T11:08:17.370684+0000","last_active":"2023-11-28T11:08:18.365642+0000","last_peered":"2023-11-28T11:08:18.365642+0000","last_clean":"2023-11-28T11:08:18.365642+0000","last_became_active":"2023-11-28T11:08:17.367787+0000","last_became_peered":"2023-11-28T11:08:17.367787+0000","last_unstale":"2023-11-28T11:08:18.365642+0000","last_undegraded":"2023-11-28T11:08:18.365642+0000","last_fullsized":"2023-11-28T11:08:18.365642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:59.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073418+0000","last_change":"2023-11-28T11:08:17.373809+0000","last_active":"2023-11-28T11:08:19.073418+0000","last_peered":"2023-11-28T11:08:19.073418+0000","last_clean":"2023-11-28T11:08:19.073418+0000","last_became_active":"2023-11-28T11:08:17.373529+0000","last_became_peered":"2023-11-28T11:08:17.373529+0000","last_unstale":"2023-11-28T11:08:19.073418+0000","last_undegraded":"2023-11-28T11:08:19.073418+0000","last_fullsized":"2023-11-28T11:08:19.073418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368409+0000","last_change":"2023-11-28T11:08:17.382413+0000","last_active":"2023-11-28T11:08:18.368409+0000","last_peered":"2023-11-28T11:08:18.368409+0000","last_clean":"2023-11-28T11:08:18.368409+0000","last_became_active":"2023-11-28T11:08:17.382132+0000","last_became_peered":"2023-11-28T11:08:17.382132+0000","last_unstale":"2023-11-28T11:08:18.368409+0000","last_undegraded":"2023-11-28T11:08:18.368409+0000","last_fullsized":"2023-11-28T11:08:18.368409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368518+0000","last_change":"2023-11-28T11:08:17.368106+0000","last_active":"2023-11-28T11:08:18.368518+0000","last_peered":"2023-11-28T11:08:18.368518+0000","last_clean":"2023-11-28T11:08:18.368518+0000","last_became_active":"2023-11-28T11:08:17.367562+0000","last_became_peered":"2023-11-28T11:08:17.367562+0000","last_unstale":"2023-11-28T11:08:18.368518+0000","last_undegraded":"2023-11-28T11:08:18.368518+0000","last_fullsized":"2023-11-28T11:08:18.368518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073955+0000","last_change":"2023-11-28T11:08:17.375521+0000","last_active":"2023-11-28T11:08:19.073955+0000","last_peered":"2023-11-28T11:08:19.073955+0000","last_clean":"2023-11-28T11:08:19.073955+0000","last_became_active":"2023-11-28T11:08:17.375256+0000","last_became_peered":"2023-11-28T11:08:17.375256+0000","last_unstale":"2023-11-28T11:08:19.073955+0000","last_undegraded":"2023-11-28T11:08:19.073955+0000","last_fullsized":"2023-11-28T11:08:19.073955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.382174+0000","last_change":"2023-11-28T11:08:16.975491+0000","last_active":"2023-11-28T11:08:17.382174+0000","last_peered":"2023-11-28T11:08:17.382174+0000","last_clean":"2023-11-28T11:08:17.382174+0000","last_became_active":"2023-11-28T11:08:16.975076+0000","last_became_peered":"2023-11-28T11:08:16.975076+0000","last_unstale":"2023-11-28T11:08:17.382174+0000","last_undegraded":"2023-11-28T11:08:17.382174+0000","last_fullsized":"2023-11-28T11:08:17.382174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365845+0000","last_change":"2023-11-28T11:08:17.371715+0000","last_active":"2023-11-28T11:08:18.365845+0000","last_peered":"2023-11-28T11:08:18.365845+0000","last_clean":"2023-11-28T11:08:18.365845+0000","last_became_active":"2023-11-28T11:08:17.368939+0000","last_became_peered":"2023-11-28T11:08:17.368939+0000","last_unstale":"2023-11-28T11:08:18.365845+0000","last_undegraded":"2023-11-28T11:08:18.365845+0000","last_fullsized":"2023-11-28T11:08:18.365845+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362665+0000","last_change":"2023-11-28T11:08:16.975573+0000","last_active":"2023-11-28T11:08:17.362665+0000","last_peered":"2023-11-28T11:08:17.362665+0000","last_clean":"2023-11-28T11:08:17.362665+0000","last_became_active":"2023-11-28T11:08:16.975216+0000","last_became_peered":"2023-11-28T11:08:16.975216+0000","last_unstale":"2023-11-28T11:08:17.362665+0000","last_undegraded":"2023-11-28T11:08:17.362665+0000","last_fullsized":"2023-11-28T11:08:17.362665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673721+0000","last_change":"2023-11-28T11:08:17.378243+0000","last_active":"2023-11-28T11:08:18.673721+0000","last_peered":"2023-11-28T11:08:18.673721+0000","last_clean":"2023-11-28T11:08:18.673721+0000","last_became_active":"2023-11-28T11:08:17.377866+0000","last_became_peered":"2023-11-28T11:08:17.377866+0000","last_unstale":"2023-11-28T11:08:18.673721+0000","last_undegraded":"2023-11-28T11:08:18.673721+0000","last_fullsized":"2023-11-28T11:08:18.673721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.356027+0000","last_change":"2023-11-28T11:08:16.972779+0000","last_active":"2023-11-28T11:08:17.356027+0000","last_peered":"2023-11-28T11:08:17.356027+0000","last_clean":"2023-11-28T11:08:17.356027+0000","last_became_active":"2023-11-28T11:08:16.972510+0000","last_became_peered":"2023-11-28T11:08:16.972510+0000","last_unstale":"2023-11-28T11:08:17.356027+0000","last_undegraded":"2023-11-28T11:08:17.356027+0000","last_fullsized":"2023-11-28T11:08:17.356027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074026+0000","last_change":"2023-11-28T11:08:17.380638+0000","last_active":"2023-11-28T11:08:19.074026+0000","last_peered":"2023-11-28T11:08:19.074026+0000","last_clean":"2023-11-28T11:08:19.074026+0000","last_became_active":"2023-11-28T11:08:17.380333+0000","last_became_peered":"2023-11-28T11:08:17.380333+0000","last_unstale":"2023-11-28T11:08:19.074026+0000","last_undegraded":"2023-11-28T11:08:19.074026+0000","last_fullsized":"2023-11-28T11:08:19.074026+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:34.711368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.361717+0000","last_change":"2023-11-28T11:08:16.976366+0000","last_active":"2023-11-28T11:08:17.361717+0000","last_peered":"2023-11-28T11:08:17.361717+0000","last_clean":"2023-11-28T11:08:17.361717+0000","last_became_active":"2023-11-28T11:08:16.976072+0000","last_became_peered":"2023-11-28T11:08:16.976072+0000","last_unstale":"2023-11-28T11:08:17.361717+0000","last_undegraded":"2023-11-28T11:08:17.361717+0000","last_fullsized":"2023-11-28T11:08:17.361717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365642+0000","last_change":"2023-11-28T11:08:17.364878+0000","last_active":"2023-11-28T11:08:18.365642+0000","last_peered":"2023-11-28T11:08:18.365642+0000","last_clean":"2023-11-28T11:08:18.365642+0000","last_became_active":"2023-11-28T11:08:17.364628+0000","last_became_peered":"2023-11-28T11:08:17.364628+0000","last_unstale":"2023-11-28T11:08:18.365642+0000","last_undegraded":"2023-11-28T11:08:18.365642+0000","last_fullsized":"2023-11-28T11:08:18.365642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:12.679494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.358873+0000","last_change":"2023-11-28T11:08:16.975388+0000","last_active":"2023-11-28T11:08:17.358873+0000","last_peered":"2023-11-28T11:08:17.358873+0000","last_clean":"2023-11-28T11:08:17.358873+0000","last_became_active":"2023-11-28T11:08:16.974977+0000","last_became_peered":"2023-11-28T11:08:16.974977+0000","last_unstale":"2023-11-28T11:08:17.358873+0000","last_undegraded":"2023-11-28T11:08:17.358873+0000","last_fullsized":"2023-11-28T11:08:17.358873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365734+0000","last_change":"2023-11-28T11:08:17.376435+0000","last_active":"2023-11-28T11:08:18.365734+0000","last_peered":"2023-11-28T11:08:18.365734+0000","last_clean":"2023-11-28T11:08:18.365734+0000","last_became_active":"2023-11-28T11:08:17.376170+0000","last_became_peered":"2023-11-28T11:08:17.376170+0000","last_unstale":"2023-11-28T11:08:18.365734+0000","last_undegraded":"2023-11-28T11:08:18.365734+0000","last_fullsized":"2023-11-28T11:08:18.365734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:24:59.277184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.381284+0000","last_change":"2023-11-28T11:08:16.975413+0000","last_active":"2023-11-28T11:08:17.381284+0000","last_peered":"2023-11-28T11:08:17.381284+0000","last_clean":"2023-11-28T11:08:17.381284+0000","last_became_active":"2023-11-28T11:08:16.975058+0000","last_became_peered":"2023-11-28T11:08:16.975058+0000","last_unstale":"2023-11-28T11:08:17.381284+0000","last_undegraded":"2023-11-28T11:08:17.381284+0000","last_fullsized":"2023-11-28T11:08:17.381284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368111+0000","last_change":"2023-11-28T11:08:17.381210+0000","last_active":"2023-11-28T11:08:18.368111+0000","last_peered":"2023-11-28T11:08:18.368111+0000","last_clean":"2023-11-28T11:08:18.368111+0000","last_became_active":"2023-11-28T11:08:17.380941+0000","last_became_peered":"2023-11-28T11:08:17.380941+0000","last_unstale":"2023-11-28T11:08:18.368111+0000","last_undegraded":"2023-11-28T11:08:18.368111+0000","last_fullsized":"2023-11-28T11:08:18.368111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:35.096193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.387746+0000","last_change":"2023-11-28T11:08:16.973937+0000","last_active":"2023-11-28T11:08:17.387746+0000","last_peered":"2023-11-28T11:08:17.387746+0000","last_clean":"2023-11-28T11:08:17.387746+0000","last_became_active":"2023-11-28T11:08:16.973594+0000","last_became_peered":"2023-11-28T11:08:16.973594+0000","last_unstale":"2023-11-28T11:08:17.387746+0000","last_undegraded":"2023-11-28T11:08:17.387746+0000","last_fullsized":"2023-11-28T11:08:17.387746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:13.313856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369025+0000","last_change":"2023-11-28T11:08:17.380894+0000","last_active":"2023-11-28T11:08:18.369025+0000","last_peered":"2023-11-28T11:08:18.369025+0000","last_clean":"2023-11-28T11:08:18.369025+0000","last_became_active":"2023-11-28T11:08:17.380600+0000","last_became_peered":"2023-11-28T11:08:17.380600+0000","last_unstale":"2023-11-28T11:08:18.369025+0000","last_undegraded":"2023-11-28T11:08:18.369025+0000","last_fullsized":"2023-11-28T11:08:18.369025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377908+0000","last_change":"2023-11-28T11:08:16.977393+0000","last_active":"2023-11-28T11:08:17.377908+0000","last_peered":"2023-11-28T11:08:17.377908+0000","last_clean":"2023-11-28T11:08:17.377908+0000","last_became_active":"2023-11-28T11:08:16.976585+0000","last_became_peered":"2023-11-28T11:08:16.976585+0000","last_unstale":"2023-11-28T11:08:17.377908+0000","last_undegraded":"2023-11-28T11:08:17.377908+0000","last_fullsized":"2023-11-28T11:08:17.377908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:36.981686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365857+0000","last_change":"2023-11-28T11:08:17.376235+0000","last_active":"2023-11-28T11:08:18.365857+0000","last_peered":"2023-11-28T11:08:18.365857+0000","last_clean":"2023-11-28T11:08:18.365857+0000","last_became_active":"2023-11-28T11:08:17.375675+0000","last_became_peered":"2023-11-28T11:08:17.375675+0000","last_unstale":"2023-11-28T11:08:18.365857+0000","last_undegraded":"2023-11-28T11:08:18.365857+0000","last_fullsized":"2023-11-28T11:08:18.365857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.359416+0000","last_change":"2023-11-28T11:08:16.974456+0000","last_active":"2023-11-28T11:08:17.359416+0000","last_peered":"2023-11-28T11:08:17.359416+0000","last_clean":"2023-11-28T11:08:17.359416+0000","last_became_active":"2023-11-28T11:08:16.974158+0000","last_became_peered":"2023-11-28T11:08:16.974158+0000","last_unstale":"2023-11-28T11:08:17.359416+0000","last_undegraded":"2023-11-28T11:08:17.359416+0000","last_fullsized":"2023-11-28T11:08:17.359416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074033+0000","last_change":"2023-11-28T11:08:17.375410+0000","last_active":"2023-11-28T11:08:19.074033+0000","last_peered":"2023-11-28T11:08:19.074033+0000","last_clean":"2023-11-28T11:08:19.074033+0000","last_became_active":"2023-11-28T11:08:17.374828+0000","last_became_peered":"2023-11-28T11:08:17.374828+0000","last_unstale":"2023-11-28T11:08:19.074033+0000","last_undegraded":"2023-11-28T11:08:19.074033+0000","last_fullsized":"2023-11-28T11:08:19.074033+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365862+0000","last_change":"2023-11-28T11:08:16.978277+0000","last_active":"2023-11-28T11:08:17.365862+0000","last_peered":"2023-11-28T11:08:17.365862+0000","last_clean":"2023-11-28T11:08:17.365862+0000","last_became_active":"2023-11-28T11:08:16.977994+0000","last_became_peered":"2023-11-28T11:08:16.977994+0000","last_unstale":"2023-11-28T11:08:17.365862+0000","last_undegraded":"2023-11-28T11:08:17.365862+0000","last_fullsized":"2023-11-28T11:08:17.365862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:55.975785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673702+0000","last_change":"2023-11-28T11:08:17.369372+0000","last_active":"2023-11-28T11:08:18.673702+0000","last_peered":"2023-11-28T11:08:18.673702+0000","last_clean":"2023-11-28T11:08:18.673702+0000","last_became_active":"2023-11-28T11:08:17.369115+0000","last_became_peered":"2023-11-28T11:08:17.369115+0000","last_unstale":"2023-11-28T11:08:18.673702+0000","last_undegraded":"2023-11-28T11:08:18.673702+0000","last_fullsized":"2023-11-28T11:08:18.673702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.386102+0000","last_change":"2023-11-28T11:08:16.974290+0000","last_active":"2023-11-28T11:08:17.386102+0000","last_peered":"2023-11-28T11:08:17.386102+0000","last_clean":"2023-11-28T11:08:17.386102+0000","last_became_active":"2023-11-28T11:08:16.973952+0000","last_became_peered":"2023-11-28T11:08:16.973952+0000","last_unstale":"2023-11-28T11:08:17.386102+0000","last_undegraded":"2023-11-28T11:08:17.386102+0000","last_fullsized":"2023-11-28T11:08:17.386102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365978+0000","last_change":"2023-11-28T11:08:17.376526+0000","last_active":"2023-11-28T11:08:18.365978+0000","last_peered":"2023-11-28T11:08:18.365978+0000","last_clean":"2023-11-28T11:08:18.365978+0000","last_became_active":"2023-11-28T11:08:17.376285+0000","last_became_peered":"2023-11-28T11:08:17.376285+0000","last_unstale":"2023-11-28T11:08:18.365978+0000","last_undegraded":"2023-11-28T11:08:18.365978+0000","last_fullsized":"2023-11-28T11:08:18.365978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377922+0000","last_change":"2023-11-28T11:08:16.976879+0000","last_active":"2023-11-28T11:08:17.377922+0000","last_peered":"2023-11-28T11:08:17.377922+0000","last_clean":"2023-11-28T11:08:17.377922+0000","last_became_active":"2023-11-28T11:08:16.976530+0000","last_became_peered":"2023-11-28T11:08:16.976530+0000","last_unstale":"2023-11-28T11:08:17.377922+0000","last_undegraded":"2023-11-28T11:08:17.377922+0000","last_fullsized":"2023-11-28T11:08:17.377922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:59.917012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073041+0000","last_change":"2023-11-28T11:08:17.375192+0000","last_active":"2023-11-28T11:08:19.073041+0000","last_peered":"2023-11-28T11:08:19.073041+0000","last_clean":"2023-11-28T11:08:19.073041+0000","last_became_active":"2023-11-28T11:08:17.374903+0000","last_became_peered":"2023-11-28T11:08:17.374903+0000","last_unstale":"2023-11-28T11:08:19.073041+0000","last_undegraded":"2023-11-28T11:08:19.073041+0000","last_fullsized":"2023-11-28T11:08:19.073041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377956+0000","last_change":"2023-11-28T11:08:16.980934+0000","last_active":"2023-11-28T11:08:17.377956+0000","last_peered":"2023-11-28T11:08:17.377956+0000","last_clean":"2023-11-28T11:08:17.377956+0000","last_became_active":"2023-11-28T11:08:16.980585+0000","last_became_peered":"2023-11-28T11:08:16.980585+0000","last_unstale":"2023-11-28T11:08:17.377956+0000","last_undegraded":"2023-11-28T11:08:17.377956+0000","last_fullsized":"2023-11-28T11:08:17.377956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:33.709483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368728+0000","last_change":"2023-11-28T11:08:17.374922+0000","last_active":"2023-11-28T11:08:18.368728+0000","last_peered":"2023-11-28T11:08:18.368728+0000","last_clean":"2023-11-28T11:08:18.368728+0000","last_became_active":"2023-11-28T11:08:17.374631+0000","last_became_peered":"2023-11-28T11:08:17.374631+0000","last_unstale":"2023-11-28T11:08:18.368728+0000","last_undegraded":"2023-11-28T11:08:18.368728+0000","last_fullsized":"2023-11-28T11:08:18.368728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:28:26.671538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.375561+0000","last_change":"2023-11-28T11:08:16.978233+0000","last_active":"2023-11-28T11:08:17.375561+0000","last_peered":"2023-11-28T11:08:17.375561+0000","last_clean":"2023-11-28T11:08:17.375561+0000","last_became_active":"2023-11-28T11:08:16.977953+0000","last_became_peered":"2023-11-28T11:08:16.977953+0000","last_unstale":"2023-11-28T11:08:17.375561+0000","last_undegraded":"2023-11-28T11:08:17.375561+0000","last_fullsized":"2023-11-28T11:08:17.375561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073837+0000","last_change":"2023-11-28T11:08:17.381282+0000","last_active":"2023-11-28T11:08:19.073837+0000","last_peered":"2023-11-28T11:08:19.073837+0000","last_clean":"2023-11-28T11:08:19.073837+0000","last_became_active":"2023-11-28T11:08:17.381013+0000","last_became_peered":"2023-11-28T11:08:17.381013+0000","last_unstale":"2023-11-28T11:08:19.073837+0000","last_undegraded":"2023-11-28T11:08:19.073837+0000","last_fullsized":"2023-11-28T11:08:19.073837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:09.760690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357896+0000","last_change":"2023-11-28T11:08:16.976378+0000","last_active":"2023-11-28T11:08:17.357896+0000","last_peered":"2023-11-28T11:08:17.357896+0000","last_clean":"2023-11-28T11:08:17.357896+0000","last_became_active":"2023-11-28T11:08:16.974426+0000","last_became_peered":"2023-11-28T11:08:16.974426+0000","last_unstale":"2023-11-28T11:08:17.357896+0000","last_undegraded":"2023-11-28T11:08:17.357896+0000","last_fullsized":"2023-11-28T11:08:17.357896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673645+0000","last_change":"2023-11-28T11:08:17.366381+0000","last_active":"2023-11-28T11:08:18.673645+0000","last_peered":"2023-11-28T11:08:18.673645+0000","last_clean":"2023-11-28T11:08:18.673645+0000","last_became_active":"2023-11-28T11:08:17.366129+0000","last_became_peered":"2023-11-28T11:08:17.366129+0000","last_unstale":"2023-11-28T11:08:18.673645+0000","last_undegraded":"2023-11-28T11:08:18.673645+0000","last_fullsized":"2023-11-28T11:08:18.673645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.380305+0000","last_change":"2023-11-28T11:08:16.973851+0000","last_active":"2023-11-28T11:08:17.380305+0000","last_peered":"2023-11-28T11:08:17.380305+0000","last_clean":"2023-11-28T11:08:17.380305+0000","last_became_active":"2023-11-28T11:08:16.973588+0000","last_became_peered":"2023-11-28T11:08:16.973588+0000","last_unstale":"2023-11-28T11:08:17.380305+0000","last_undegraded":"2023-11-28T11:08:17.380305+0000","last_fullsized":"2023-11-28T11:08:17.380305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369164+0000","last_change":"2023-11-28T11:08:17.372448+0000","last_active":"2023-11-28T11:08:18.369164+0000","last_peered":"2023-11-28T11:08:18.369164+0000","last_clean":"2023-11-28T11:08:18.369164+0000","last_became_active":"2023-11-28T11:08:17.372132+0000","last_became_peered":"2023-11-28T11:08:17.372132+0000","last_unstale":"2023-11-28T11:08:18.369164+0000","last_undegraded":"2023-11-28T11:08:18.369164+0000","last_fullsized":"2023-11-28T11:08:18.369164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:12.679494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.371186+0000","last_change":"2023-11-28T11:08:16.975018+0000","last_active":"2023-11-28T11:08:17.371186+0000","last_peered":"2023-11-28T11:08:17.371186+0000","last_clean":"2023-11-28T11:08:17.371186+0000","last_became_active":"2023-11-28T11:08:16.974709+0000","last_became_peered":"2023-11-28T11:08:16.974709+0000","last_unstale":"2023-11-28T11:08:17.371186+0000","last_undegraded":"2023-11-28T11:08:17.371186+0000","last_fullsized":"2023-11-28T11:08:17.371186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369338+0000","last_change":"2023-11-28T11:08:17.372568+0000","last_active":"2023-11-28T11:08:18.369338+0000","last_peered":"2023-11-28T11:08:18.369338+0000","last_clean":"2023-11-28T11:08:18.369338+0000","last_became_active":"2023-11-28T11:08:17.372279+0000","last_became_peered":"2023-11-28T11:08:17.372279+0000","last_unstale":"2023-11-28T11:08:18.369338+0000","last_undegraded":"2023-11-28T11:08:18.369338+0000","last_fullsized":"2023-11-28T11:08:18.369338+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:02.566272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364207+0000","last_change":"2023-11-28T11:08:16.976973+0000","last_active":"2023-11-28T11:08:17.364207+0000","last_peered":"2023-11-28T11:08:17.364207+0000","last_clean":"2023-11-28T11:08:17.364207+0000","last_became_active":"2023-11-28T11:08:16.976661+0000","last_became_peered":"2023-11-28T11:08:16.976661+0000","last_unstale":"2023-11-28T11:08:17.364207+0000","last_undegraded":"2023-11-28T11:08:17.364207+0000","last_fullsized":"2023-11-28T11:08:17.364207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:43.575239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369260+0000","last_change":"2023-11-28T11:08:17.381198+0000","last_active":"2023-11-28T11:08:18.369260+0000","last_peered":"2023-11-28T11:08:18.369260+0000","last_clean":"2023-11-28T11:08:18.369260+0000","last_became_active":"2023-11-28T11:08:17.380937+0000","last_became_peered":"2023-11-28T11:08:17.380937+0000","last_unstale":"2023-11-28T11:08:18.369260+0000","last_undegraded":"2023-11-28T11:08:18.369260+0000","last_fullsized":"2023-11-28T11:08:18.369260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.355350+0000","last_change":"2023-11-28T11:08:16.966279+0000","last_active":"2023-11-28T11:08:17.355350+0000","last_peered":"2023-11-28T11:08:17.355350+0000","last_clean":"2023-11-28T11:08:17.355350+0000","last_became_active":"2023-11-28T11:08:16.965878+0000","last_became_peered":"2023-11-28T11:08:16.965878+0000","last_unstale":"2023-11-28T11:08:17.355350+0000","last_undegraded":"2023-11-28T11:08:17.355350+0000","last_fullsized":"2023-11-28T11:08:17.355350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368520+0000","last_change":"2023-11-28T11:08:17.373270+0000","last_active":"2023-11-28T11:08:18.368520+0000","last_peered":"2023-11-28T11:08:18.368520+0000","last_clean":"2023-11-28T11:08:18.368520+0000","last_became_active":"2023-11-28T11:08:17.373017+0000","last_became_peered":"2023-11-28T11:08:17.373017+0000","last_unstale":"2023-11-28T11:08:18.368520+0000","last_undegraded":"2023-11-28T11:08:18.368520+0000","last_fullsized":"2023-11-28T11:08:18.368520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365774+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:08:17.365774+0000","last_peered":"2023-11-28T11:08:17.365774+0000","last_clean":"2023-11-28T11:08:17.365774+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:08:17.365774+0000","last_undegraded":"2023-11-28T11:08:17.365774+0000","last_fullsized":"2023-11-28T11:08:17.365774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.358583+0000","last_change":"2023-11-28T11:08:16.975874+0000","last_active":"2023-11-28T11:08:17.358583+0000","last_peered":"2023-11-28T11:08:17.358583+0000","last_clean":"2023-11-28T11:08:17.358583+0000","last_became_active":"2023-11-28T11:08:16.975225+0000","last_became_peered":"2023-11-28T11:08:16.975225+0000","last_unstale":"2023-11-28T11:08:17.358583+0000","last_undegraded":"2023-11-28T11:08:17.358583+0000","last_fullsized":"2023-11-28T11:08:17.358583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368149+0000","last_change":"2023-11-28T11:08:17.371631+0000","last_active":"2023-11-28T11:08:18.368149+0000","last_peered":"2023-11-28T11:08:18.368149+0000","last_clean":"2023-11-28T11:08:18.368149+0000","last_became_active":"2023-11-28T11:08:17.371342+0000","last_became_peered":"2023-11-28T11:08:17.371342+0000","last_unstale":"2023-11-28T11:08:18.368149+0000","last_undegraded":"2023-11-28T11:08:18.368149+0000","last_fullsized":"2023-11-28T11:08:18.368149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357509+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:08:17.357509+0000","last_peered":"2023-11-28T11:08:17.357509+0000","last_clean":"2023-11-28T11:08:17.357509+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:08:17.357509+0000","last_undegraded":"2023-11-28T11:08:17.357509+0000","last_fullsized":"2023-11-28T11:08:17.357509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362711+0000","last_change":"2023-11-28T11:08:16.974904+0000","last_active":"2023-11-28T11:08:17.362711+0000","last_peered":"2023-11-28T11:08:17.362711+0000","last_clean":"2023-11-28T11:08:17.362711+0000","last_became_active":"2023-11-28T11:08:16.974634+0000","last_became_peered":"2023-11-28T11:08:16.974634+0000","last_unstale":"2023-11-28T11:08:17.362711+0000","last_undegraded":"2023-11-28T11:08:17.362711+0000","last_fullsized":"2023-11-28T11:08:17.362711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368410+0000","last_change":"2023-11-28T11:08:17.371181+0000","last_active":"2023-11-28T11:08:18.368410+0000","last_peered":"2023-11-28T11:08:18.368410+0000","last_clean":"2023-11-28T11:08:18.368410+0000","last_became_active":"2023-11-28T11:08:17.370931+0000","last_became_peered":"2023-11-28T11:08:17.370931+0000","last_unstale":"2023-11-28T11:08:18.368410+0000","last_undegraded":"2023-11-28T11:08:18.368410+0000","last_fullsized":"2023-11-28T11:08:18.368410+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.359510+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:08:17.359510+0000","last_peered":"2023-11-28T11:08:17.359510+0000","last_clean":"2023-11-28T11:08:17.359510+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:08:17.359510+0000","last_undegraded":"2023-11-28T11:08:17.359510+0000","last_fullsized":"2023-11-28T11:08:17.359510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362780+0000","last_change":"2023-11-28T11:08:16.972359+0000","last_active":"2023-11-28T11:08:17.362780+0000","last_peered":"2023-11-28T11:08:17.362780+0000","last_clean":"2023-11-28T11:08:17.362780+0000","last_became_active":"2023-11-28T11:08:16.972074+0000","last_became_peered":"2023-11-28T11:08:16.972074+0000","last_unstale":"2023-11-28T11:08:17.362780+0000","last_undegraded":"2023-11-28T11:08:17.362780+0000","last_fullsized":"2023-11-28T11:08:17.362780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368494+0000","last_change":"2023-11-28T11:08:17.369522+0000","last_active":"2023-11-28T11:08:18.368494+0000","last_peered":"2023-11-28T11:08:18.368494+0000","last_clean":"2023-11-28T11:08:18.368494+0000","last_became_active":"2023-11-28T11:08:17.368628+0000","last_became_peered":"2023-11-28T11:08:17.368628+0000","last_unstale":"2023-11-28T11:08:18.368494+0000","last_undegraded":"2023-11-28T11:08:18.368494+0000","last_fullsized":"2023-11-28T11:08:18.368494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357776+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:08:17.357776+0000","last_peered":"2023-11-28T11:08:17.357776+0000","last_clean":"2023-11-28T11:08:17.357776+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:08:17.357776+0000","last_undegraded":"2023-11-28T11:08:17.357776+0000","last_fullsized":"2023-11-28T11:08:17.357776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364014+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:08:17.364014+0000","last_peered":"2023-11-28T11:08:17.364014+0000","last_clean":"2023-11-28T11:08:17.364014+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:08:17.364014+0000","last_undegraded":"2023-11-28T11:08:17.364014+0000","last_fullsized":"2023-11-28T11:08:17.364014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.355796+0000","last_change":"2023-11-28T11:08:16.968819+0000","last_active":"2023-11-28T11:08:17.355796+0000","last_peered":"2023-11-28T11:08:17.355796+0000","last_clean":"2023-11-28T11:08:17.355796+0000","last_became_active":"2023-11-28T11:08:16.968536+0000","last_became_peered":"2023-11-28T11:08:16.968536+0000","last_unstale":"2023-11-28T11:08:17.355796+0000","last_undegraded":"2023-11-28T11:08:17.355796+0000","last_fullsized":"2023-11-28T11:08:17.355796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073714+0000","last_change":"2023-11-28T11:08:17.381098+0000","last_active":"2023-11-28T11:08:19.073714+0000","last_peered":"2023-11-28T11:08:19.073714+0000","last_clean":"2023-11-28T11:08:19.073714+0000","last_became_active":"2023-11-28T11:08:17.380823+0000","last_became_peered":"2023-11-28T11:08:17.380823+0000","last_unstale":"2023-11-28T11:08:19.073714+0000","last_undegraded":"2023-11-28T11:08:19.073714+0000","last_fullsized":"2023-11-28T11:08:19.073714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:56.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364256+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:08:17.364256+0000","last_peered":"2023-11-28T11:08:17.364256+0000","last_clean":"2023-11-28T11:08:17.364256+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:08:17.364256+0000","last_undegraded":"2023-11-28T11:08:17.364256+0000","last_fullsized":"2023-11-28T11:08:17.364256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.360038+0000","last_change":"2023-11-28T11:08:16.978195+0000","last_active":"2023-11-28T11:08:17.360038+0000","last_peered":"2023-11-28T11:08:17.360038+0000","last_clean":"2023-11-28T11:08:17.360038+0000","last_became_active":"2023-11-28T11:08:16.977906+0000","last_became_peered":"2023-11-28T11:08:16.977906+0000","last_unstale":"2023-11-28T11:08:17.360038+0000","last_undegraded":"2023-11-28T11:08:17.360038+0000","last_fullsized":"2023-11-28T11:08:17.360038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673777+0000","last_change":"2023-11-28T11:08:17.369849+0000","last_active":"2023-11-28T11:08:18.673777+0000","last_peered":"2023-11-28T11:08:18.673777+0000","last_clean":"2023-11-28T11:08:18.673777+0000","last_became_active":"2023-11-28T11:08:17.369614+0000","last_became_peered":"2023-11-28T11:08:17.369614+0000","last_unstale":"2023-11-28T11:08:18.673777+0000","last_undegraded":"2023-11-28T11:08:18.673777+0000","last_fullsized":"2023-11-28T11:08:18.673777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.368123+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:08:17.368123+0000","last_peered":"2023-11-28T11:08:17.368123+0000","last_clean":"2023-11-28T11:08:17.368123+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:08:17.368123+0000","last_undegraded":"2023-11-28T11:08:17.368123+0000","last_fullsized":"2023-11-28T11:08:17.368123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365891+0000","last_change":"2023-11-28T11:08:16.975131+0000","last_active":"2023-11-28T11:08:17.365891+0000","last_peered":"2023-11-28T11:08:17.365891+0000","last_clean":"2023-11-28T11:08:17.365891+0000","last_became_active":"2023-11-28T11:08:16.974846+0000","last_became_peered":"2023-11-28T11:08:16.974846+0000","last_unstale":"2023-11-28T11:08:17.365891+0000","last_undegraded":"2023-11-28T11:08:17.365891+0000","last_fullsized":"2023-11-28T11:08:17.365891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673513+0000","last_change":"2023-11-28T11:08:17.366819+0000","last_active":"2023-11-28T11:08:18.673513+0000","last_peered":"2023-11-28T11:08:18.673513+0000","last_clean":"2023-11-28T11:08:18.673513+0000","last_became_active":"2023-11-28T11:08:17.366576+0000","last_became_peered":"2023-11-28T11:08:17.366576+0000","last_unstale":"2023-11-28T11:08:18.673513+0000","last_undegraded":"2023-11-28T11:08:18.673513+0000","last_fullsized":"2023-11-28T11:08:18.673513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.379003+0000","last_change":"2023-11-28T11:08:16.973768+0000","last_active":"2023-11-28T11:08:17.379003+0000","last_peered":"2023-11-28T11:08:17.379003+0000","last_clean":"2023-11-28T11:08:17.379003+0000","last_became_active":"2023-11-28T11:08:16.973505+0000","last_became_peered":"2023-11-28T11:08:16.973505+0000","last_unstale":"2023-11-28T11:08:17.379003+0000","last_undegraded":"2023-11-28T11:08:17.379003+0000","last_fullsized":"2023-11-28T11:08:17.379003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.367956+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:08:17.367956+0000","last_peered":"2023-11-28T11:08:17.367956+0000","last_clean":"2023-11-28T11:08:17.367956+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:08:17.367956+0000","last_undegraded":"2023-11-28T11:08:17.367956+0000","last_fullsized":"2023-11-28T11:08:17.367956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365501+0000","last_change":"2023-11-28T11:08:17.375741+0000","last_active":"2023-11-28T11:08:18.365501+0000","last_peered":"2023-11-28T11:08:18.365501+0000","last_clean":"2023-11-28T11:08:18.365501+0000","last_became_active":"2023-11-28T11:08:17.375349+0000","last_became_peered":"2023-11-28T11:08:17.375349+0000","last_unstale":"2023-11-28T11:08:18.365501+0000","last_undegraded":"2023-11-28T11:08:18.365501+0000","last_fullsized":"2023-11-28T11:08:18.365501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.372744+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:08:17.372744+0000","last_peered":"2023-11-28T11:08:17.372744+0000","last_clean":"2023-11-28T11:08:17.372744+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:08:17.372744+0000","last_undegraded":"2023-11-28T11:08:17.372744+0000","last_fullsized":"2023-11-28T11:08:17.372744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.363328+0000","last_change":"2023-11-28T11:08:16.974692+0000","last_active":"2023-11-28T11:08:17.363328+0000","last_peered":"2023-11-28T11:08:17.363328+0000","last_clean":"2023-11-28T11:08:17.363328+0000","last_became_active":"2023-11-28T11:08:16.974255+0000","last_became_peered":"2023-11-28T11:08:16.974255+0000","last_unstale":"2023-11-28T11:08:17.363328+0000","last_undegraded":"2023-11-28T11:08:17.363328+0000","last_fullsized":"2023-11-28T11:08:17.363328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368842+0000","last_change":"2023-11-28T11:08:17.373312+0000","last_active":"2023-11-28T11:08:18.368842+0000","last_peered":"2023-11-28T11:08:18.368842+0000","last_clean":"2023-11-28T11:08:18.368842+0000","last_became_active":"2023-11-28T11:08:17.373033+0000","last_became_peered":"2023-11-28T11:08:17.373033+0000","last_unstale":"2023-11-28T11:08:18.368842+0000","last_undegraded":"2023-11-28T11:08:18.368842+0000","last_fullsized":"2023-11-28T11:08:18.368842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:25.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377530+0000","last_change":"2023-11-28T11:08:16.976409+0000","last_active":"2023-11-28T11:08:17.377530+0000","last_peered":"2023-11-28T11:08:17.377530+0000","last_clean":"2023-11-28T11:08:17.377530+0000","last_became_active":"2023-11-28T11:08:16.976127+0000","last_became_peered":"2023-11-28T11:08:16.976127+0000","last_unstale":"2023-11-28T11:08:17.377530+0000","last_undegraded":"2023-11-28T11:08:17.377530+0000","last_fullsized":"2023-11-28T11:08:17.377530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:11.421587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072815+0000","last_change":"2023-11-28T11:08:17.374219+0000","last_active":"2023-11-28T11:08:19.072815+0000","last_peered":"2023-11-28T11:08:19.072815+0000","last_clean":"2023-11-28T11:08:19.072815+0000","last_became_active":"2023-11-28T11:08:17.374010+0000","last_became_peered":"2023-11-28T11:08:17.374010+0000","last_unstale":"2023-11-28T11:08:19.072815+0000","last_undegraded":"2023-11-28T11:08:19.072815+0000","last_fullsized":"2023-11-28T11:08:19.072815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:50.027648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.375928+0000","last_change":"2023-11-28T11:08:16.978678+0000","last_active":"2023-11-28T11:08:17.375928+0000","last_peered":"2023-11-28T11:08:17.375928+0000","last_clean":"2023-11-28T11:08:17.375928+0000","last_became_active":"2023-11-28T11:08:16.978317+0000","last_became_peered":"2023-11-28T11:08:16.978317+0000","last_unstale":"2023-11-28T11:08:17.375928+0000","last_undegraded":"2023-11-28T11:08:17.375928+0000","last_fullsized":"2023-11-28T11:08:17.375928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368851+0000","last_change":"2023-11-28T11:08:17.373647+0000","last_active":"2023-11-28T11:08:18.368851+0000","last_peered":"2023-11-28T11:08:18.368851+0000","last_clean":"2023-11-28T11:08:18.368851+0000","last_became_active":"2023-11-28T11:08:17.373288+0000","last_became_peered":"2023-11-28T11:08:17.373288+0000","last_unstale":"2023-11-28T11:08:18.368851+0000","last_undegraded":"2023-11-28T11:08:18.368851+0000","last_fullsized":"2023-11-28T11:08:18.368851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:59.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.376913+0000","last_change":"2023-11-28T11:08:16.976048+0000","last_active":"2023-11-28T11:08:17.376913+0000","last_peered":"2023-11-28T11:08:17.376913+0000","last_clean":"2023-11-28T11:08:17.376913+0000","last_became_active":"2023-11-28T11:08:16.975600+0000","last_became_peered":"2023-11-28T11:08:16.975600+0000","last_unstale":"2023-11-28T11:08:17.376913+0000","last_undegraded":"2023-11-28T11:08:17.376913+0000","last_fullsized":"2023-11-28T11:08:17.376913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368940+0000","last_change":"2023-11-28T11:08:17.372086+0000","last_active":"2023-11-28T11:08:18.368940+0000","last_peered":"2023-11-28T11:08:18.368940+0000","last_clean":"2023-11-28T11:08:18.368940+0000","last_became_active":"2023-11-28T11:08:17.371833+0000","last_became_peered":"2023-11-28T11:08:17.371833+0000","last_unstale":"2023-11-28T11:08:18.368940+0000","last_undegraded":"2023-11-28T11:08:18.368940+0000","last_fullsized":"2023-11-28T11:08:18.368940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:46.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.380725+0000","last_change":"2023-11-28T11:08:16.964126+0000","last_active":"2023-11-28T11:08:17.380725+0000","last_peered":"2023-11-28T11:08:17.380725+0000","last_clean":"2023-11-28T11:08:17.380725+0000","last_became_active":"2023-11-28T11:08:16.963838+0000","last_became_peered":"2023-11-28T11:08:16.963838+0000","last_unstale":"2023-11-28T11:08:17.380725+0000","last_undegraded":"2023-11-28T11:08:17.380725+0000","last_fullsized":"2023-11-28T11:08:17.380725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365611+0000","last_change":"2023-11-28T11:08:17.361919+0000","last_active":"2023-11-28T11:08:18.365611+0000","last_peered":"2023-11-28T11:08:18.365611+0000","last_clean":"2023-11-28T11:08:18.365611+0000","last_became_active":"2023-11-28T11:08:17.361654+0000","last_became_peered":"2023-11-28T11:08:17.361654+0000","last_unstale":"2023-11-28T11:08:18.365611+0000","last_undegraded":"2023-11-28T11:08:18.365611+0000","last_fullsized":"2023-11-28T11:08:18.365611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:46.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.372346+0000","last_change":"2023-11-28T11:08:16.974232+0000","last_active":"2023-11-28T11:08:17.372346+0000","last_peered":"2023-11-28T11:08:17.372346+0000","last_clean":"2023-11-28T11:08:17.372346+0000","last_became_active":"2023-11-28T11:08:16.972983+0000","last_became_peered":"2023-11-28T11:08:16.972983+0000","last_unstale":"2023-11-28T11:08:17.372346+0000","last_undegraded":"2023-11-28T11:08:17.372346+0000","last_fullsized":"2023-11-28T11:08:17.372346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368650+0000","last_change":"2023-11-28T11:08:17.386923+0000","last_active":"2023-11-28T11:08:18.368650+0000","last_peered":"2023-11-28T11:08:18.368650+0000","last_clean":"2023-11-28T11:08:18.368650+0000","last_became_active":"2023-11-28T11:08:17.386690+0000","last_became_peered":"2023-11-28T11:08:17.386690+0000","last_unstale":"2023-11-28T11:08:18.368650+0000","last_undegraded":"2023-11-28T11:08:18.368650+0000","last_fullsized":"2023-11-28T11:08:18.368650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:35.096193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074332+0000","last_change":"2023-11-28T11:08:17.385964+0000","last_active":"2023-11-28T11:08:19.074332+0000","last_peered":"2023-11-28T11:08:19.074332+0000","last_clean":"2023-11-28T11:08:19.074332+0000","last_became_active":"2023-11-28T11:08:17.385673+0000","last_became_peered":"2023-11-28T11:08:17.385673+0000","last_unstale":"2023-11-28T11:08:19.074332+0000","last_undegraded":"2023-11-28T11:08:19.074332+0000","last_fullsized":"2023-11-28T11:08:19.074332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:37.983570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365540+0000","last_change":"2023-11-28T11:08:17.372667+0000","last_active":"2023-11-28T11:08:18.365540+0000","last_peered":"2023-11-28T11:08:18.365540+0000","last_clean":"2023-11-28T11:08:18.365540+0000","last_became_active":"2023-11-28T11:08:17.372363+0000","last_became_peered":"2023-11-28T11:08:17.372363+0000","last_unstale":"2023-11-28T11:08:18.365540+0000","last_undegraded":"2023-11-28T11:08:18.365540+0000","last_fullsized":"2023-11-28T11:08:18.365540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:25.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368769+0000","last_change":"2023-11-28T11:08:17.386764+0000","last_active":"2023-11-28T11:08:18.368769+0000","last_peered":"2023-11-28T11:08:18.368769+0000","last_clean":"2023-11-28T11:08:18.368769+0000","last_became_active":"2023-11-28T11:08:17.386525+0000","last_became_peered":"2023-11-28T11:08:17.386525+0000","last_unstale":"2023-11-28T11:08:18.368769+0000","last_undegraded":"2023-11-28T11:08:18.368769+0000","last_fullsized":"2023-11-28T11:08:18.368769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.366116+0000","last_change":"2023-11-28T11:08:17.378211+0000","last_active":"2023-11-28T11:08:18.366116+0000","last_peered":"2023-11-28T11:08:18.366116+0000","last_clean":"2023-11-28T11:08:18.366116+0000","last_became_active":"2023-11-28T11:08:17.376746+0000","last_became_peered":"2023-11-28T11:08:17.376746+0000","last_unstale":"2023-11-28T11:08:18.366116+0000","last_undegraded":"2023-11-28T11:08:18.366116+0000","last_fullsized":"2023-11-28T11:08:18.366116+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368303+0000","last_change":"2023-11-28T11:08:17.375172+0000","last_active":"2023-11-28T11:08:18.368303+0000","last_peered":"2023-11-28T11:08:18.368303+0000","last_clean":"2023-11-28T11:08:18.368303+0000","last_became_active":"2023-11-28T11:08:17.374911+0000","last_became_peered":"2023-11-28T11:08:17.374911+0000","last_unstale":"2023-11-28T11:08:18.368303+0000","last_undegraded":"2023-11-28T11:08:18.368303+0000","last_fullsized":"2023-11-28T11:08:18.368303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9108,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362732,"statfs":{"total":96636764160,"available":96627437568,"internally_reserved":0,"allocated":864256,"data_stored":681731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7988,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363852,"statfs":{"total":96636764160,"available":96628584448,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7988,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363852,"statfs":{"total":96636764160,"available":96628584448,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":680533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7924,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363916,"statfs":{"total":96636764160,"available":96628649984,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8056,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363784,"statfs":{"total":96636764160,"available":96628514816,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":91203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7996,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363844,"statfs":{"total":96636764160,"available":96628576256,"internally_reserved":0,"allocated":249856,"data_stored":90235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:08:22.570 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T11:08:22.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.724+0000 7f9569c75700 1 Processor -- start 2023-11-28T11:08:22.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.724+0000 7f9569c75700 1 -- start start 2023-11-28T11:08:22.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.724+0000 7f9569c75700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9564071040 0x7f9564071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.724+0000 7f9569c75700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f956406dee0 0x7f956406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.724+0000 7f9569c75700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f956406cdc0 0x7f956406d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.724+0000 7f9569c75700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9564071950 con 0x7f956406dee0 2023-11-28T11:08:22.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.724+0000 7f9569c75700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956415d060 con 0x7f9564071040 2023-11-28T11:08:22.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.724+0000 7f9569c75700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956415d340 con 0x7f956406cdc0 2023-11-28T11:08:22.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.724+0000 7f9562ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f956406dee0 0x7f956406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.724+0000 7f9562ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f956406dee0 0x7f956406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60130/0 (socket says 172.21.15.156:60130) 2023-11-28T11:08:22.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.725+0000 7f9562ffd700 1 -- 172.21.15.156:0/3228221008 learned_addr learned my addr 172.21.15.156:0/3228221008 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:22.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.725+0000 7f95637fe700 1 --2- 172.21.15.156:0/3228221008 >> v2:172.21.15.186:3300/0 conn(0x7f9564071040 0x7f9564071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.725+0000 7f9563fff700 1 --2- 172.21.15.156:0/3228221008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f956406cdc0 0x7f956406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.725+0000 7f9562ffd700 1 -- 172.21.15.156:0/3228221008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f956406cdc0 msgr2=0x7f956406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:22.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.725+0000 7f9562ffd700 1 --2- 172.21.15.156:0/3228221008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f956406cdc0 0x7f956406d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.725+0000 7f9562ffd700 1 -- 172.21.15.156:0/3228221008 >> v2:172.21.15.186:3300/0 conn(0x7f9564071040 msgr2=0x7f9564071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:22.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.725+0000 7f9562ffd700 1 --2- 172.21.15.156:0/3228221008 >> v2:172.21.15.186:3300/0 conn(0x7f9564071040 0x7f9564071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.725+0000 7f9562ffd700 1 -- 172.21.15.156:0/3228221008 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956415d620 con 0x7f956406dee0 2023-11-28T11:08:22.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.725+0000 7f9562ffd700 1 --2- 172.21.15.156:0/3228221008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f956406dee0 0x7f956406e330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f95640d0f50 tx=0x7f9554009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936bf019d9131847 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:22.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.725+0000 7f95627fc700 1 -- 172.21.15.156:0/3228221008 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554011830 con 0x7f956406dee0 2023-11-28T11:08:22.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.725+0000 7f95627fc700 1 -- 172.21.15.156:0/3228221008 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9554011990 con 0x7f956406dee0 2023-11-28T11:08:22.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.725+0000 7f95627fc700 1 -- 172.21.15.156:0/3228221008 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554011d20 con 0x7f956406dee0 2023-11-28T11:08:22.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.726+0000 7f9569c75700 1 -- 172.21.15.156:0/3228221008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f956406dee0 msgr2=0x7f956406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:22.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.726+0000 7f9569c75700 1 --2- 172.21.15.156:0/3228221008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f956406dee0 0x7f956406e330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f95640d0f50 tx=0x7f9554009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.726+0000 7f9569c75700 1 -- 172.21.15.156:0/3228221008 shutdown_connections 2023-11-28T11:08:22.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.726+0000 7f9569c75700 1 --2- 172.21.15.156:0/3228221008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f956406cdc0 0x7f956406d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.726+0000 7f9569c75700 1 --2- 172.21.15.156:0/3228221008 >> v2:172.21.15.186:3300/0 conn(0x7f9564071040 0x7f9564071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.726+0000 7f9569c75700 1 --2- 172.21.15.156:0/3228221008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f956406dee0 0x7f956406e330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.726+0000 7f9569c75700 1 -- 172.21.15.156:0/3228221008 >> 172.21.15.156:0/3228221008 conn(0x7f95640d0000 msgr2=0x7f95640d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:22.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.726+0000 7f9569c75700 1 -- 172.21.15.156:0/3228221008 shutdown_connections 2023-11-28T11:08:22.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.726+0000 7f9569c75700 1 -- 172.21.15.156:0/3228221008 wait complete. 2023-11-28T11:08:22.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.727+0000 7f9569c75700 1 Processor -- start 2023-11-28T11:08:22.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.728+0000 7f9569c75700 1 -- start start 2023-11-28T11:08:22.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.728+0000 7f9569c75700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f956406cdc0 0x7f9564147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.728+0000 7f9569c75700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f956406dee0 0x7f9564147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.728+0000 7f9569c75700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9564071040 0x7f95640b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.728+0000 7f9569c75700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956415a420 con 0x7f9564071040 2023-11-28T11:08:22.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.728+0000 7f9569c75700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956415a9c0 con 0x7f956406cdc0 2023-11-28T11:08:22.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.728+0000 7f9569c75700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95640c6e80 con 0x7f956406dee0 2023-11-28T11:08:22.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f9563fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9564071040 0x7f95640b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f9563fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9564071040 0x7f95640b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60138/0 (socket says 172.21.15.156:60138) 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f9563fff700 1 -- 172.21.15.156:0/3339243433 learned_addr learned my addr 172.21.15.156:0/3339243433 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f95637fe700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f956406cdc0 0x7f9564147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f9562ffd700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f956406dee0 0x7f9564147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f9563fff700 1 -- 172.21.15.156:0/3339243433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f956406dee0 msgr2=0x7f9564147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f9563fff700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f956406dee0 0x7f9564147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f9563fff700 1 -- 172.21.15.156:0/3339243433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f956406cdc0 msgr2=0x7f9564147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f9563fff700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f956406cdc0 0x7f9564147030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f9563fff700 1 -- 172.21.15.156:0/3339243433 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9554009550 con 0x7f9564071040 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f9563fff700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9564071040 0x7f95640b3e80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f955c00eed0 tx=0x7f955c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f955bfff700 1 -- 172.21.15.156:0/3339243433 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c010040 con 0x7f9564071040 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f955bfff700 1 -- 172.21.15.156:0/3339243433 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c00b9f0 con 0x7f9564071040 2023-11-28T11:08:22.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f9569c75700 1 -- 172.21.15.156:0/3339243433 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956406e870 con 0x7f9564071040 2023-11-28T11:08:22.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f955bfff700 1 -- 172.21.15.156:0/3339243433 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c00bda0 con 0x7f9564071040 2023-11-28T11:08:22.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.729+0000 7f9569c75700 1 -- 172.21.15.156:0/3339243433 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956415d900 con 0x7f9564071040 2023-11-28T11:08:22.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.730+0000 7f9569c75700 1 -- 172.21.15.156:0/3339243433 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956415ccd0 con 0x7f9564071040 2023-11-28T11:08:22.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.731+0000 7f955bfff700 1 -- 172.21.15.156:0/3339243433 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f955c026420 con 0x7f9564071040 2023-11-28T11:08:22.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.731+0000 7f955bfff700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f95440682f0 0x7f954406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:22.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.731+0000 7f955bfff700 1 -- 172.21.15.156:0/3339243433 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f955c0868f0 con 0x7f9564071040 2023-11-28T11:08:22.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.731+0000 7f95637fe700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f95440682f0 0x7f954406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:22.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.732+0000 7f95637fe700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f95440682f0 0x7f954406a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f954c009b40 tx=0x7f954c009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:22.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.735+0000 7f955bfff700 1 -- 172.21.15.156:0/3339243433 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f955c053080 con 0x7f9564071040 2023-11-28T11:08:22.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.997+0000 7f9569c75700 1 -- 172.21.15.156:0/3339243433 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f95640720a0 con 0x7f9564071040 2023-11-28T11:08:22.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.997+0000 7f955bfff700 1 -- 172.21.15.156:0/3339243433 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1011 (secure 0 0 0) 0x7f955c0566a0 con 0x7f9564071040 2023-11-28T11:08:22.998 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:22.998 INFO:teuthology.orchestra.run.smithi156.stdout:{"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1701169685,"num_in_osds":8,"osd_in_since":1701169677,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592698,"bytes_used":67805184,"bytes_avail":773026308096,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4287}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-11-28T11:07:54.981680+0000","services":{}},"progress_events":{}} 2023-11-28T11:08:23.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.999+0000 7f9569c75700 1 -- 172.21.15.156:0/3339243433 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f95440682f0 msgr2=0x7f954406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:23.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.999+0000 7f9569c75700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f95440682f0 0x7f954406a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f954c009b40 tx=0x7f954c009210 comp rx=0 tx=0).stop 2023-11-28T11:08:23.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.999+0000 7f9569c75700 1 -- 172.21.15.156:0/3339243433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9564071040 msgr2=0x7f95640b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:23.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.999+0000 7f9569c75700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9564071040 0x7f95640b3e80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f955c00eed0 tx=0x7f955c00ef00 comp rx=0 tx=0).stop 2023-11-28T11:08:23.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.999+0000 7f9569c75700 1 -- 172.21.15.156:0/3339243433 shutdown_connections 2023-11-28T11:08:23.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.999+0000 7f9569c75700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9564071040 0x7f95640b3e80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.999+0000 7f9569c75700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f956406cdc0 0x7f9564147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.999+0000 7f9569c75700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f95440682f0 0x7f954406a7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.999+0000 7f9569c75700 1 --2- 172.21.15.156:0/3339243433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f956406dee0 0x7f9564147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.999+0000 7f9569c75700 1 -- 172.21.15.156:0/3339243433 >> 172.21.15.156:0/3339243433 conn(0x7f95640d0000 msgr2=0x7f956406bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:23.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.999+0000 7f9569c75700 1 -- 172.21.15.156:0/3339243433 shutdown_connections 2023-11-28T11:08:23.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:22.999+0000 7f9569c75700 1 -- 172.21.15.156:0/3339243433 wait complete. 2023-11-28T11:08:23.012 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T11:08:23.012 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:08:23.076 INFO:tasks.ceph.mds.b.smithi186.stderr:2023-11-28T11:08:23.075+0000 7f1e806d8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:08:23.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.131+0000 7fca531ec700 1 Processor -- start 2023-11-28T11:08:23.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.131+0000 7fca531ec700 1 -- start start 2023-11-28T11:08:23.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.131+0000 7fca531ec700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c071040 0x7fca4c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:23.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.131+0000 7fca531ec700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fca4c160540 0x7fca4c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:23.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.131+0000 7fca531ec700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4c071950 con 0x7fca4c06dee0 2023-11-28T11:08:23.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.131+0000 7fca531ec700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4c15d060 con 0x7fca4c160540 2023-11-28T11:08:23.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.131+0000 7fca531ec700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c15d340 con 0x7fca4c071040 2023-11-28T11:08:23.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4bfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fca4c06dee0 0x7fca4c06e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46788/0 (socket says 172.21.15.156:46788) 2023-11-28T11:08:23.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4bfff700 1 -- 172.21.15.156:0/3845343576 learned_addr learned my addr 172.21.15.156:0/3845343576 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:23.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323508410 0 0) 0x7fca4c071950 con 0x7fca4c06dee0 2023-11-28T11:08:23.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca3c0035b0 con 0x7fca4c06dee0 2023-11-28T11:08:23.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca51789700 1 --2- 172.21.15.156:0/3845343576 >> v2:172.21.15.186:3300/0 conn(0x7fca4c160540 0x7fca4c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:23.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca50f88700 1 --2- 172.21.15.156:0/3845343576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c071040 0x7fca4c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:23.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4274571688 0 0) 0x7fca3c0035b0 con 0x7fca4c06dee0 2023-11-28T11:08:23.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c071040 msgr2=0x7fca4c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:23.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 --2- 172.21.15.156:0/3845343576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c071040 0x7fca4c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 >> v2:172.21.15.186:3300/0 conn(0x7fca4c160540 msgr2=0x7fca4c1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:23.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 --2- 172.21.15.156:0/3845343576 >> v2:172.21.15.186:3300/0 conn(0x7fca4c160540 0x7fca4c1609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c15d620 con 0x7fca4c06dee0 2023-11-28T11:08:23.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fca38001670 con 0x7fca4c06dee0 2023-11-28T11:08:23.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 >> v1:172.21.15.156:6789/0 conn(0x7fca4c06dee0 legacy=0x7fca4c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:23.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 --2- 172.21.15.156:0/3845343576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca3c003a00 0x7fca3c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:23.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 --2- 172.21.15.156:0/3845343576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c071040 0x7fca3c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:23.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 --2- 172.21.15.156:0/3845343576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca3c004d90 0x7fca3c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:23.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c15d060 con 0x7fca3c004d90 2023-11-28T11:08:23.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c15d340 con 0x7fca3c003a00 2023-11-28T11:08:23.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c00ba90 con 0x7fca4c071040 2023-11-28T11:08:23.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca51789700 1 --2- 172.21.15.156:0/3845343576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca3c004d90 0x7fca3c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:23.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca50f88700 1 --2- 172.21.15.156:0/3845343576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca3c003a00 0x7fca3c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:23.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.132+0000 7fca4bfff700 1 --2- 172.21.15.156:0/3845343576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c071040 0x7fca3c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:23.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca51789700 1 -- 172.21.15.156:0/3845343576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c071040 msgr2=0x7fca3c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:23.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca51789700 1 --2- 172.21.15.156:0/3845343576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c071040 0x7fca3c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca51789700 1 -- 172.21.15.156:0/3845343576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca3c003a00 msgr2=0x7fca3c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:23.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca51789700 1 --2- 172.21.15.156:0/3845343576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca3c003a00 0x7fca3c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca51789700 1 -- 172.21.15.156:0/3845343576 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0035b0 con 0x7fca3c004d90 2023-11-28T11:08:23.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca51789700 1 --2- 172.21.15.156:0/3845343576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca3c004d90 0x7fca3c005200 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fca44009fc0 tx=0x7fca3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b36380f1c60dca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:23.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4400ec30 con 0x7fca3c004d90 2023-11-28T11:08:23.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4400f040 con 0x7fca3c004d90 2023-11-28T11:08:23.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca4b7fe700 1 -- 172.21.15.156:0/3845343576 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44014480 con 0x7fca3c004d90 2023-11-28T11:08:23.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca531ec700 1 -- 172.21.15.156:0/3845343576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca3c004d90 msgr2=0x7fca3c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:23.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca531ec700 1 --2- 172.21.15.156:0/3845343576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca3c004d90 0x7fca3c005200 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fca44009fc0 tx=0x7fca3c005880 comp rx=0 tx=0).stop 2023-11-28T11:08:23.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca51789700 1 -- 172.21.15.156:0/3845343576 reap_dead start 2023-11-28T11:08:23.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca531ec700 1 -- 172.21.15.156:0/3845343576 shutdown_connections 2023-11-28T11:08:23.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca531ec700 1 -- 172.21.15.156:0/3845343576 >> 172.21.15.156:0/3845343576 conn(0x7fca4c0d0000 msgr2=0x7fca4c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:23.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca531ec700 1 -- 172.21.15.156:0/3845343576 shutdown_connections 2023-11-28T11:08:23.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.133+0000 7fca531ec700 1 -- 172.21.15.156:0/3845343576 wait complete. 2023-11-28T11:08:23.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.135+0000 7fca531ec700 1 Processor -- start 2023-11-28T11:08:23.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.135+0000 7fca531ec700 1 -- start start 2023-11-28T11:08:23.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.135+0000 7fca531ec700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca3c004d90 0x7fca4c0ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:23.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.135+0000 7fca531ec700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c157da0 0x7fca4c158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:23.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca531ec700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca4c0aece0 0x7fca4c0af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:23.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca531ec700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c1f6670 con 0x7fca4c0aece0 2023-11-28T11:08:23.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca531ec700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c152d70 con 0x7fca3c004d90 2023-11-28T11:08:23.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca531ec700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c158750 con 0x7fca4c157da0 2023-11-28T11:08:23.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca51789700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca4c0aece0 0x7fca4c0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:23.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca51789700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca4c0aece0 0x7fca4c0af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60158/0 (socket says 172.21.15.156:60158) 2023-11-28T11:08:23.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca51789700 1 -- 172.21.15.156:0/3016879496 learned_addr learned my addr 172.21.15.156:0/3016879496 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:23.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca50f88700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca3c004d90 0x7fca4c0ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:23.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca4bfff700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c157da0 0x7fca4c158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:23.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca51789700 1 -- 172.21.15.156:0/3016879496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c157da0 msgr2=0x7fca4c158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:23.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca51789700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c157da0 0x7fca4c158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca51789700 1 -- 172.21.15.156:0/3016879496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca3c004d90 msgr2=0x7fca4c0ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:23.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca51789700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca3c004d90 0x7fca4c0ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca51789700 1 -- 172.21.15.156:0/3016879496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c15ccd0 con 0x7fca4c0aece0 2023-11-28T11:08:23.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca50f88700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca3c004d90 0x7fca4c0ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:23.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca4bfff700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c157da0 0x7fca4c158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:23.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca51789700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca4c0aece0 0x7fca4c0af150 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca44009f90 tx=0x7fca44009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:23.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca497fa700 1 -- 172.21.15.156:0/3016879496 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44007000 con 0x7fca4c0aece0 2023-11-28T11:08:23.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca497fa700 1 -- 172.21.15.156:0/3016879496 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca44015b30 con 0x7fca4c0aece0 2023-11-28T11:08:23.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca531ec700 1 -- 172.21.15.156:0/3016879496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4c15d900 con 0x7fca4c0aece0 2023-11-28T11:08:23.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca497fa700 1 -- 172.21.15.156:0/3016879496 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4401d420 con 0x7fca4c0aece0 2023-11-28T11:08:23.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.136+0000 7fca531ec700 1 -- 172.21.15.156:0/3016879496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c06a270 con 0x7fca4c0aece0 2023-11-28T11:08:23.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.137+0000 7fca531ec700 1 -- 172.21.15.156:0/3016879496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4c0af690 con 0x7fca4c0aece0 2023-11-28T11:08:23.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.138+0000 7fca497fa700 1 -- 172.21.15.156:0/3016879496 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fca4402d030 con 0x7fca4c0aece0 2023-11-28T11:08:23.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.138+0000 7fca497fa700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fca300682f0 0x7fca3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:23.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.138+0000 7fca50f88700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fca300682f0 0x7fca3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:23.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.138+0000 7fca497fa700 1 -- 172.21.15.156:0/3016879496 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fca440873e0 con 0x7fca4c0aece0 2023-11-28T11:08:23.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.138+0000 7fca50f88700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fca300682f0 0x7fca3006a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca40009ae0 tx=0x7fca40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:23.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.142+0000 7fca497fa700 1 -- 172.21.15.156:0/3016879496 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fca44053c40 con 0x7fca4c0aece0 2023-11-28T11:08:23.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.319+0000 7fca531ec700 1 -- 172.21.15.156:0/3016879496 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca4c0b1120 con 0x7fca300682f0 2023-11-28T11:08:23.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.327+0000 7fca497fa700 1 -- 172.21.15.156:0/3016879496 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286716 (secure 0 0 0) 0x7fca4c0b1120 con 0x7fca300682f0 2023-11-28T11:08:23.331 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:23.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.330+0000 7fca531ec700 1 -- 172.21.15.156:0/3016879496 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fca300682f0 msgr2=0x7fca3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:23.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.330+0000 7fca531ec700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fca300682f0 0x7fca3006a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca40009ae0 tx=0x7fca40008040 comp rx=0 tx=0).stop 2023-11-28T11:08:23.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.330+0000 7fca531ec700 1 -- 172.21.15.156:0/3016879496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca4c0aece0 msgr2=0x7fca4c0af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:23.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.330+0000 7fca531ec700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca4c0aece0 0x7fca4c0af150 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca44009f90 tx=0x7fca44009750 comp rx=0 tx=0).stop 2023-11-28T11:08:23.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.330+0000 7fca531ec700 1 -- 172.21.15.156:0/3016879496 shutdown_connections 2023-11-28T11:08:23.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.330+0000 7fca531ec700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca4c0aece0 0x7fca4c0af150 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.330+0000 7fca531ec700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca3c004d90 0x7fca4c0ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.330+0000 7fca531ec700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fca300682f0 0x7fca3006a7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.330+0000 7fca531ec700 1 --2- 172.21.15.156:0/3016879496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca4c157da0 0x7fca4c158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:23.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.330+0000 7fca531ec700 1 -- 172.21.15.156:0/3016879496 >> 172.21.15.156:0/3016879496 conn(0x7fca4c0d0000 msgr2=0x7fca4c0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:23.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.330+0000 7fca531ec700 1 -- 172.21.15.156:0/3016879496 shutdown_connections 2023-11-28T11:08:23.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:23.330+0000 7fca531ec700 1 -- 172.21.15.156:0/3016879496 wait complete. 2023-11-28T11:08:23.335 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:08:23.343 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-28T11:08:22.526031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":198,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66216,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":754908504,"statfs":{"total":773094113280,"available":773026308096,"internally_reserved":0,"allocated":3186688,"data_stored":1902188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001301"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072799+0000","last_change":"2023-11-28T11:08:17.367751+0000","last_active":"2023-11-28T11:08:19.072799+0000","last_peered":"2023-11-28T11:08:19.072799+0000","last_clean":"2023-11-28T11:08:19.072799+0000","last_became_active":"2023-11-28T11:08:17.367481+0000","last_became_peered":"2023-11-28T11:08:17.367481+0000","last_unstale":"2023-11-28T11:08:19.072799+0000","last_undegraded":"2023-11-28T11:08:19.072799+0000","last_fullsized":"2023-11-28T11:08:19.072799+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:09.760690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073854+0000","last_change":"2023-11-28T11:08:17.369749+0000","last_active":"2023-11-28T11:08:19.073854+0000","last_peered":"2023-11-28T11:08:19.073854+0000","last_clean":"2023-11-28T11:08:19.073854+0000","last_became_active":"2023-11-28T11:08:17.369479+0000","last_became_peered":"2023-11-28T11:08:17.369479+0000","last_unstale":"2023-11-28T11:08:19.073854+0000","last_undegraded":"2023-11-28T11:08:19.073854+0000","last_fullsized":"2023-11-28T11:08:19.073854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369159+0000","last_change":"2023-11-28T11:08:17.370726+0000","last_active":"2023-11-28T11:08:18.369159+0000","last_peered":"2023-11-28T11:08:18.369159+0000","last_clean":"2023-11-28T11:08:18.369159+0000","last_became_active":"2023-11-28T11:08:17.367062+0000","last_became_peered":"2023-11-28T11:08:17.367062+0000","last_unstale":"2023-11-28T11:08:18.369159+0000","last_undegraded":"2023-11-28T11:08:18.369159+0000","last_fullsized":"2023-11-28T11:08:18.369159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:27.219209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073872+0000","last_change":"2023-11-28T11:08:17.383989+0000","last_active":"2023-11-28T11:08:19.073872+0000","last_peered":"2023-11-28T11:08:19.073872+0000","last_clean":"2023-11-28T11:08:19.073872+0000","last_became_active":"2023-11-28T11:08:17.381618+0000","last_became_peered":"2023-11-28T11:08:17.381618+0000","last_unstale":"2023-11-28T11:08:19.073872+0000","last_undegraded":"2023-11-28T11:08:19.073872+0000","last_fullsized":"2023-11-28T11:08:19.073872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:09:00.918897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673454+0000","last_change":"2023-11-28T11:08:17.377110+0000","last_active":"2023-11-28T11:08:18.673454+0000","last_peered":"2023-11-28T11:08:18.673454+0000","last_clean":"2023-11-28T11:08:18.673454+0000","last_became_active":"2023-11-28T11:08:17.373583+0000","last_became_peered":"2023-11-28T11:08:17.373583+0000","last_unstale":"2023-11-28T11:08:18.673454+0000","last_undegraded":"2023-11-28T11:08:18.673454+0000","last_fullsized":"2023-11-28T11:08:18.673454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:24:59.277184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368780+0000","last_change":"2023-11-28T11:08:17.370700+0000","last_active":"2023-11-28T11:08:18.368780+0000","last_peered":"2023-11-28T11:08:18.368780+0000","last_clean":"2023-11-28T11:08:18.368780+0000","last_became_active":"2023-11-28T11:08:17.367222+0000","last_became_peered":"2023-11-28T11:08:17.367222+0000","last_unstale":"2023-11-28T11:08:18.368780+0000","last_undegraded":"2023-11-28T11:08:18.368780+0000","last_fullsized":"2023-11-28T11:08:18.368780+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:22.140101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073497+0000","last_change":"2023-11-28T11:08:17.374848+0000","last_active":"2023-11-28T11:08:19.073497+0000","last_peered":"2023-11-28T11:08:19.073497+0000","last_clean":"2023-11-28T11:08:19.073497+0000","last_became_active":"2023-11-28T11:08:17.374536+0000","last_became_peered":"2023-11-28T11:08:17.374536+0000","last_unstale":"2023-11-28T11:08:19.073497+0000","last_undegraded":"2023-11-28T11:08:19.073497+0000","last_fullsized":"2023-11-28T11:08:19.073497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074252+0000","last_change":"2023-11-28T11:08:17.366207+0000","last_active":"2023-11-28T11:08:19.074252+0000","last_peered":"2023-11-28T11:08:19.074252+0000","last_clean":"2023-11-28T11:08:19.074252+0000","last_became_active":"2023-11-28T11:08:17.365918+0000","last_became_peered":"2023-11-28T11:08:17.365918+0000","last_unstale":"2023-11-28T11:08:19.074252+0000","last_undegraded":"2023-11-28T11:08:19.074252+0000","last_fullsized":"2023-11-28T11:08:19.074252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673623+0000","last_change":"2023-11-28T11:08:17.376435+0000","last_active":"2023-11-28T11:08:18.673623+0000","last_peered":"2023-11-28T11:08:18.673623+0000","last_clean":"2023-11-28T11:08:18.673623+0000","last_became_active":"2023-11-28T11:08:17.373543+0000","last_became_peered":"2023-11-28T11:08:17.373543+0000","last_unstale":"2023-11-28T11:08:18.673623+0000","last_undegraded":"2023-11-28T11:08:18.673623+0000","last_fullsized":"2023-11-28T11:08:18.673623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074030+0000","last_change":"2023-11-28T11:08:17.367284+0000","last_active":"2023-11-28T11:08:19.074030+0000","last_peered":"2023-11-28T11:08:19.074030+0000","last_clean":"2023-11-28T11:08:19.074030+0000","last_became_active":"2023-11-28T11:08:17.366992+0000","last_became_peered":"2023-11-28T11:08:17.366992+0000","last_unstale":"2023-11-28T11:08:19.074030+0000","last_undegraded":"2023-11-28T11:08:19.074030+0000","last_fullsized":"2023-11-28T11:08:19.074030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369258+0000","last_change":"2023-11-28T11:08:17.376061+0000","last_active":"2023-11-28T11:08:18.369258+0000","last_peered":"2023-11-28T11:08:18.369258+0000","last_clean":"2023-11-28T11:08:18.369258+0000","last_became_active":"2023-11-28T11:08:17.375789+0000","last_became_peered":"2023-11-28T11:08:17.375789+0000","last_unstale":"2023-11-28T11:08:18.369258+0000","last_undegraded":"2023-11-28T11:08:18.369258+0000","last_fullsized":"2023-11-28T11:08:18.369258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:10.040305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072686+0000","last_change":"2023-11-28T11:08:17.372872+0000","last_active":"2023-11-28T11:08:19.072686+0000","last_peered":"2023-11-28T11:08:19.072686+0000","last_clean":"2023-11-28T11:08:19.072686+0000","last_became_active":"2023-11-28T11:08:17.372624+0000","last_became_peered":"2023-11-28T11:08:17.372624+0000","last_unstale":"2023-11-28T11:08:19.072686+0000","last_undegraded":"2023-11-28T11:08:19.072686+0000","last_fullsized":"2023-11-28T11:08:19.072686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:34.711368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072664+0000","last_change":"2023-11-28T11:08:17.374949+0000","last_active":"2023-11-28T11:08:19.072664+0000","last_peered":"2023-11-28T11:08:19.072664+0000","last_clean":"2023-11-28T11:08:19.072664+0000","last_became_active":"2023-11-28T11:08:17.374690+0000","last_became_peered":"2023-11-28T11:08:17.374690+0000","last_unstale":"2023-11-28T11:08:19.072664+0000","last_undegraded":"2023-11-28T11:08:19.072664+0000","last_fullsized":"2023-11-28T11:08:19.072664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:52.301973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073186+0000","last_change":"2023-11-28T11:08:17.361341+0000","last_active":"2023-11-28T11:08:19.073186+0000","last_peered":"2023-11-28T11:08:19.073186+0000","last_clean":"2023-11-28T11:08:19.073186+0000","last_became_active":"2023-11-28T11:08:17.361053+0000","last_became_peered":"2023-11-28T11:08:17.361053+0000","last_unstale":"2023-11-28T11:08:19.073186+0000","last_undegraded":"2023-11-28T11:08:19.073186+0000","last_fullsized":"2023-11-28T11:08:19.073186+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073689+0000","last_change":"2023-11-28T11:08:17.377341+0000","last_active":"2023-11-28T11:08:19.073689+0000","last_peered":"2023-11-28T11:08:19.073689+0000","last_clean":"2023-11-28T11:08:19.073689+0000","last_became_active":"2023-11-28T11:08:17.376840+0000","last_became_peered":"2023-11-28T11:08:17.376840+0000","last_unstale":"2023-11-28T11:08:19.073689+0000","last_undegraded":"2023-11-28T11:08:19.073689+0000","last_fullsized":"2023-11-28T11:08:19.073689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368722+0000","last_change":"2023-11-28T11:08:17.370526+0000","last_active":"2023-11-28T11:08:18.368722+0000","last_peered":"2023-11-28T11:08:18.368722+0000","last_clean":"2023-11-28T11:08:18.368722+0000","last_became_active":"2023-11-28T11:08:17.367599+0000","last_became_peered":"2023-11-28T11:08:17.367599+0000","last_unstale":"2023-11-28T11:08:18.368722+0000","last_undegraded":"2023-11-28T11:08:18.368722+0000","last_fullsized":"2023-11-28T11:08:18.368722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:21.226857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673664+0000","last_change":"2023-11-28T11:08:17.377283+0000","last_active":"2023-11-28T11:08:18.673664+0000","last_peered":"2023-11-28T11:08:18.673664+0000","last_clean":"2023-11-28T11:08:18.673664+0000","last_became_active":"2023-11-28T11:08:17.373226+0000","last_became_peered":"2023-11-28T11:08:17.373226+0000","last_unstale":"2023-11-28T11:08:18.673664+0000","last_undegraded":"2023-11-28T11:08:18.673664+0000","last_fullsized":"2023-11-28T11:08:18.673664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369217+0000","last_change":"2023-11-28T11:08:17.373771+0000","last_active":"2023-11-28T11:08:18.369217+0000","last_peered":"2023-11-28T11:08:18.369217+0000","last_clean":"2023-11-28T11:08:18.369217+0000","last_became_active":"2023-11-28T11:08:17.373510+0000","last_became_peered":"2023-11-28T11:08:17.373510+0000","last_unstale":"2023-11-28T11:08:18.369217+0000","last_undegraded":"2023-11-28T11:08:18.369217+0000","last_fullsized":"2023-11-28T11:08:18.369217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:14.696335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072793+0000","last_change":"2023-11-28T11:08:17.381345+0000","last_active":"2023-11-28T11:08:19.072793+0000","last_peered":"2023-11-28T11:08:19.072793+0000","last_clean":"2023-11-28T11:08:19.072793+0000","last_became_active":"2023-11-28T11:08:17.381044+0000","last_became_peered":"2023-11-28T11:08:17.381044+0000","last_unstale":"2023-11-28T11:08:19.072793+0000","last_undegraded":"2023-11-28T11:08:19.072793+0000","last_fullsized":"2023-11-28T11:08:19.072793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:09.877040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369092+0000","last_change":"2023-11-28T11:08:17.387391+0000","last_active":"2023-11-28T11:08:18.369092+0000","last_peered":"2023-11-28T11:08:18.369092+0000","last_clean":"2023-11-28T11:08:18.369092+0000","last_became_active":"2023-11-28T11:08:17.387127+0000","last_became_peered":"2023-11-28T11:08:17.387127+0000","last_unstale":"2023-11-28T11:08:18.369092+0000","last_undegraded":"2023-11-28T11:08:18.369092+0000","last_fullsized":"2023-11-28T11:08:18.369092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074429+0000","last_change":"2023-11-28T11:08:17.380537+0000","last_active":"2023-11-28T11:08:19.074429+0000","last_peered":"2023-11-28T11:08:19.074429+0000","last_clean":"2023-11-28T11:08:19.074429+0000","last_became_active":"2023-11-28T11:08:17.380267+0000","last_became_peered":"2023-11-28T11:08:17.380267+0000","last_unstale":"2023-11-28T11:08:19.074429+0000","last_undegraded":"2023-11-28T11:08:19.074429+0000","last_fullsized":"2023-11-28T11:08:19.074429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673493+0000","last_change":"2023-11-28T11:08:17.371837+0000","last_active":"2023-11-28T11:08:18.673493+0000","last_peered":"2023-11-28T11:08:18.673493+0000","last_clean":"2023-11-28T11:08:18.673493+0000","last_became_active":"2023-11-28T11:08:17.371543+0000","last_became_peered":"2023-11-28T11:08:17.371543+0000","last_unstale":"2023-11-28T11:08:18.673493+0000","last_undegraded":"2023-11-28T11:08:18.673493+0000","last_fullsized":"2023-11-28T11:08:18.673493+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365642+0000","last_change":"2023-11-28T11:08:17.370684+0000","last_active":"2023-11-28T11:08:18.365642+0000","last_peered":"2023-11-28T11:08:18.365642+0000","last_clean":"2023-11-28T11:08:18.365642+0000","last_became_active":"2023-11-28T11:08:17.367787+0000","last_became_peered":"2023-11-28T11:08:17.367787+0000","last_unstale":"2023-11-28T11:08:18.365642+0000","last_undegraded":"2023-11-28T11:08:18.365642+0000","last_fullsized":"2023-11-28T11:08:18.365642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:59.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073418+0000","last_change":"2023-11-28T11:08:17.373809+0000","last_active":"2023-11-28T11:08:19.073418+0000","last_peered":"2023-11-28T11:08:19.073418+0000","last_clean":"2023-11-28T11:08:19.073418+0000","last_became_active":"2023-11-28T11:08:17.373529+0000","last_became_peered":"2023-11-28T11:08:17.373529+0000","last_unstale":"2023-11-28T11:08:19.073418+0000","last_undegraded":"2023-11-28T11:08:19.073418+0000","last_fullsized":"2023-11-28T11:08:19.073418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368409+0000","last_change":"2023-11-28T11:08:17.382413+0000","last_active":"2023-11-28T11:08:18.368409+0000","last_peered":"2023-11-28T11:08:18.368409+0000","last_clean":"2023-11-28T11:08:18.368409+0000","last_became_active":"2023-11-28T11:08:17.382132+0000","last_became_peered":"2023-11-28T11:08:17.382132+0000","last_unstale":"2023-11-28T11:08:18.368409+0000","last_undegraded":"2023-11-28T11:08:18.368409+0000","last_fullsized":"2023-11-28T11:08:18.368409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368518+0000","last_change":"2023-11-28T11:08:17.368106+0000","last_active":"2023-11-28T11:08:18.368518+0000","last_peered":"2023-11-28T11:08:18.368518+0000","last_clean":"2023-11-28T11:08:18.368518+0000","last_became_active":"2023-11-28T11:08:17.367562+0000","last_became_peered":"2023-11-28T11:08:17.367562+0000","last_unstale":"2023-11-28T11:08:18.368518+0000","last_undegraded":"2023-11-28T11:08:18.368518+0000","last_fullsized":"2023-11-28T11:08:18.368518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073955+0000","last_change":"2023-11-28T11:08:17.375521+0000","last_active":"2023-11-28T11:08:19.073955+0000","last_peered":"2023-11-28T11:08:19.073955+0000","last_clean":"2023-11-28T11:08:19.073955+0000","last_became_active":"2023-11-28T11:08:17.375256+0000","last_became_peered":"2023-11-28T11:08:17.375256+0000","last_unstale":"2023-11-28T11:08:19.073955+0000","last_undegraded":"2023-11-28T11:08:19.073955+0000","last_fullsized":"2023-11-28T11:08:19.073955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.382174+0000","last_change":"2023-11-28T11:08:16.975491+0000","last_active":"2023-11-28T11:08:17.382174+0000","last_peered":"2023-11-28T11:08:17.382174+0000","last_clean":"2023-11-28T11:08:17.382174+0000","last_became_active":"2023-11-28T11:08:16.975076+0000","last_became_peered":"2023-11-28T11:08:16.975076+0000","last_unstale":"2023-11-28T11:08:17.382174+0000","last_undegraded":"2023-11-28T11:08:17.382174+0000","last_fullsized":"2023-11-28T11:08:17.382174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365845+0000","last_change":"2023-11-28T11:08:17.371715+0000","last_active":"2023-11-28T11:08:18.365845+0000","last_peered":"2023-11-28T11:08:18.365845+0000","last_clean":"2023-11-28T11:08:18.365845+0000","last_became_active":"2023-11-28T11:08:17.368939+0000","last_became_peered":"2023-11-28T11:08:17.368939+0000","last_unstale":"2023-11-28T11:08:18.365845+0000","last_undegraded":"2023-11-28T11:08:18.365845+0000","last_fullsized":"2023-11-28T11:08:18.365845+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362665+0000","last_change":"2023-11-28T11:08:16.975573+0000","last_active":"2023-11-28T11:08:17.362665+0000","last_peered":"2023-11-28T11:08:17.362665+0000","last_clean":"2023-11-28T11:08:17.362665+0000","last_became_active":"2023-11-28T11:08:16.975216+0000","last_became_peered":"2023-11-28T11:08:16.975216+0000","last_unstale":"2023-11-28T11:08:17.362665+0000","last_undegraded":"2023-11-28T11:08:17.362665+0000","last_fullsized":"2023-11-28T11:08:17.362665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673721+0000","last_change":"2023-11-28T11:08:17.378243+0000","last_active":"2023-11-28T11:08:18.673721+0000","last_peered":"2023-11-28T11:08:18.673721+0000","last_clean":"2023-11-28T11:08:18.673721+0000","last_became_active":"2023-11-28T11:08:17.377866+0000","last_became_peered":"2023-11-28T11:08:17.377866+0000","last_unstale":"2023-11-28T11:08:18.673721+0000","last_undegraded":"2023-11-28T11:08:18.673721+0000","last_fullsized":"2023-11-28T11:08:18.673721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.356027+0000","last_change":"2023-11-28T11:08:16.972779+0000","last_active":"2023-11-28T11:08:17.356027+0000","last_peered":"2023-11-28T11:08:17.356027+0000","last_clean":"2023-11-28T11:08:17.356027+0000","last_became_active":"2023-11-28T11:08:16.972510+0000","last_became_peered":"2023-11-28T11:08:16.972510+0000","last_unstale":"2023-11-28T11:08:17.356027+0000","last_undegraded":"2023-11-28T11:08:17.356027+0000","last_fullsized":"2023-11-28T11:08:17.356027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074026+0000","last_change":"2023-11-28T11:08:17.380638+0000","last_active":"2023-11-28T11:08:19.074026+0000","last_peered":"2023-11-28T11:08:19.074026+0000","last_clean":"2023-11-28T11:08:19.074026+0000","last_became_active":"2023-11-28T11:08:17.380333+0000","last_became_peered":"2023-11-28T11:08:17.380333+0000","last_unstale":"2023-11-28T11:08:19.074026+0000","last_undegraded":"2023-11-28T11:08:19.074026+0000","last_fullsized":"2023-11-28T11:08:19.074026+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:34.711368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.361717+0000","last_change":"2023-11-28T11:08:16.976366+0000","last_active":"2023-11-28T11:08:17.361717+0000","last_peered":"2023-11-28T11:08:17.361717+0000","last_clean":"2023-11-28T11:08:17.361717+0000","last_became_active":"2023-11-28T11:08:16.976072+0000","last_became_peered":"2023-11-28T11:08:16.976072+0000","last_unstale":"2023-11-28T11:08:17.361717+0000","last_undegraded":"2023-11-28T11:08:17.361717+0000","last_fullsized":"2023-11-28T11:08:17.361717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365642+0000","last_change":"2023-11-28T11:08:17.364878+0000","last_active":"2023-11-28T11:08:18.365642+0000","last_peered":"2023-11-28T11:08:18.365642+0000","last_clean":"2023-11-28T11:08:18.365642+0000","last_became_active":"2023-11-28T11:08:17.364628+0000","last_became_peered":"2023-11-28T11:08:17.364628+0000","last_unstale":"2023-11-28T11:08:18.365642+0000","last_undegraded":"2023-11-28T11:08:18.365642+0000","last_fullsized":"2023-11-28T11:08:18.365642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:12.679494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.358873+0000","last_change":"2023-11-28T11:08:16.975388+0000","last_active":"2023-11-28T11:08:17.358873+0000","last_peered":"2023-11-28T11:08:17.358873+0000","last_clean":"2023-11-28T11:08:17.358873+0000","last_became_active":"2023-11-28T11:08:16.974977+0000","last_became_peered":"2023-11-28T11:08:16.974977+0000","last_unstale":"2023-11-28T11:08:17.358873+0000","last_undegraded":"2023-11-28T11:08:17.358873+0000","last_fullsized":"2023-11-28T11:08:17.358873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365734+0000","last_change":"2023-11-28T11:08:17.376435+0000","last_active":"2023-11-28T11:08:18.365734+0000","last_peered":"2023-11-28T11:08:18.365734+0000","last_clean":"2023-11-28T11:08:18.365734+0000","last_became_active":"2023-11-28T11:08:17.376170+0000","last_became_peered":"2023-11-28T11:08:17.376170+0000","last_unstale":"2023-11-28T11:08:18.365734+0000","last_undegraded":"2023-11-28T11:08:18.365734+0000","last_fullsized":"2023-11-28T11:08:18.365734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:24:59.277184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.381284+0000","last_change":"2023-11-28T11:08:16.975413+0000","last_active":"2023-11-28T11:08:17.381284+0000","last_peered":"2023-11-28T11:08:17.381284+0000","last_clean":"2023-11-28T11:08:17.381284+0000","last_became_active":"2023-11-28T11:08:16.975058+0000","last_became_peered":"2023-11-28T11:08:16.975058+0000","last_unstale":"2023-11-28T11:08:17.381284+0000","last_undegraded":"2023-11-28T11:08:17.381284+0000","last_fullsized":"2023-11-28T11:08:17.381284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368111+0000","last_change":"2023-11-28T11:08:17.381210+0000","last_active":"2023-11-28T11:08:18.368111+0000","last_peered":"2023-11-28T11:08:18.368111+0000","last_clean":"2023-11-28T11:08:18.368111+0000","last_became_active":"2023-11-28T11:08:17.380941+0000","last_became_peered":"2023-11-28T11:08:17.380941+0000","last_unstale":"2023-11-28T11:08:18.368111+0000","last_undegraded":"2023-11-28T11:08:18.368111+0000","last_fullsized":"2023-11-28T11:08:18.368111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:35.096193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.387746+0000","last_change":"2023-11-28T11:08:16.973937+0000","last_active":"2023-11-28T11:08:17.387746+0000","last_peered":"2023-11-28T11:08:17.387746+0000","last_clean":"2023-11-28T11:08:17.387746+0000","last_became_active":"2023-11-28T11:08:16.973594+0000","last_became_peered":"2023-11-28T11:08:16.973594+0000","last_unstale":"2023-11-28T11:08:17.387746+0000","last_undegraded":"2023-11-28T11:08:17.387746+0000","last_fullsized":"2023-11-28T11:08:17.387746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:13.313856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369025+0000","last_change":"2023-11-28T11:08:17.380894+0000","last_active":"2023-11-28T11:08:18.369025+0000","last_peered":"2023-11-28T11:08:18.369025+0000","last_clean":"2023-11-28T11:08:18.369025+0000","last_became_active":"2023-11-28T11:08:17.380600+0000","last_became_peered":"2023-11-28T11:08:17.380600+0000","last_unstale":"2023-11-28T11:08:18.369025+0000","last_undegraded":"2023-11-28T11:08:18.369025+0000","last_fullsized":"2023-11-28T11:08:18.369025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377908+0000","last_change":"2023-11-28T11:08:16.977393+0000","last_active":"2023-11-28T11:08:17.377908+0000","last_peered":"2023-11-28T11:08:17.377908+0000","last_clean":"2023-11-28T11:08:17.377908+0000","last_became_active":"2023-11-28T11:08:16.976585+0000","last_became_peered":"2023-11-28T11:08:16.976585+0000","last_unstale":"2023-11-28T11:08:17.377908+0000","last_undegraded":"2023-11-28T11:08:17.377908+0000","last_fullsized":"2023-11-28T11:08:17.377908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:36.981686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365857+0000","last_change":"2023-11-28T11:08:17.376235+0000","last_active":"2023-11-28T11:08:18.365857+0000","last_peered":"2023-11-28T11:08:18.365857+0000","last_clean":"2023-11-28T11:08:18.365857+0000","last_became_active":"2023-11-28T11:08:17.375675+0000","last_became_peered":"2023-11-28T11:08:17.375675+0000","last_unstale":"2023-11-28T11:08:18.365857+0000","last_undegraded":"2023-11-28T11:08:18.365857+0000","last_fullsized":"2023-11-28T11:08:18.365857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.359416+0000","last_change":"2023-11-28T11:08:16.974456+0000","last_active":"2023-11-28T11:08:17.359416+0000","last_peered":"2023-11-28T11:08:17.359416+0000","last_clean":"2023-11-28T11:08:17.359416+0000","last_became_active":"2023-11-28T11:08:16.974158+0000","last_became_peered":"2023-11-28T11:08:16.974158+0000","last_unstale":"2023-11-28T11:08:17.359416+0000","last_undegraded":"2023-11-28T11:08:17.359416+0000","last_fullsized":"2023-11-28T11:08:17.359416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074033+0000","last_change":"2023-11-28T11:08:17.375410+0000","last_active":"2023-11-28T11:08:19.074033+0000","last_peered":"2023-11-28T11:08:19.074033+0000","last_clean":"2023-11-28T11:08:19.074033+0000","last_became_active":"2023-11-28T11:08:17.374828+0000","last_became_peered":"2023-11-28T11:08:17.374828+0000","last_unstale":"2023-11-28T11:08:19.074033+0000","last_undegraded":"2023-11-28T11:08:19.074033+0000","last_fullsized":"2023-11-28T11:08:19.074033+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365862+0000","last_change":"2023-11-28T11:08:16.978277+0000","last_active":"2023-11-28T11:08:17.365862+0000","last_peered":"2023-11-28T11:08:17.365862+0000","last_clean":"2023-11-28T11:08:17.365862+0000","last_became_active":"2023-11-28T11:08:16.977994+0000","last_became_peered":"2023-11-28T11:08:16.977994+0000","last_unstale":"2023-11-28T11:08:17.365862+0000","last_undegraded":"2023-11-28T11:08:17.365862+0000","last_fullsized":"2023-11-28T11:08:17.365862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:55.975785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673702+0000","last_change":"2023-11-28T11:08:17.369372+0000","last_active":"2023-11-28T11:08:18.673702+0000","last_peered":"2023-11-28T11:08:18.673702+0000","last_clean":"2023-11-28T11:08:18.673702+0000","last_became_active":"2023-11-28T11:08:17.369115+0000","last_became_peered":"2023-11-28T11:08:17.369115+0000","last_unstale":"2023-11-28T11:08:18.673702+0000","last_undegraded":"2023-11-28T11:08:18.673702+0000","last_fullsized":"2023-11-28T11:08:18.673702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.386102+0000","last_change":"2023-11-28T11:08:16.974290+0000","last_active":"2023-11-28T11:08:17.386102+0000","last_peered":"2023-11-28T11:08:17.386102+0000","last_clean":"2023-11-28T11:08:17.386102+0000","last_became_active":"2023-11-28T11:08:16.973952+0000","last_became_peered":"2023-11-28T11:08:16.973952+0000","last_unstale":"2023-11-28T11:08:17.386102+0000","last_undegraded":"2023-11-28T11:08:17.386102+0000","last_fullsized":"2023-11-28T11:08:17.386102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365978+0000","last_change":"2023-11-28T11:08:17.376526+0000","last_active":"2023-11-28T11:08:18.365978+0000","last_peered":"2023-11-28T11:08:18.365978+0000","last_clean":"2023-11-28T11:08:18.365978+0000","last_became_active":"2023-11-28T11:08:17.376285+0000","last_became_peered":"2023-11-28T11:08:17.376285+0000","last_unstale":"2023-11-28T11:08:18.365978+0000","last_undegraded":"2023-11-28T11:08:18.365978+0000","last_fullsized":"2023-11-28T11:08:18.365978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377922+0000","last_change":"2023-11-28T11:08:16.976879+0000","last_active":"2023-11-28T11:08:17.377922+0000","last_peered":"2023-11-28T11:08:17.377922+0000","last_clean":"2023-11-28T11:08:17.377922+0000","last_became_active":"2023-11-28T11:08:16.976530+0000","last_became_peered":"2023-11-28T11:08:16.976530+0000","last_unstale":"2023-11-28T11:08:17.377922+0000","last_undegraded":"2023-11-28T11:08:17.377922+0000","last_fullsized":"2023-11-28T11:08:17.377922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:59.917012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073041+0000","last_change":"2023-11-28T11:08:17.375192+0000","last_active":"2023-11-28T11:08:19.073041+0000","last_peered":"2023-11-28T11:08:19.073041+0000","last_clean":"2023-11-28T11:08:19.073041+0000","last_became_active":"2023-11-28T11:08:17.374903+0000","last_became_peered":"2023-11-28T11:08:17.374903+0000","last_unstale":"2023-11-28T11:08:19.073041+0000","last_undegraded":"2023-11-28T11:08:19.073041+0000","last_fullsized":"2023-11-28T11:08:19.073041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377956+0000","last_change":"2023-11-28T11:08:16.980934+0000","last_active":"2023-11-28T11:08:17.377956+0000","last_peered":"2023-11-28T11:08:17.377956+0000","last_clean":"2023-11-28T11:08:17.377956+0000","last_became_active":"2023-11-28T11:08:16.980585+0000","last_became_peered":"2023-11-28T11:08:16.980585+0000","last_unstale":"2023-11-28T11:08:17.377956+0000","last_undegraded":"2023-11-28T11:08:17.377956+0000","last_fullsized":"2023-11-28T11:08:17.377956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:33.709483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368728+0000","last_change":"2023-11-28T11:08:17.374922+0000","last_active":"2023-11-28T11:08:18.368728+0000","last_peered":"2023-11-28T11:08:18.368728+0000","last_clean":"2023-11-28T11:08:18.368728+0000","last_became_active":"2023-11-28T11:08:17.374631+0000","last_became_peered":"2023-11-28T11:08:17.374631+0000","last_unstale":"2023-11-28T11:08:18.368728+0000","last_undegraded":"2023-11-28T11:08:18.368728+0000","last_fullsized":"2023-11-28T11:08:18.368728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:28:26.671538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.375561+0000","last_change":"2023-11-28T11:08:16.978233+0000","last_active":"2023-11-28T11:08:17.375561+0000","last_peered":"2023-11-28T11:08:17.375561+0000","last_clean":"2023-11-28T11:08:17.375561+0000","last_became_active":"2023-11-28T11:08:16.977953+0000","last_became_peered":"2023-11-28T11:08:16.977953+0000","last_unstale":"2023-11-28T11:08:17.375561+0000","last_undegraded":"2023-11-28T11:08:17.375561+0000","last_fullsized":"2023-11-28T11:08:17.375561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073837+0000","last_change":"2023-11-28T11:08:17.381282+0000","last_active":"2023-11-28T11:08:19.073837+0000","last_peered":"2023-11-28T11:08:19.073837+0000","last_clean":"2023-11-28T11:08:19.073837+0000","last_became_active":"2023-11-28T11:08:17.381013+0000","last_became_peered":"2023-11-28T11:08:17.381013+0000","last_unstale":"2023-11-28T11:08:19.073837+0000","last_undegraded":"2023-11-28T11:08:19.073837+0000","last_fullsized":"2023-11-28T11:08:19.073837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:09.760690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357896+0000","last_change":"2023-11-28T11:08:16.976378+0000","last_active":"2023-11-28T11:08:17.357896+0000","last_peered":"2023-11-28T11:08:17.357896+0000","last_clean":"2023-11-28T11:08:17.357896+0000","last_became_active":"2023-11-28T11:08:16.974426+0000","last_became_peered":"2023-11-28T11:08:16.974426+0000","last_unstale":"2023-11-28T11:08:17.357896+0000","last_undegraded":"2023-11-28T11:08:17.357896+0000","last_fullsized":"2023-11-28T11:08:17.357896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673645+0000","last_change":"2023-11-28T11:08:17.366381+0000","last_active":"2023-11-28T11:08:18.673645+0000","last_peered":"2023-11-28T11:08:18.673645+0000","last_clean":"2023-11-28T11:08:18.673645+0000","last_became_active":"2023-11-28T11:08:17.366129+0000","last_became_peered":"2023-11-28T11:08:17.366129+0000","last_unstale":"2023-11-28T11:08:18.673645+0000","last_undegraded":"2023-11-28T11:08:18.673645+0000","last_fullsized":"2023-11-28T11:08:18.673645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.380305+0000","last_change":"2023-11-28T11:08:16.973851+0000","last_active":"2023-11-28T11:08:17.380305+0000","last_peered":"2023-11-28T11:08:17.380305+0000","last_clean":"2023-11-28T11:08:17.380305+0000","last_became_active":"2023-11-28T11:08:16.973588+0000","last_became_peered":"2023-11-28T11:08:16.973588+0000","last_unstale":"2023-11-28T11:08:17.380305+0000","last_undegraded":"2023-11-28T11:08:17.380305+0000","last_fullsized":"2023-11-28T11:08:17.380305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369164+0000","last_change":"2023-11-28T11:08:17.372448+0000","last_active":"2023-11-28T11:08:18.369164+0000","last_peered":"2023-11-28T11:08:18.369164+0000","last_clean":"2023-11-28T11:08:18.369164+0000","last_became_active":"2023-11-28T11:08:17.372132+0000","last_became_peered":"2023-11-28T11:08:17.372132+0000","last_unstale":"2023-11-28T11:08:18.369164+0000","last_undegraded":"2023-11-28T11:08:18.369164+0000","last_fullsized":"2023-11-28T11:08:18.369164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:12.679494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.371186+0000","last_change":"2023-11-28T11:08:16.975018+0000","last_active":"2023-11-28T11:08:17.371186+0000","last_peered":"2023-11-28T11:08:17.371186+0000","last_clean":"2023-11-28T11:08:17.371186+0000","last_became_active":"2023-11-28T11:08:16.974709+0000","last_became_peered":"2023-11-28T11:08:16.974709+0000","last_unstale":"2023-11-28T11:08:17.371186+0000","last_undegraded":"2023-11-28T11:08:17.371186+0000","last_fullsized":"2023-11-28T11:08:17.371186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369338+0000","last_change":"2023-11-28T11:08:17.372568+0000","last_active":"2023-11-28T11:08:18.369338+0000","last_peered":"2023-11-28T11:08:18.369338+0000","last_clean":"2023-11-28T11:08:18.369338+0000","last_became_active":"2023-11-28T11:08:17.372279+0000","last_became_peered":"2023-11-28T11:08:17.372279+0000","last_unstale":"2023-11-28T11:08:18.369338+0000","last_undegraded":"2023-11-28T11:08:18.369338+0000","last_fullsized":"2023-11-28T11:08:18.369338+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:02.566272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364207+0000","last_change":"2023-11-28T11:08:16.976973+0000","last_active":"2023-11-28T11:08:17.364207+0000","last_peered":"2023-11-28T11:08:17.364207+0000","last_clean":"2023-11-28T11:08:17.364207+0000","last_became_active":"2023-11-28T11:08:16.976661+0000","last_became_peered":"2023-11-28T11:08:16.976661+0000","last_unstale":"2023-11-28T11:08:17.364207+0000","last_undegraded":"2023-11-28T11:08:17.364207+0000","last_fullsized":"2023-11-28T11:08:17.364207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:43.575239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369260+0000","last_change":"2023-11-28T11:08:17.381198+0000","last_active":"2023-11-28T11:08:18.369260+0000","last_peered":"2023-11-28T11:08:18.369260+0000","last_clean":"2023-11-28T11:08:18.369260+0000","last_became_active":"2023-11-28T11:08:17.380937+0000","last_became_peered":"2023-11-28T11:08:17.380937+0000","last_unstale":"2023-11-28T11:08:18.369260+0000","last_undegraded":"2023-11-28T11:08:18.369260+0000","last_fullsized":"2023-11-28T11:08:18.369260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.355350+0000","last_change":"2023-11-28T11:08:16.966279+0000","last_active":"2023-11-28T11:08:17.355350+0000","last_peered":"2023-11-28T11:08:17.355350+0000","last_clean":"2023-11-28T11:08:17.355350+0000","last_became_active":"2023-11-28T11:08:16.965878+0000","last_became_peered":"2023-11-28T11:08:16.965878+0000","last_unstale":"2023-11-28T11:08:17.355350+0000","last_undegraded":"2023-11-28T11:08:17.355350+0000","last_fullsized":"2023-11-28T11:08:17.355350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368520+0000","last_change":"2023-11-28T11:08:17.373270+0000","last_active":"2023-11-28T11:08:18.368520+0000","last_peered":"2023-11-28T11:08:18.368520+0000","last_clean":"2023-11-28T11:08:18.368520+0000","last_became_active":"2023-11-28T11:08:17.373017+0000","last_became_peered":"2023-11-28T11:08:17.373017+0000","last_unstale":"2023-11-28T11:08:18.368520+0000","last_undegraded":"2023-11-28T11:08:18.368520+0000","last_fullsized":"2023-11-28T11:08:18.368520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365774+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:08:17.365774+0000","last_peered":"2023-11-28T11:08:17.365774+0000","last_clean":"2023-11-28T11:08:17.365774+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:08:17.365774+0000","last_undegraded":"2023-11-28T11:08:17.365774+0000","last_fullsized":"2023-11-28T11:08:17.365774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.358583+0000","last_change":"2023-11-28T11:08:16.975874+0000","last_active":"2023-11-28T11:08:17.358583+0000","last_peered":"2023-11-28T11:08:17.358583+0000","last_clean":"2023-11-28T11:08:17.358583+0000","last_became_active":"2023-11-28T11:08:16.975225+0000","last_became_peered":"2023-11-28T11:08:16.975225+0000","last_unstale":"2023-11-28T11:08:17.358583+0000","last_undegraded":"2023-11-28T11:08:17.358583+0000","last_fullsized":"2023-11-28T11:08:17.358583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368149+0000","last_change":"2023-11-28T11:08:17.371631+0000","last_active":"2023-11-28T11:08:18.368149+0000","last_peered":"2023-11-28T11:08:18.368149+0000","last_clean":"2023-11-28T11:08:18.368149+0000","last_became_active":"2023-11-28T11:08:17.371342+0000","last_became_peered":"2023-11-28T11:08:17.371342+0000","last_unstale":"2023-11-28T11:08:18.368149+0000","last_undegraded":"2023-11-28T11:08:18.368149+0000","last_fullsized":"2023-11-28T11:08:18.368149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357509+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:08:17.357509+0000","last_peered":"2023-11-28T11:08:17.357509+0000","last_clean":"2023-11-28T11:08:17.357509+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:08:17.357509+0000","last_undegraded":"2023-11-28T11:08:17.357509+0000","last_fullsized":"2023-11-28T11:08:17.357509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362711+0000","last_change":"2023-11-28T11:08:16.974904+0000","last_active":"2023-11-28T11:08:17.362711+0000","last_peered":"2023-11-28T11:08:17.362711+0000","last_clean":"2023-11-28T11:08:17.362711+0000","last_became_active":"2023-11-28T11:08:16.974634+0000","last_became_peered":"2023-11-28T11:08:16.974634+0000","last_unstale":"2023-11-28T11:08:17.362711+0000","last_undegraded":"2023-11-28T11:08:17.362711+0000","last_fullsized":"2023-11-28T11:08:17.362711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368410+0000","last_change":"2023-11-28T11:08:17.371181+0000","last_active":"2023-11-28T11:08:18.368410+0000","last_peered":"2023-11-28T11:08:18.368410+0000","last_clean":"2023-11-28T11:08:18.368410+0000","last_became_active":"2023-11-28T11:08:17.370931+0000","last_became_peered":"2023-11-28T11:08:17.370931+0000","last_unstale":"2023-11-28T11:08:18.368410+0000","last_undegraded":"2023-11-28T11:08:18.368410+0000","last_fullsized":"2023-11-28T11:08:18.368410+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.359510+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:08:17.359510+0000","last_peered":"2023-11-28T11:08:17.359510+0000","last_clean":"2023-11-28T11:08:17.359510+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:08:17.359510+0000","last_undegraded":"2023-11-28T11:08:17.359510+0000","last_fullsized":"2023-11-28T11:08:17.359510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362780+0000","last_change":"2023-11-28T11:08:16.972359+0000","last_active":"2023-11-28T11:08:17.362780+0000","last_peered":"2023-11-28T11:08:17.362780+0000","last_clean":"2023-11-28T11:08:17.362780+0000","last_became_active":"2023-11-28T11:08:16.972074+0000","last_became_peered":"2023-11-28T11:08:16.972074+0000","last_unstale":"2023-11-28T11:08:17.362780+0000","last_undegraded":"2023-11-28T11:08:17.362780+0000","last_fullsized":"2023-11-28T11:08:17.362780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368494+0000","last_change":"2023-11-28T11:08:17.369522+0000","last_active":"2023-11-28T11:08:18.368494+0000","last_peered":"2023-11-28T11:08:18.368494+0000","last_clean":"2023-11-28T11:08:18.368494+0000","last_became_active":"2023-11-28T11:08:17.368628+0000","last_became_peered":"2023-11-28T11:08:17.368628+0000","last_unstale":"2023-11-28T11:08:18.368494+0000","last_undegraded":"2023-11-28T11:08:18.368494+0000","last_fullsized":"2023-11-28T11:08:18.368494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357776+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:08:17.357776+0000","last_peered":"2023-11-28T11:08:17.357776+0000","last_clean":"2023-11-28T11:08:17.357776+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:08:17.357776+0000","last_undegraded":"2023-11-28T11:08:17.357776+0000","last_fullsized":"2023-11-28T11:08:17.357776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364014+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:08:17.364014+0000","last_peered":"2023-11-28T11:08:17.364014+0000","last_clean":"2023-11-28T11:08:17.364014+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:08:17.364014+0000","last_undegraded":"2023-11-28T11:08:17.364014+0000","last_fullsized":"2023-11-28T11:08:17.364014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.355796+0000","last_change":"2023-11-28T11:08:16.968819+0000","last_active":"2023-11-28T11:08:17.355796+0000","last_peered":"2023-11-28T11:08:17.355796+0000","last_clean":"2023-11-28T11:08:17.355796+0000","last_became_active":"2023-11-28T11:08:16.968536+0000","last_became_peered":"2023-11-28T11:08:16.968536+0000","last_unstale":"2023-11-28T11:08:17.355796+0000","last_undegraded":"2023-11-28T11:08:17.355796+0000","last_fullsized":"2023-11-28T11:08:17.355796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073714+0000","last_change":"2023-11-28T11:08:17.381098+0000","last_active":"2023-11-28T11:08:19.073714+0000","last_peered":"2023-11-28T11:08:19.073714+0000","last_clean":"2023-11-28T11:08:19.073714+0000","last_became_active":"2023-11-28T11:08:17.380823+0000","last_became_peered":"2023-11-28T11:08:17.380823+0000","last_unstale":"2023-11-28T11:08:19.073714+0000","last_undegraded":"2023-11-28T11:08:19.073714+0000","last_fullsized":"2023-11-28T11:08:19.073714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:56.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364256+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:08:17.364256+0000","last_peered":"2023-11-28T11:08:17.364256+0000","last_clean":"2023-11-28T11:08:17.364256+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:08:17.364256+0000","last_undegraded":"2023-11-28T11:08:17.364256+0000","last_fullsized":"2023-11-28T11:08:17.364256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.360038+0000","last_change":"2023-11-28T11:08:16.978195+0000","last_active":"2023-11-28T11:08:17.360038+0000","last_peered":"2023-11-28T11:08:17.360038+0000","last_clean":"2023-11-28T11:08:17.360038+0000","last_became_active":"2023-11-28T11:08:16.977906+0000","last_became_peered":"2023-11-28T11:08:16.977906+0000","last_unstale":"2023-11-28T11:08:17.360038+0000","last_undegraded":"2023-11-28T11:08:17.360038+0000","last_fullsized":"2023-11-28T11:08:17.360038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673777+0000","last_change":"2023-11-28T11:08:17.369849+0000","last_active":"2023-11-28T11:08:18.673777+0000","last_peered":"2023-11-28T11:08:18.673777+0000","last_clean":"2023-11-28T11:08:18.673777+0000","last_became_active":"2023-11-28T11:08:17.369614+0000","last_became_peered":"2023-11-28T11:08:17.369614+0000","last_unstale":"2023-11-28T11:08:18.673777+0000","last_undegraded":"2023-11-28T11:08:18.673777+0000","last_fullsized":"2023-11-28T11:08:18.673777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.368123+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:08:17.368123+0000","last_peered":"2023-11-28T11:08:17.368123+0000","last_clean":"2023-11-28T11:08:17.368123+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:08:17.368123+0000","last_undegraded":"2023-11-28T11:08:17.368123+0000","last_fullsized":"2023-11-28T11:08:17.368123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365891+0000","last_change":"2023-11-28T11:08:16.975131+0000","last_active":"2023-11-28T11:08:17.365891+0000","last_peered":"2023-11-28T11:08:17.365891+0000","last_clean":"2023-11-28T11:08:17.365891+0000","last_became_active":"2023-11-28T11:08:16.974846+0000","last_became_peered":"2023-11-28T11:08:16.974846+0000","last_unstale":"2023-11-28T11:08:17.365891+0000","last_undegraded":"2023-11-28T11:08:17.365891+0000","last_fullsized":"2023-11-28T11:08:17.365891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673513+0000","last_change":"2023-11-28T11:08:17.366819+0000","last_active":"2023-11-28T11:08:18.673513+0000","last_peered":"2023-11-28T11:08:18.673513+0000","last_clean":"2023-11-28T11:08:18.673513+0000","last_became_active":"2023-11-28T11:08:17.366576+0000","last_became_peered":"2023-11-28T11:08:17.366576+0000","last_unstale":"2023-11-28T11:08:18.673513+0000","last_undegraded":"2023-11-28T11:08:18.673513+0000","last_fullsized":"2023-11-28T11:08:18.673513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.379003+0000","last_change":"2023-11-28T11:08:16.973768+0000","last_active":"2023-11-28T11:08:17.379003+0000","last_peered":"2023-11-28T11:08:17.379003+0000","last_clean":"2023-11-28T11:08:17.379003+0000","last_became_active":"2023-11-28T11:08:16.973505+0000","last_became_peered":"2023-11-28T11:08:16.973505+0000","last_unstale":"2023-11-28T11:08:17.379003+0000","last_undegraded":"2023-11-28T11:08:17.379003+0000","last_fullsized":"2023-11-28T11:08:17.379003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.367956+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:08:17.367956+0000","last_peered":"2023-11-28T11:08:17.367956+0000","last_clean":"2023-11-28T11:08:17.367956+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:08:17.367956+0000","last_undegraded":"2023-11-28T11:08:17.367956+0000","last_fullsized":"2023-11-28T11:08:17.367956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365501+0000","last_change":"2023-11-28T11:08:17.375741+0000","last_active":"2023-11-28T11:08:18.365501+0000","last_peered":"2023-11-28T11:08:18.365501+0000","last_clean":"2023-11-28T11:08:18.365501+0000","last_became_active":"2023-11-28T11:08:17.375349+0000","last_became_peered":"2023-11-28T11:08:17.375349+0000","last_unstale":"2023-11-28T11:08:18.365501+0000","last_undegraded":"2023-11-28T11:08:18.365501+0000","last_fullsized":"2023-11-28T11:08:18.365501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.372744+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:08:17.372744+0000","last_peered":"2023-11-28T11:08:17.372744+0000","last_clean":"2023-11-28T11:08:17.372744+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:08:17.372744+0000","last_undegraded":"2023-11-28T11:08:17.372744+0000","last_fullsized":"2023-11-28T11:08:17.372744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.363328+0000","last_change":"2023-11-28T11:08:16.974692+0000","last_active":"2023-11-28T11:08:17.363328+0000","last_peered":"2023-11-28T11:08:17.363328+0000","last_clean":"2023-11-28T11:08:17.363328+0000","last_became_active":"2023-11-28T11:08:16.974255+0000","last_became_peered":"2023-11-28T11:08:16.974255+0000","last_unstale":"2023-11-28T11:08:17.363328+0000","last_undegraded":"2023-11-28T11:08:17.363328+0000","last_fullsized":"2023-11-28T11:08:17.363328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368842+0000","last_change":"2023-11-28T11:08:17.373312+0000","last_active":"2023-11-28T11:08:18.368842+0000","last_peered":"2023-11-28T11:08:18.368842+0000","last_clean":"2023-11-28T11:08:18.368842+0000","last_became_active":"2023-11-28T11:08:17.373033+0000","last_became_peered":"2023-11-28T11:08:17.373033+0000","last_unstale":"2023-11-28T11:08:18.368842+0000","last_undegraded":"2023-11-28T11:08:18.368842+0000","last_fullsized":"2023-11-28T11:08:18.368842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:25.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377530+0000","last_change":"2023-11-28T11:08:16.976409+0000","last_active":"2023-11-28T11:08:17.377530+0000","last_peered":"2023-11-28T11:08:17.377530+0000","last_clean":"2023-11-28T11:08:17.377530+0000","last_became_active":"2023-11-28T11:08:16.976127+0000","last_became_peered":"2023-11-28T11:08:16.976127+0000","last_unstale":"2023-11-28T11:08:17.377530+0000","last_undegraded":"2023-11-28T11:08:17.377530+0000","last_fullsized":"2023-11-28T11:08:17.377530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:11.421587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072815+0000","last_change":"2023-11-28T11:08:17.374219+0000","last_active":"2023-11-28T11:08:19.072815+0000","last_peered":"2023-11-28T11:08:19.072815+0000","last_clean":"2023-11-28T11:08:19.072815+0000","last_became_active":"2023-11-28T11:08:17.374010+0000","last_became_peered":"2023-11-28T11:08:17.374010+0000","last_unstale":"2023-11-28T11:08:19.072815+0000","last_undegraded":"2023-11-28T11:08:19.072815+0000","last_fullsized":"2023-11-28T11:08:19.072815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:50.027648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.375928+0000","last_change":"2023-11-28T11:08:16.978678+0000","last_active":"2023-11-28T11:08:17.375928+0000","last_peered":"2023-11-28T11:08:17.375928+0000","last_clean":"2023-11-28T11:08:17.375928+0000","last_became_active":"2023-11-28T11:08:16.978317+0000","last_became_peered":"2023-11-28T11:08:16.978317+0000","last_unstale":"2023-11-28T11:08:17.375928+0000","last_undegraded":"2023-11-28T11:08:17.375928+0000","last_fullsized":"2023-11-28T11:08:17.375928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368851+0000","last_change":"2023-11-28T11:08:17.373647+0000","last_active":"2023-11-28T11:08:18.368851+0000","last_peered":"2023-11-28T11:08:18.368851+0000","last_clean":"2023-11-28T11:08:18.368851+0000","last_became_active":"2023-11-28T11:08:17.373288+0000","last_became_peered":"2023-11-28T11:08:17.373288+0000","last_unstale":"2023-11-28T11:08:18.368851+0000","last_undegraded":"2023-11-28T11:08:18.368851+0000","last_fullsized":"2023-11-28T11:08:18.368851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:59.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.376913+0000","last_change":"2023-11-28T11:08:16.976048+0000","last_active":"2023-11-28T11:08:17.376913+0000","last_peered":"2023-11-28T11:08:17.376913+0000","last_clean":"2023-11-28T11:08:17.376913+0000","last_became_active":"2023-11-28T11:08:16.975600+0000","last_became_peered":"2023-11-28T11:08:16.975600+0000","last_unstale":"2023-11-28T11:08:17.376913+0000","last_undegraded":"2023-11-28T11:08:17.376913+0000","last_fullsized":"2023-11-28T11:08:17.376913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368940+0000","last_change":"2023-11-28T11:08:17.372086+0000","last_active":"2023-11-28T11:08:18.368940+0000","last_peered":"2023-11-28T11:08:18.368940+0000","last_clean":"2023-11-28T11:08:18.368940+0000","last_became_active":"2023-11-28T11:08:17.371833+0000","last_became_peered":"2023-11-28T11:08:17.371833+0000","last_unstale":"2023-11-28T11:08:18.368940+0000","last_undegraded":"2023-11-28T11:08:18.368940+0000","last_fullsized":"2023-11-28T11:08:18.368940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:46.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.380725+0000","last_change":"2023-11-28T11:08:16.964126+0000","last_active":"2023-11-28T11:08:17.380725+0000","last_peered":"2023-11-28T11:08:17.380725+0000","last_clean":"2023-11-28T11:08:17.380725+0000","last_became_active":"2023-11-28T11:08:16.963838+0000","last_became_peered":"2023-11-28T11:08:16.963838+0000","last_unstale":"2023-11-28T11:08:17.380725+0000","last_undegraded":"2023-11-28T11:08:17.380725+0000","last_fullsized":"2023-11-28T11:08:17.380725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365611+0000","last_change":"2023-11-28T11:08:17.361919+0000","last_active":"2023-11-28T11:08:18.365611+0000","last_peered":"2023-11-28T11:08:18.365611+0000","last_clean":"2023-11-28T11:08:18.365611+0000","last_became_active":"2023-11-28T11:08:17.361654+0000","last_became_peered":"2023-11-28T11:08:17.361654+0000","last_unstale":"2023-11-28T11:08:18.365611+0000","last_undegraded":"2023-11-28T11:08:18.365611+0000","last_fullsized":"2023-11-28T11:08:18.365611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:46.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.372346+0000","last_change":"2023-11-28T11:08:16.974232+0000","last_active":"2023-11-28T11:08:17.372346+0000","last_peered":"2023-11-28T11:08:17.372346+0000","last_clean":"2023-11-28T11:08:17.372346+0000","last_became_active":"2023-11-28T11:08:16.972983+0000","last_became_peered":"2023-11-28T11:08:16.972983+0000","last_unstale":"2023-11-28T11:08:17.372346+0000","last_undegraded":"2023-11-28T11:08:17.372346+0000","last_fullsized":"2023-11-28T11:08:17.372346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368650+0000","last_change":"2023-11-28T11:08:17.386923+0000","last_active":"2023-11-28T11:08:18.368650+0000","last_peered":"2023-11-28T11:08:18.368650+0000","last_clean":"2023-11-28T11:08:18.368650+0000","last_became_active":"2023-11-28T11:08:17.386690+0000","last_became_peered":"2023-11-28T11:08:17.386690+0000","last_unstale":"2023-11-28T11:08:18.368650+0000","last_undegraded":"2023-11-28T11:08:18.368650+0000","last_fullsized":"2023-11-28T11:08:18.368650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:35.096193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074332+0000","last_change":"2023-11-28T11:08:17.385964+0000","last_active":"2023-11-28T11:08:19.074332+0000","last_peered":"2023-11-28T11:08:19.074332+0000","last_clean":"2023-11-28T11:08:19.074332+0000","last_became_active":"2023-11-28T11:08:17.385673+0000","last_became_peered":"2023-11-28T11:08:17.385673+0000","last_unstale":"2023-11-28T11:08:19.074332+0000","last_undegraded":"2023-11-28T11:08:19.074332+0000","last_fullsized":"2023-11-28T11:08:19.074332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:37.983570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365540+0000","last_change":"2023-11-28T11:08:17.372667+0000","last_active":"2023-11-28T11:08:18.365540+0000","last_peered":"2023-11-28T11:08:18.365540+0000","last_clean":"2023-11-28T11:08:18.365540+0000","last_became_active":"2023-11-28T11:08:17.372363+0000","last_became_peered":"2023-11-28T11:08:17.372363+0000","last_unstale":"2023-11-28T11:08:18.365540+0000","last_undegraded":"2023-11-28T11:08:18.365540+0000","last_fullsized":"2023-11-28T11:08:18.365540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:25.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368769+0000","last_change":"2023-11-28T11:08:17.386764+0000","last_active":"2023-11-28T11:08:18.368769+0000","last_peered":"2023-11-28T11:08:18.368769+0000","last_clean":"2023-11-28T11:08:18.368769+0000","last_became_active":"2023-11-28T11:08:17.386525+0000","last_became_peered":"2023-11-28T11:08:17.386525+0000","last_unstale":"2023-11-28T11:08:18.368769+0000","last_undegraded":"2023-11-28T11:08:18.368769+0000","last_fullsized":"2023-11-28T11:08:18.368769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.366116+0000","last_change":"2023-11-28T11:08:17.378211+0000","last_active":"2023-11-28T11:08:18.366116+0000","last_peered":"2023-11-28T11:08:18.366116+0000","last_clean":"2023-11-28T11:08:18.366116+0000","last_became_active":"2023-11-28T11:08:17.376746+0000","last_became_peered":"2023-11-28T11:08:17.376746+0000","last_unstale":"2023-11-28T11:08:18.366116+0000","last_undegraded":"2023-11-28T11:08:18.366116+0000","last_fullsized":"2023-11-28T11:08:18.366116+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368303+0000","last_change":"2023-11-28T11:08:17.375172+0000","last_active":"2023-11-28T11:08:18.368303+0000","last_peered":"2023-11-28T11:08:18.368303+0000","last_clean":"2023-11-28T11:08:18.368303+0000","last_became_active":"2023-11-28T11:08:17.374911+0000","last_became_peered":"2023-11-28T11:08:17.374911+0000","last_unstale":"2023-11-28T11:08:18.368303+0000","last_undegraded":"2023-11-28T11:08:18.368303+0000","last_fullsized":"2023-11-28T11:08:18.368303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9108,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362732,"statfs":{"total":96636764160,"available":96627437568,"internally_reserved":0,"allocated":864256,"data_stored":681731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7988,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363852,"statfs":{"total":96636764160,"available":96628584448,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7988,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363852,"statfs":{"total":96636764160,"available":96628584448,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":680533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7924,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363916,"statfs":{"total":96636764160,"available":96628649984,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8056,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363784,"statfs":{"total":96636764160,"available":96628514816,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":91203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7996,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363844,"statfs":{"total":96636764160,"available":96628576256,"internally_reserved":0,"allocated":249856,"data_stored":90235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:08:26.349 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:08:26.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.480+0000 7fd89abf6700 1 Processor -- start 2023-11-28T11:08:26.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.480+0000 7fd89abf6700 1 -- start start 2023-11-28T11:08:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.481+0000 7fd89abf6700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd89406ba90 0x7fd8940af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.481+0000 7fd89abf6700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8940cc5f0 con 0x7fd8940cbdf0 2023-11-28T11:08:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.481+0000 7fd89abf6700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89415d470 con 0x7fd89415cef0 2023-11-28T11:08:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.481+0000 7fd89abf6700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89415d750 con 0x7fd89406ba90 2023-11-28T11:08:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.481+0000 7fd899193700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd8940cbdf0 0x7fd8940ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46810/0 (socket says 172.21.15.156:46810) 2023-11-28T11:08:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.481+0000 7fd899193700 1 -- 172.21.15.156:0/469098824 learned_addr learned my addr 172.21.15.156:0/469098824 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.482+0000 7fd893fff700 1 --2- 172.21.15.156:0/469098824 >> v2:172.21.15.186:3300/0 conn(0x7fd89406ba90 0x7fd8940af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.482+0000 7fd8937fe700 1 -- 172.21.15.156:0/469098824 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534345053 0 0) 0x7fd8940cc5f0 con 0x7fd8940cbdf0 2023-11-28T11:08:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.482+0000 7fd8937fe700 1 -- 172.21.15.156:0/469098824 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8840035b0 con 0x7fd8940cbdf0 2023-11-28T11:08:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.482+0000 7fd893fff700 1 -- 172.21.15.156:0/469098824 >> v1:172.21.15.186:6789/0 conn(0x7fd89415cef0 legacy=0x7fd89406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.482+0000 7fd893fff700 1 -- 172.21.15.156:0/469098824 >> v1:172.21.15.156:6789/0 conn(0x7fd8940cbdf0 legacy=0x7fd8940ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.482+0000 7fd893fff700 1 -- 172.21.15.156:0/469098824 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89415da30 con 0x7fd89406ba90 2023-11-28T11:08:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.482+0000 7fd893fff700 1 --2- 172.21.15.156:0/469098824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd89406ba90 0x7fd8940af400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd880009840 tx=0x7fd880009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d210e3055d10d80e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.482+0000 7fd8937fe700 1 -- 172.21.15.156:0/469098824 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880011830 con 0x7fd89406ba90 2023-11-28T11:08:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.482+0000 7fd8937fe700 1 -- 172.21.15.156:0/469098824 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd880011990 con 0x7fd89406ba90 2023-11-28T11:08:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.482+0000 7fd8937fe700 1 -- 172.21.15.156:0/469098824 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880011d20 con 0x7fd89406ba90 2023-11-28T11:08:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.483+0000 7fd89abf6700 1 -- 172.21.15.156:0/469098824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd89406ba90 msgr2=0x7fd8940af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.483+0000 7fd89abf6700 1 --2- 172.21.15.156:0/469098824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd89406ba90 0x7fd8940af400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd880009840 tx=0x7fd880009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.483+0000 7fd89abf6700 1 -- 172.21.15.156:0/469098824 shutdown_connections 2023-11-28T11:08:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.483+0000 7fd89abf6700 1 --2- 172.21.15.156:0/469098824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd89406ba90 0x7fd8940af400 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.483+0000 7fd89abf6700 1 -- 172.21.15.156:0/469098824 >> 172.21.15.156:0/469098824 conn(0x7fd8940d0000 msgr2=0x7fd8940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.483+0000 7fd89abf6700 1 -- 172.21.15.156:0/469098824 shutdown_connections 2023-11-28T11:08:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.483+0000 7fd89abf6700 1 -- 172.21.15.156:0/469098824 wait complete. 2023-11-28T11:08:26.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.484+0000 7fd89abf6700 1 Processor -- start 2023-11-28T11:08:26.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.484+0000 7fd89abf6700 1 -- start start 2023-11-28T11:08:26.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd89abf6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd89406ba90 0x7fd8940c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:26.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd89abf6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8940cbdf0 0x7fd8940ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:26.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd89abf6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd89415cef0 0x7fd8940aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:26.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd89abf6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940c3e40 con 0x7fd89406ba90 2023-11-28T11:08:26.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd89abf6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941f02a0 con 0x7fd89415cef0 2023-11-28T11:08:26.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd89abf6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941effc0 con 0x7fd8940cbdf0 2023-11-28T11:08:26.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd898992700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd89406ba90 0x7fd8940c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:26.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd898992700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd89406ba90 0x7fd8940c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60166/0 (socket says 172.21.15.156:60166) 2023-11-28T11:08:26.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd898992700 1 -- 172.21.15.156:0/3732697339 learned_addr learned my addr 172.21.15.156:0/3732697339 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:26.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd899193700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd89415cef0 0x7fd8940aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:26.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd893fff700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8940cbdf0 0x7fd8940ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:26.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd898992700 1 -- 172.21.15.156:0/3732697339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8940cbdf0 msgr2=0x7fd8940ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:26.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd898992700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8940cbdf0 0x7fd8940ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:26.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd898992700 1 -- 172.21.15.156:0/3732697339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd89415cef0 msgr2=0x7fd8940aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:26.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd898992700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd89415cef0 0x7fd8940aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd898992700 1 -- 172.21.15.156:0/3732697339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8800094a0 con 0x7fd89406ba90 2023-11-28T11:08:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.485+0000 7fd893fff700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8940cbdf0 0x7fd8940ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.486+0000 7fd898992700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd89406ba90 0x7fd8940c6310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd88800c480 tx=0x7fd88800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.486+0000 7fd8917fa700 1 -- 172.21.15.156:0/3732697339 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8880052d0 con 0x7fd89406ba90 2023-11-28T11:08:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.486+0000 7fd8917fa700 1 -- 172.21.15.156:0/3732697339 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd888005430 con 0x7fd89406ba90 2023-11-28T11:08:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.486+0000 7fd89abf6700 1 -- 172.21.15.156:0/3732697339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89415dff0 con 0x7fd89406ba90 2023-11-28T11:08:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.486+0000 7fd8917fa700 1 -- 172.21.15.156:0/3732697339 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8880038c0 con 0x7fd89406ba90 2023-11-28T11:08:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.486+0000 7fd89abf6700 1 -- 172.21.15.156:0/3732697339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89415dd10 con 0x7fd89406ba90 2023-11-28T11:08:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.487+0000 7fd89abf6700 1 -- 172.21.15.156:0/3732697339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8940cad60 con 0x7fd89406ba90 2023-11-28T11:08:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.487+0000 7fd8917fa700 1 -- 172.21.15.156:0/3732697339 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd888019030 con 0x7fd89406ba90 2023-11-28T11:08:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.487+0000 7fd8917fa700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd8780682f0 0x7fd87806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.488+0000 7fd8917fa700 1 -- 172.21.15.156:0/3732697339 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd888084f60 con 0x7fd89406ba90 2023-11-28T11:08:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.488+0000 7fd893fff700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd8780682f0 0x7fd87806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.488+0000 7fd893fff700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd8780682f0 0x7fd87806a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd8940c4c80 tx=0x7fd880008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:26.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.492+0000 7fd8917fa700 1 -- 172.21.15.156:0/3732697339 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd8880517c0 con 0x7fd89406ba90 2023-11-28T11:08:26.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.673+0000 7fd89abf6700 1 -- 172.21.15.156:0/3732697339 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd89406c230 con 0x7fd8780682f0 2023-11-28T11:08:26.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.680+0000 7fd8917fa700 1 -- 172.21.15.156:0/3732697339 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286716 (secure 0 0 0) 0x7fd89406c230 con 0x7fd8780682f0 2023-11-28T11:08:26.685 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:26.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.683+0000 7fd89abf6700 1 -- 172.21.15.156:0/3732697339 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd8780682f0 msgr2=0x7fd87806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:26.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.683+0000 7fd89abf6700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd8780682f0 0x7fd87806a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd8940c4c80 tx=0x7fd880008030 comp rx=0 tx=0).stop 2023-11-28T11:08:26.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.683+0000 7fd89abf6700 1 -- 172.21.15.156:0/3732697339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd89406ba90 msgr2=0x7fd8940c6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:26.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.683+0000 7fd89abf6700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd89406ba90 0x7fd8940c6310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd88800c480 tx=0x7fd88800c840 comp rx=0 tx=0).stop 2023-11-28T11:08:26.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.683+0000 7fd89abf6700 1 -- 172.21.15.156:0/3732697339 shutdown_connections 2023-11-28T11:08:26.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.683+0000 7fd89abf6700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd89415cef0 0x7fd8940aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:26.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.683+0000 7fd89abf6700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd89406ba90 0x7fd8940c6310 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:26.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.683+0000 7fd89abf6700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd8780682f0 0x7fd87806a7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:26.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.683+0000 7fd89abf6700 1 --2- 172.21.15.156:0/3732697339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8940cbdf0 0x7fd8940ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:26.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.683+0000 7fd89abf6700 1 -- 172.21.15.156:0/3732697339 >> 172.21.15.156:0/3732697339 conn(0x7fd8940d0000 msgr2=0x7fd89406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:26.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.683+0000 7fd89abf6700 1 -- 172.21.15.156:0/3732697339 shutdown_connections 2023-11-28T11:08:26.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.683+0000 7fd89abf6700 1 -- 172.21.15.156:0/3732697339 wait complete. 2023-11-28T11:08:26.689 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:08:26.697 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-11-28T11:08:26.526894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":198,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66216,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":754908504,"statfs":{"total":773094113280,"available":773026308096,"internally_reserved":0,"allocated":3186688,"data_stored":1902188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.172770"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072799+0000","last_change":"2023-11-28T11:08:17.367751+0000","last_active":"2023-11-28T11:08:19.072799+0000","last_peered":"2023-11-28T11:08:19.072799+0000","last_clean":"2023-11-28T11:08:19.072799+0000","last_became_active":"2023-11-28T11:08:17.367481+0000","last_became_peered":"2023-11-28T11:08:17.367481+0000","last_unstale":"2023-11-28T11:08:19.072799+0000","last_undegraded":"2023-11-28T11:08:19.072799+0000","last_fullsized":"2023-11-28T11:08:19.072799+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:09.760690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073854+0000","last_change":"2023-11-28T11:08:17.369749+0000","last_active":"2023-11-28T11:08:19.073854+0000","last_peered":"2023-11-28T11:08:19.073854+0000","last_clean":"2023-11-28T11:08:19.073854+0000","last_became_active":"2023-11-28T11:08:17.369479+0000","last_became_peered":"2023-11-28T11:08:17.369479+0000","last_unstale":"2023-11-28T11:08:19.073854+0000","last_undegraded":"2023-11-28T11:08:19.073854+0000","last_fullsized":"2023-11-28T11:08:19.073854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369159+0000","last_change":"2023-11-28T11:08:17.370726+0000","last_active":"2023-11-28T11:08:18.369159+0000","last_peered":"2023-11-28T11:08:18.369159+0000","last_clean":"2023-11-28T11:08:18.369159+0000","last_became_active":"2023-11-28T11:08:17.367062+0000","last_became_peered":"2023-11-28T11:08:17.367062+0000","last_unstale":"2023-11-28T11:08:18.369159+0000","last_undegraded":"2023-11-28T11:08:18.369159+0000","last_fullsized":"2023-11-28T11:08:18.369159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:27.219209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073872+0000","last_change":"2023-11-28T11:08:17.383989+0000","last_active":"2023-11-28T11:08:19.073872+0000","last_peered":"2023-11-28T11:08:19.073872+0000","last_clean":"2023-11-28T11:08:19.073872+0000","last_became_active":"2023-11-28T11:08:17.381618+0000","last_became_peered":"2023-11-28T11:08:17.381618+0000","last_unstale":"2023-11-28T11:08:19.073872+0000","last_undegraded":"2023-11-28T11:08:19.073872+0000","last_fullsized":"2023-11-28T11:08:19.073872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:09:00.918897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673454+0000","last_change":"2023-11-28T11:08:17.377110+0000","last_active":"2023-11-28T11:08:18.673454+0000","last_peered":"2023-11-28T11:08:18.673454+0000","last_clean":"2023-11-28T11:08:18.673454+0000","last_became_active":"2023-11-28T11:08:17.373583+0000","last_became_peered":"2023-11-28T11:08:17.373583+0000","last_unstale":"2023-11-28T11:08:18.673454+0000","last_undegraded":"2023-11-28T11:08:18.673454+0000","last_fullsized":"2023-11-28T11:08:18.673454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:24:59.277184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368780+0000","last_change":"2023-11-28T11:08:17.370700+0000","last_active":"2023-11-28T11:08:18.368780+0000","last_peered":"2023-11-28T11:08:18.368780+0000","last_clean":"2023-11-28T11:08:18.368780+0000","last_became_active":"2023-11-28T11:08:17.367222+0000","last_became_peered":"2023-11-28T11:08:17.367222+0000","last_unstale":"2023-11-28T11:08:18.368780+0000","last_undegraded":"2023-11-28T11:08:18.368780+0000","last_fullsized":"2023-11-28T11:08:18.368780+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:22.140101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073497+0000","last_change":"2023-11-28T11:08:17.374848+0000","last_active":"2023-11-28T11:08:19.073497+0000","last_peered":"2023-11-28T11:08:19.073497+0000","last_clean":"2023-11-28T11:08:19.073497+0000","last_became_active":"2023-11-28T11:08:17.374536+0000","last_became_peered":"2023-11-28T11:08:17.374536+0000","last_unstale":"2023-11-28T11:08:19.073497+0000","last_undegraded":"2023-11-28T11:08:19.073497+0000","last_fullsized":"2023-11-28T11:08:19.073497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074252+0000","last_change":"2023-11-28T11:08:17.366207+0000","last_active":"2023-11-28T11:08:19.074252+0000","last_peered":"2023-11-28T11:08:19.074252+0000","last_clean":"2023-11-28T11:08:19.074252+0000","last_became_active":"2023-11-28T11:08:17.365918+0000","last_became_peered":"2023-11-28T11:08:17.365918+0000","last_unstale":"2023-11-28T11:08:19.074252+0000","last_undegraded":"2023-11-28T11:08:19.074252+0000","last_fullsized":"2023-11-28T11:08:19.074252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673623+0000","last_change":"2023-11-28T11:08:17.376435+0000","last_active":"2023-11-28T11:08:18.673623+0000","last_peered":"2023-11-28T11:08:18.673623+0000","last_clean":"2023-11-28T11:08:18.673623+0000","last_became_active":"2023-11-28T11:08:17.373543+0000","last_became_peered":"2023-11-28T11:08:17.373543+0000","last_unstale":"2023-11-28T11:08:18.673623+0000","last_undegraded":"2023-11-28T11:08:18.673623+0000","last_fullsized":"2023-11-28T11:08:18.673623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074030+0000","last_change":"2023-11-28T11:08:17.367284+0000","last_active":"2023-11-28T11:08:19.074030+0000","last_peered":"2023-11-28T11:08:19.074030+0000","last_clean":"2023-11-28T11:08:19.074030+0000","last_became_active":"2023-11-28T11:08:17.366992+0000","last_became_peered":"2023-11-28T11:08:17.366992+0000","last_unstale":"2023-11-28T11:08:19.074030+0000","last_undegraded":"2023-11-28T11:08:19.074030+0000","last_fullsized":"2023-11-28T11:08:19.074030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369258+0000","last_change":"2023-11-28T11:08:17.376061+0000","last_active":"2023-11-28T11:08:18.369258+0000","last_peered":"2023-11-28T11:08:18.369258+0000","last_clean":"2023-11-28T11:08:18.369258+0000","last_became_active":"2023-11-28T11:08:17.375789+0000","last_became_peered":"2023-11-28T11:08:17.375789+0000","last_unstale":"2023-11-28T11:08:18.369258+0000","last_undegraded":"2023-11-28T11:08:18.369258+0000","last_fullsized":"2023-11-28T11:08:18.369258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:10.040305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072686+0000","last_change":"2023-11-28T11:08:17.372872+0000","last_active":"2023-11-28T11:08:19.072686+0000","last_peered":"2023-11-28T11:08:19.072686+0000","last_clean":"2023-11-28T11:08:19.072686+0000","last_became_active":"2023-11-28T11:08:17.372624+0000","last_became_peered":"2023-11-28T11:08:17.372624+0000","last_unstale":"2023-11-28T11:08:19.072686+0000","last_undegraded":"2023-11-28T11:08:19.072686+0000","last_fullsized":"2023-11-28T11:08:19.072686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:34.711368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072664+0000","last_change":"2023-11-28T11:08:17.374949+0000","last_active":"2023-11-28T11:08:19.072664+0000","last_peered":"2023-11-28T11:08:19.072664+0000","last_clean":"2023-11-28T11:08:19.072664+0000","last_became_active":"2023-11-28T11:08:17.374690+0000","last_became_peered":"2023-11-28T11:08:17.374690+0000","last_unstale":"2023-11-28T11:08:19.072664+0000","last_undegraded":"2023-11-28T11:08:19.072664+0000","last_fullsized":"2023-11-28T11:08:19.072664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:52.301973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073186+0000","last_change":"2023-11-28T11:08:17.361341+0000","last_active":"2023-11-28T11:08:19.073186+0000","last_peered":"2023-11-28T11:08:19.073186+0000","last_clean":"2023-11-28T11:08:19.073186+0000","last_became_active":"2023-11-28T11:08:17.361053+0000","last_became_peered":"2023-11-28T11:08:17.361053+0000","last_unstale":"2023-11-28T11:08:19.073186+0000","last_undegraded":"2023-11-28T11:08:19.073186+0000","last_fullsized":"2023-11-28T11:08:19.073186+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073689+0000","last_change":"2023-11-28T11:08:17.377341+0000","last_active":"2023-11-28T11:08:19.073689+0000","last_peered":"2023-11-28T11:08:19.073689+0000","last_clean":"2023-11-28T11:08:19.073689+0000","last_became_active":"2023-11-28T11:08:17.376840+0000","last_became_peered":"2023-11-28T11:08:17.376840+0000","last_unstale":"2023-11-28T11:08:19.073689+0000","last_undegraded":"2023-11-28T11:08:19.073689+0000","last_fullsized":"2023-11-28T11:08:19.073689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368722+0000","last_change":"2023-11-28T11:08:17.370526+0000","last_active":"2023-11-28T11:08:18.368722+0000","last_peered":"2023-11-28T11:08:18.368722+0000","last_clean":"2023-11-28T11:08:18.368722+0000","last_became_active":"2023-11-28T11:08:17.367599+0000","last_became_peered":"2023-11-28T11:08:17.367599+0000","last_unstale":"2023-11-28T11:08:18.368722+0000","last_undegraded":"2023-11-28T11:08:18.368722+0000","last_fullsized":"2023-11-28T11:08:18.368722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:21.226857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673664+0000","last_change":"2023-11-28T11:08:17.377283+0000","last_active":"2023-11-28T11:08:18.673664+0000","last_peered":"2023-11-28T11:08:18.673664+0000","last_clean":"2023-11-28T11:08:18.673664+0000","last_became_active":"2023-11-28T11:08:17.373226+0000","last_became_peered":"2023-11-28T11:08:17.373226+0000","last_unstale":"2023-11-28T11:08:18.673664+0000","last_undegraded":"2023-11-28T11:08:18.673664+0000","last_fullsized":"2023-11-28T11:08:18.673664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369217+0000","last_change":"2023-11-28T11:08:17.373771+0000","last_active":"2023-11-28T11:08:18.369217+0000","last_peered":"2023-11-28T11:08:18.369217+0000","last_clean":"2023-11-28T11:08:18.369217+0000","last_became_active":"2023-11-28T11:08:17.373510+0000","last_became_peered":"2023-11-28T11:08:17.373510+0000","last_unstale":"2023-11-28T11:08:18.369217+0000","last_undegraded":"2023-11-28T11:08:18.369217+0000","last_fullsized":"2023-11-28T11:08:18.369217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:14.696335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072793+0000","last_change":"2023-11-28T11:08:17.381345+0000","last_active":"2023-11-28T11:08:19.072793+0000","last_peered":"2023-11-28T11:08:19.072793+0000","last_clean":"2023-11-28T11:08:19.072793+0000","last_became_active":"2023-11-28T11:08:17.381044+0000","last_became_peered":"2023-11-28T11:08:17.381044+0000","last_unstale":"2023-11-28T11:08:19.072793+0000","last_undegraded":"2023-11-28T11:08:19.072793+0000","last_fullsized":"2023-11-28T11:08:19.072793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:09.877040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369092+0000","last_change":"2023-11-28T11:08:17.387391+0000","last_active":"2023-11-28T11:08:18.369092+0000","last_peered":"2023-11-28T11:08:18.369092+0000","last_clean":"2023-11-28T11:08:18.369092+0000","last_became_active":"2023-11-28T11:08:17.387127+0000","last_became_peered":"2023-11-28T11:08:17.387127+0000","last_unstale":"2023-11-28T11:08:18.369092+0000","last_undegraded":"2023-11-28T11:08:18.369092+0000","last_fullsized":"2023-11-28T11:08:18.369092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074429+0000","last_change":"2023-11-28T11:08:17.380537+0000","last_active":"2023-11-28T11:08:19.074429+0000","last_peered":"2023-11-28T11:08:19.074429+0000","last_clean":"2023-11-28T11:08:19.074429+0000","last_became_active":"2023-11-28T11:08:17.380267+0000","last_became_peered":"2023-11-28T11:08:17.380267+0000","last_unstale":"2023-11-28T11:08:19.074429+0000","last_undegraded":"2023-11-28T11:08:19.074429+0000","last_fullsized":"2023-11-28T11:08:19.074429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673493+0000","last_change":"2023-11-28T11:08:17.371837+0000","last_active":"2023-11-28T11:08:18.673493+0000","last_peered":"2023-11-28T11:08:18.673493+0000","last_clean":"2023-11-28T11:08:18.673493+0000","last_became_active":"2023-11-28T11:08:17.371543+0000","last_became_peered":"2023-11-28T11:08:17.371543+0000","last_unstale":"2023-11-28T11:08:18.673493+0000","last_undegraded":"2023-11-28T11:08:18.673493+0000","last_fullsized":"2023-11-28T11:08:18.673493+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365642+0000","last_change":"2023-11-28T11:08:17.370684+0000","last_active":"2023-11-28T11:08:18.365642+0000","last_peered":"2023-11-28T11:08:18.365642+0000","last_clean":"2023-11-28T11:08:18.365642+0000","last_became_active":"2023-11-28T11:08:17.367787+0000","last_became_peered":"2023-11-28T11:08:17.367787+0000","last_unstale":"2023-11-28T11:08:18.365642+0000","last_undegraded":"2023-11-28T11:08:18.365642+0000","last_fullsized":"2023-11-28T11:08:18.365642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:59.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073418+0000","last_change":"2023-11-28T11:08:17.373809+0000","last_active":"2023-11-28T11:08:19.073418+0000","last_peered":"2023-11-28T11:08:19.073418+0000","last_clean":"2023-11-28T11:08:19.073418+0000","last_became_active":"2023-11-28T11:08:17.373529+0000","last_became_peered":"2023-11-28T11:08:17.373529+0000","last_unstale":"2023-11-28T11:08:19.073418+0000","last_undegraded":"2023-11-28T11:08:19.073418+0000","last_fullsized":"2023-11-28T11:08:19.073418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368409+0000","last_change":"2023-11-28T11:08:17.382413+0000","last_active":"2023-11-28T11:08:18.368409+0000","last_peered":"2023-11-28T11:08:18.368409+0000","last_clean":"2023-11-28T11:08:18.368409+0000","last_became_active":"2023-11-28T11:08:17.382132+0000","last_became_peered":"2023-11-28T11:08:17.382132+0000","last_unstale":"2023-11-28T11:08:18.368409+0000","last_undegraded":"2023-11-28T11:08:18.368409+0000","last_fullsized":"2023-11-28T11:08:18.368409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368518+0000","last_change":"2023-11-28T11:08:17.368106+0000","last_active":"2023-11-28T11:08:18.368518+0000","last_peered":"2023-11-28T11:08:18.368518+0000","last_clean":"2023-11-28T11:08:18.368518+0000","last_became_active":"2023-11-28T11:08:17.367562+0000","last_became_peered":"2023-11-28T11:08:17.367562+0000","last_unstale":"2023-11-28T11:08:18.368518+0000","last_undegraded":"2023-11-28T11:08:18.368518+0000","last_fullsized":"2023-11-28T11:08:18.368518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073955+0000","last_change":"2023-11-28T11:08:17.375521+0000","last_active":"2023-11-28T11:08:19.073955+0000","last_peered":"2023-11-28T11:08:19.073955+0000","last_clean":"2023-11-28T11:08:19.073955+0000","last_became_active":"2023-11-28T11:08:17.375256+0000","last_became_peered":"2023-11-28T11:08:17.375256+0000","last_unstale":"2023-11-28T11:08:19.073955+0000","last_undegraded":"2023-11-28T11:08:19.073955+0000","last_fullsized":"2023-11-28T11:08:19.073955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.382174+0000","last_change":"2023-11-28T11:08:16.975491+0000","last_active":"2023-11-28T11:08:17.382174+0000","last_peered":"2023-11-28T11:08:17.382174+0000","last_clean":"2023-11-28T11:08:17.382174+0000","last_became_active":"2023-11-28T11:08:16.975076+0000","last_became_peered":"2023-11-28T11:08:16.975076+0000","last_unstale":"2023-11-28T11:08:17.382174+0000","last_undegraded":"2023-11-28T11:08:17.382174+0000","last_fullsized":"2023-11-28T11:08:17.382174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365845+0000","last_change":"2023-11-28T11:08:17.371715+0000","last_active":"2023-11-28T11:08:18.365845+0000","last_peered":"2023-11-28T11:08:18.365845+0000","last_clean":"2023-11-28T11:08:18.365845+0000","last_became_active":"2023-11-28T11:08:17.368939+0000","last_became_peered":"2023-11-28T11:08:17.368939+0000","last_unstale":"2023-11-28T11:08:18.365845+0000","last_undegraded":"2023-11-28T11:08:18.365845+0000","last_fullsized":"2023-11-28T11:08:18.365845+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362665+0000","last_change":"2023-11-28T11:08:16.975573+0000","last_active":"2023-11-28T11:08:17.362665+0000","last_peered":"2023-11-28T11:08:17.362665+0000","last_clean":"2023-11-28T11:08:17.362665+0000","last_became_active":"2023-11-28T11:08:16.975216+0000","last_became_peered":"2023-11-28T11:08:16.975216+0000","last_unstale":"2023-11-28T11:08:17.362665+0000","last_undegraded":"2023-11-28T11:08:17.362665+0000","last_fullsized":"2023-11-28T11:08:17.362665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673721+0000","last_change":"2023-11-28T11:08:17.378243+0000","last_active":"2023-11-28T11:08:18.673721+0000","last_peered":"2023-11-28T11:08:18.673721+0000","last_clean":"2023-11-28T11:08:18.673721+0000","last_became_active":"2023-11-28T11:08:17.377866+0000","last_became_peered":"2023-11-28T11:08:17.377866+0000","last_unstale":"2023-11-28T11:08:18.673721+0000","last_undegraded":"2023-11-28T11:08:18.673721+0000","last_fullsized":"2023-11-28T11:08:18.673721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.356027+0000","last_change":"2023-11-28T11:08:16.972779+0000","last_active":"2023-11-28T11:08:17.356027+0000","last_peered":"2023-11-28T11:08:17.356027+0000","last_clean":"2023-11-28T11:08:17.356027+0000","last_became_active":"2023-11-28T11:08:16.972510+0000","last_became_peered":"2023-11-28T11:08:16.972510+0000","last_unstale":"2023-11-28T11:08:17.356027+0000","last_undegraded":"2023-11-28T11:08:17.356027+0000","last_fullsized":"2023-11-28T11:08:17.356027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074026+0000","last_change":"2023-11-28T11:08:17.380638+0000","last_active":"2023-11-28T11:08:19.074026+0000","last_peered":"2023-11-28T11:08:19.074026+0000","last_clean":"2023-11-28T11:08:19.074026+0000","last_became_active":"2023-11-28T11:08:17.380333+0000","last_became_peered":"2023-11-28T11:08:17.380333+0000","last_unstale":"2023-11-28T11:08:19.074026+0000","last_undegraded":"2023-11-28T11:08:19.074026+0000","last_fullsized":"2023-11-28T11:08:19.074026+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:34.711368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.361717+0000","last_change":"2023-11-28T11:08:16.976366+0000","last_active":"2023-11-28T11:08:17.361717+0000","last_peered":"2023-11-28T11:08:17.361717+0000","last_clean":"2023-11-28T11:08:17.361717+0000","last_became_active":"2023-11-28T11:08:16.976072+0000","last_became_peered":"2023-11-28T11:08:16.976072+0000","last_unstale":"2023-11-28T11:08:17.361717+0000","last_undegraded":"2023-11-28T11:08:17.361717+0000","last_fullsized":"2023-11-28T11:08:17.361717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365642+0000","last_change":"2023-11-28T11:08:17.364878+0000","last_active":"2023-11-28T11:08:18.365642+0000","last_peered":"2023-11-28T11:08:18.365642+0000","last_clean":"2023-11-28T11:08:18.365642+0000","last_became_active":"2023-11-28T11:08:17.364628+0000","last_became_peered":"2023-11-28T11:08:17.364628+0000","last_unstale":"2023-11-28T11:08:18.365642+0000","last_undegraded":"2023-11-28T11:08:18.365642+0000","last_fullsized":"2023-11-28T11:08:18.365642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:12.679494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.358873+0000","last_change":"2023-11-28T11:08:16.975388+0000","last_active":"2023-11-28T11:08:17.358873+0000","last_peered":"2023-11-28T11:08:17.358873+0000","last_clean":"2023-11-28T11:08:17.358873+0000","last_became_active":"2023-11-28T11:08:16.974977+0000","last_became_peered":"2023-11-28T11:08:16.974977+0000","last_unstale":"2023-11-28T11:08:17.358873+0000","last_undegraded":"2023-11-28T11:08:17.358873+0000","last_fullsized":"2023-11-28T11:08:17.358873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365734+0000","last_change":"2023-11-28T11:08:17.376435+0000","last_active":"2023-11-28T11:08:18.365734+0000","last_peered":"2023-11-28T11:08:18.365734+0000","last_clean":"2023-11-28T11:08:18.365734+0000","last_became_active":"2023-11-28T11:08:17.376170+0000","last_became_peered":"2023-11-28T11:08:17.376170+0000","last_unstale":"2023-11-28T11:08:18.365734+0000","last_undegraded":"2023-11-28T11:08:18.365734+0000","last_fullsized":"2023-11-28T11:08:18.365734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:24:59.277184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.381284+0000","last_change":"2023-11-28T11:08:16.975413+0000","last_active":"2023-11-28T11:08:17.381284+0000","last_peered":"2023-11-28T11:08:17.381284+0000","last_clean":"2023-11-28T11:08:17.381284+0000","last_became_active":"2023-11-28T11:08:16.975058+0000","last_became_peered":"2023-11-28T11:08:16.975058+0000","last_unstale":"2023-11-28T11:08:17.381284+0000","last_undegraded":"2023-11-28T11:08:17.381284+0000","last_fullsized":"2023-11-28T11:08:17.381284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368111+0000","last_change":"2023-11-28T11:08:17.381210+0000","last_active":"2023-11-28T11:08:18.368111+0000","last_peered":"2023-11-28T11:08:18.368111+0000","last_clean":"2023-11-28T11:08:18.368111+0000","last_became_active":"2023-11-28T11:08:17.380941+0000","last_became_peered":"2023-11-28T11:08:17.380941+0000","last_unstale":"2023-11-28T11:08:18.368111+0000","last_undegraded":"2023-11-28T11:08:18.368111+0000","last_fullsized":"2023-11-28T11:08:18.368111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:35.096193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.387746+0000","last_change":"2023-11-28T11:08:16.973937+0000","last_active":"2023-11-28T11:08:17.387746+0000","last_peered":"2023-11-28T11:08:17.387746+0000","last_clean":"2023-11-28T11:08:17.387746+0000","last_became_active":"2023-11-28T11:08:16.973594+0000","last_became_peered":"2023-11-28T11:08:16.973594+0000","last_unstale":"2023-11-28T11:08:17.387746+0000","last_undegraded":"2023-11-28T11:08:17.387746+0000","last_fullsized":"2023-11-28T11:08:17.387746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:13.313856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369025+0000","last_change":"2023-11-28T11:08:17.380894+0000","last_active":"2023-11-28T11:08:18.369025+0000","last_peered":"2023-11-28T11:08:18.369025+0000","last_clean":"2023-11-28T11:08:18.369025+0000","last_became_active":"2023-11-28T11:08:17.380600+0000","last_became_peered":"2023-11-28T11:08:17.380600+0000","last_unstale":"2023-11-28T11:08:18.369025+0000","last_undegraded":"2023-11-28T11:08:18.369025+0000","last_fullsized":"2023-11-28T11:08:18.369025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377908+0000","last_change":"2023-11-28T11:08:16.977393+0000","last_active":"2023-11-28T11:08:17.377908+0000","last_peered":"2023-11-28T11:08:17.377908+0000","last_clean":"2023-11-28T11:08:17.377908+0000","last_became_active":"2023-11-28T11:08:16.976585+0000","last_became_peered":"2023-11-28T11:08:16.976585+0000","last_unstale":"2023-11-28T11:08:17.377908+0000","last_undegraded":"2023-11-28T11:08:17.377908+0000","last_fullsized":"2023-11-28T11:08:17.377908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:36.981686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365857+0000","last_change":"2023-11-28T11:08:17.376235+0000","last_active":"2023-11-28T11:08:18.365857+0000","last_peered":"2023-11-28T11:08:18.365857+0000","last_clean":"2023-11-28T11:08:18.365857+0000","last_became_active":"2023-11-28T11:08:17.375675+0000","last_became_peered":"2023-11-28T11:08:17.375675+0000","last_unstale":"2023-11-28T11:08:18.365857+0000","last_undegraded":"2023-11-28T11:08:18.365857+0000","last_fullsized":"2023-11-28T11:08:18.365857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.359416+0000","last_change":"2023-11-28T11:08:16.974456+0000","last_active":"2023-11-28T11:08:17.359416+0000","last_peered":"2023-11-28T11:08:17.359416+0000","last_clean":"2023-11-28T11:08:17.359416+0000","last_became_active":"2023-11-28T11:08:16.974158+0000","last_became_peered":"2023-11-28T11:08:16.974158+0000","last_unstale":"2023-11-28T11:08:17.359416+0000","last_undegraded":"2023-11-28T11:08:17.359416+0000","last_fullsized":"2023-11-28T11:08:17.359416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074033+0000","last_change":"2023-11-28T11:08:17.375410+0000","last_active":"2023-11-28T11:08:19.074033+0000","last_peered":"2023-11-28T11:08:19.074033+0000","last_clean":"2023-11-28T11:08:19.074033+0000","last_became_active":"2023-11-28T11:08:17.374828+0000","last_became_peered":"2023-11-28T11:08:17.374828+0000","last_unstale":"2023-11-28T11:08:19.074033+0000","last_undegraded":"2023-11-28T11:08:19.074033+0000","last_fullsized":"2023-11-28T11:08:19.074033+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365862+0000","last_change":"2023-11-28T11:08:16.978277+0000","last_active":"2023-11-28T11:08:17.365862+0000","last_peered":"2023-11-28T11:08:17.365862+0000","last_clean":"2023-11-28T11:08:17.365862+0000","last_became_active":"2023-11-28T11:08:16.977994+0000","last_became_peered":"2023-11-28T11:08:16.977994+0000","last_unstale":"2023-11-28T11:08:17.365862+0000","last_undegraded":"2023-11-28T11:08:17.365862+0000","last_fullsized":"2023-11-28T11:08:17.365862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:55.975785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673702+0000","last_change":"2023-11-28T11:08:17.369372+0000","last_active":"2023-11-28T11:08:18.673702+0000","last_peered":"2023-11-28T11:08:18.673702+0000","last_clean":"2023-11-28T11:08:18.673702+0000","last_became_active":"2023-11-28T11:08:17.369115+0000","last_became_peered":"2023-11-28T11:08:17.369115+0000","last_unstale":"2023-11-28T11:08:18.673702+0000","last_undegraded":"2023-11-28T11:08:18.673702+0000","last_fullsized":"2023-11-28T11:08:18.673702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.386102+0000","last_change":"2023-11-28T11:08:16.974290+0000","last_active":"2023-11-28T11:08:17.386102+0000","last_peered":"2023-11-28T11:08:17.386102+0000","last_clean":"2023-11-28T11:08:17.386102+0000","last_became_active":"2023-11-28T11:08:16.973952+0000","last_became_peered":"2023-11-28T11:08:16.973952+0000","last_unstale":"2023-11-28T11:08:17.386102+0000","last_undegraded":"2023-11-28T11:08:17.386102+0000","last_fullsized":"2023-11-28T11:08:17.386102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365978+0000","last_change":"2023-11-28T11:08:17.376526+0000","last_active":"2023-11-28T11:08:18.365978+0000","last_peered":"2023-11-28T11:08:18.365978+0000","last_clean":"2023-11-28T11:08:18.365978+0000","last_became_active":"2023-11-28T11:08:17.376285+0000","last_became_peered":"2023-11-28T11:08:17.376285+0000","last_unstale":"2023-11-28T11:08:18.365978+0000","last_undegraded":"2023-11-28T11:08:18.365978+0000","last_fullsized":"2023-11-28T11:08:18.365978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377922+0000","last_change":"2023-11-28T11:08:16.976879+0000","last_active":"2023-11-28T11:08:17.377922+0000","last_peered":"2023-11-28T11:08:17.377922+0000","last_clean":"2023-11-28T11:08:17.377922+0000","last_became_active":"2023-11-28T11:08:16.976530+0000","last_became_peered":"2023-11-28T11:08:16.976530+0000","last_unstale":"2023-11-28T11:08:17.377922+0000","last_undegraded":"2023-11-28T11:08:17.377922+0000","last_fullsized":"2023-11-28T11:08:17.377922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:59.917012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073041+0000","last_change":"2023-11-28T11:08:17.375192+0000","last_active":"2023-11-28T11:08:19.073041+0000","last_peered":"2023-11-28T11:08:19.073041+0000","last_clean":"2023-11-28T11:08:19.073041+0000","last_became_active":"2023-11-28T11:08:17.374903+0000","last_became_peered":"2023-11-28T11:08:17.374903+0000","last_unstale":"2023-11-28T11:08:19.073041+0000","last_undegraded":"2023-11-28T11:08:19.073041+0000","last_fullsized":"2023-11-28T11:08:19.073041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377956+0000","last_change":"2023-11-28T11:08:16.980934+0000","last_active":"2023-11-28T11:08:17.377956+0000","last_peered":"2023-11-28T11:08:17.377956+0000","last_clean":"2023-11-28T11:08:17.377956+0000","last_became_active":"2023-11-28T11:08:16.980585+0000","last_became_peered":"2023-11-28T11:08:16.980585+0000","last_unstale":"2023-11-28T11:08:17.377956+0000","last_undegraded":"2023-11-28T11:08:17.377956+0000","last_fullsized":"2023-11-28T11:08:17.377956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:33.709483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368728+0000","last_change":"2023-11-28T11:08:17.374922+0000","last_active":"2023-11-28T11:08:18.368728+0000","last_peered":"2023-11-28T11:08:18.368728+0000","last_clean":"2023-11-28T11:08:18.368728+0000","last_became_active":"2023-11-28T11:08:17.374631+0000","last_became_peered":"2023-11-28T11:08:17.374631+0000","last_unstale":"2023-11-28T11:08:18.368728+0000","last_undegraded":"2023-11-28T11:08:18.368728+0000","last_fullsized":"2023-11-28T11:08:18.368728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:28:26.671538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.375561+0000","last_change":"2023-11-28T11:08:16.978233+0000","last_active":"2023-11-28T11:08:17.375561+0000","last_peered":"2023-11-28T11:08:17.375561+0000","last_clean":"2023-11-28T11:08:17.375561+0000","last_became_active":"2023-11-28T11:08:16.977953+0000","last_became_peered":"2023-11-28T11:08:16.977953+0000","last_unstale":"2023-11-28T11:08:17.375561+0000","last_undegraded":"2023-11-28T11:08:17.375561+0000","last_fullsized":"2023-11-28T11:08:17.375561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073837+0000","last_change":"2023-11-28T11:08:17.381282+0000","last_active":"2023-11-28T11:08:19.073837+0000","last_peered":"2023-11-28T11:08:19.073837+0000","last_clean":"2023-11-28T11:08:19.073837+0000","last_became_active":"2023-11-28T11:08:17.381013+0000","last_became_peered":"2023-11-28T11:08:17.381013+0000","last_unstale":"2023-11-28T11:08:19.073837+0000","last_undegraded":"2023-11-28T11:08:19.073837+0000","last_fullsized":"2023-11-28T11:08:19.073837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:09.760690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357896+0000","last_change":"2023-11-28T11:08:16.976378+0000","last_active":"2023-11-28T11:08:17.357896+0000","last_peered":"2023-11-28T11:08:17.357896+0000","last_clean":"2023-11-28T11:08:17.357896+0000","last_became_active":"2023-11-28T11:08:16.974426+0000","last_became_peered":"2023-11-28T11:08:16.974426+0000","last_unstale":"2023-11-28T11:08:17.357896+0000","last_undegraded":"2023-11-28T11:08:17.357896+0000","last_fullsized":"2023-11-28T11:08:17.357896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673645+0000","last_change":"2023-11-28T11:08:17.366381+0000","last_active":"2023-11-28T11:08:18.673645+0000","last_peered":"2023-11-28T11:08:18.673645+0000","last_clean":"2023-11-28T11:08:18.673645+0000","last_became_active":"2023-11-28T11:08:17.366129+0000","last_became_peered":"2023-11-28T11:08:17.366129+0000","last_unstale":"2023-11-28T11:08:18.673645+0000","last_undegraded":"2023-11-28T11:08:18.673645+0000","last_fullsized":"2023-11-28T11:08:18.673645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.380305+0000","last_change":"2023-11-28T11:08:16.973851+0000","last_active":"2023-11-28T11:08:17.380305+0000","last_peered":"2023-11-28T11:08:17.380305+0000","last_clean":"2023-11-28T11:08:17.380305+0000","last_became_active":"2023-11-28T11:08:16.973588+0000","last_became_peered":"2023-11-28T11:08:16.973588+0000","last_unstale":"2023-11-28T11:08:17.380305+0000","last_undegraded":"2023-11-28T11:08:17.380305+0000","last_fullsized":"2023-11-28T11:08:17.380305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369164+0000","last_change":"2023-11-28T11:08:17.372448+0000","last_active":"2023-11-28T11:08:18.369164+0000","last_peered":"2023-11-28T11:08:18.369164+0000","last_clean":"2023-11-28T11:08:18.369164+0000","last_became_active":"2023-11-28T11:08:17.372132+0000","last_became_peered":"2023-11-28T11:08:17.372132+0000","last_unstale":"2023-11-28T11:08:18.369164+0000","last_undegraded":"2023-11-28T11:08:18.369164+0000","last_fullsized":"2023-11-28T11:08:18.369164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:12.679494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.371186+0000","last_change":"2023-11-28T11:08:16.975018+0000","last_active":"2023-11-28T11:08:17.371186+0000","last_peered":"2023-11-28T11:08:17.371186+0000","last_clean":"2023-11-28T11:08:17.371186+0000","last_became_active":"2023-11-28T11:08:16.974709+0000","last_became_peered":"2023-11-28T11:08:16.974709+0000","last_unstale":"2023-11-28T11:08:17.371186+0000","last_undegraded":"2023-11-28T11:08:17.371186+0000","last_fullsized":"2023-11-28T11:08:17.371186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369338+0000","last_change":"2023-11-28T11:08:17.372568+0000","last_active":"2023-11-28T11:08:18.369338+0000","last_peered":"2023-11-28T11:08:18.369338+0000","last_clean":"2023-11-28T11:08:18.369338+0000","last_became_active":"2023-11-28T11:08:17.372279+0000","last_became_peered":"2023-11-28T11:08:17.372279+0000","last_unstale":"2023-11-28T11:08:18.369338+0000","last_undegraded":"2023-11-28T11:08:18.369338+0000","last_fullsized":"2023-11-28T11:08:18.369338+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:02.566272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364207+0000","last_change":"2023-11-28T11:08:16.976973+0000","last_active":"2023-11-28T11:08:17.364207+0000","last_peered":"2023-11-28T11:08:17.364207+0000","last_clean":"2023-11-28T11:08:17.364207+0000","last_became_active":"2023-11-28T11:08:16.976661+0000","last_became_peered":"2023-11-28T11:08:16.976661+0000","last_unstale":"2023-11-28T11:08:17.364207+0000","last_undegraded":"2023-11-28T11:08:17.364207+0000","last_fullsized":"2023-11-28T11:08:17.364207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:43.575239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369260+0000","last_change":"2023-11-28T11:08:17.381198+0000","last_active":"2023-11-28T11:08:18.369260+0000","last_peered":"2023-11-28T11:08:18.369260+0000","last_clean":"2023-11-28T11:08:18.369260+0000","last_became_active":"2023-11-28T11:08:17.380937+0000","last_became_peered":"2023-11-28T11:08:17.380937+0000","last_unstale":"2023-11-28T11:08:18.369260+0000","last_undegraded":"2023-11-28T11:08:18.369260+0000","last_fullsized":"2023-11-28T11:08:18.369260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.355350+0000","last_change":"2023-11-28T11:08:16.966279+0000","last_active":"2023-11-28T11:08:17.355350+0000","last_peered":"2023-11-28T11:08:17.355350+0000","last_clean":"2023-11-28T11:08:17.355350+0000","last_became_active":"2023-11-28T11:08:16.965878+0000","last_became_peered":"2023-11-28T11:08:16.965878+0000","last_unstale":"2023-11-28T11:08:17.355350+0000","last_undegraded":"2023-11-28T11:08:17.355350+0000","last_fullsized":"2023-11-28T11:08:17.355350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368520+0000","last_change":"2023-11-28T11:08:17.373270+0000","last_active":"2023-11-28T11:08:18.368520+0000","last_peered":"2023-11-28T11:08:18.368520+0000","last_clean":"2023-11-28T11:08:18.368520+0000","last_became_active":"2023-11-28T11:08:17.373017+0000","last_became_peered":"2023-11-28T11:08:17.373017+0000","last_unstale":"2023-11-28T11:08:18.368520+0000","last_undegraded":"2023-11-28T11:08:18.368520+0000","last_fullsized":"2023-11-28T11:08:18.368520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365774+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:08:17.365774+0000","last_peered":"2023-11-28T11:08:17.365774+0000","last_clean":"2023-11-28T11:08:17.365774+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:08:17.365774+0000","last_undegraded":"2023-11-28T11:08:17.365774+0000","last_fullsized":"2023-11-28T11:08:17.365774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.358583+0000","last_change":"2023-11-28T11:08:16.975874+0000","last_active":"2023-11-28T11:08:17.358583+0000","last_peered":"2023-11-28T11:08:17.358583+0000","last_clean":"2023-11-28T11:08:17.358583+0000","last_became_active":"2023-11-28T11:08:16.975225+0000","last_became_peered":"2023-11-28T11:08:16.975225+0000","last_unstale":"2023-11-28T11:08:17.358583+0000","last_undegraded":"2023-11-28T11:08:17.358583+0000","last_fullsized":"2023-11-28T11:08:17.358583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368149+0000","last_change":"2023-11-28T11:08:17.371631+0000","last_active":"2023-11-28T11:08:18.368149+0000","last_peered":"2023-11-28T11:08:18.368149+0000","last_clean":"2023-11-28T11:08:18.368149+0000","last_became_active":"2023-11-28T11:08:17.371342+0000","last_became_peered":"2023-11-28T11:08:17.371342+0000","last_unstale":"2023-11-28T11:08:18.368149+0000","last_undegraded":"2023-11-28T11:08:18.368149+0000","last_fullsized":"2023-11-28T11:08:18.368149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357509+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:08:17.357509+0000","last_peered":"2023-11-28T11:08:17.357509+0000","last_clean":"2023-11-28T11:08:17.357509+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:08:17.357509+0000","last_undegraded":"2023-11-28T11:08:17.357509+0000","last_fullsized":"2023-11-28T11:08:17.357509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362711+0000","last_change":"2023-11-28T11:08:16.974904+0000","last_active":"2023-11-28T11:08:17.362711+0000","last_peered":"2023-11-28T11:08:17.362711+0000","last_clean":"2023-11-28T11:08:17.362711+0000","last_became_active":"2023-11-28T11:08:16.974634+0000","last_became_peered":"2023-11-28T11:08:16.974634+0000","last_unstale":"2023-11-28T11:08:17.362711+0000","last_undegraded":"2023-11-28T11:08:17.362711+0000","last_fullsized":"2023-11-28T11:08:17.362711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368410+0000","last_change":"2023-11-28T11:08:17.371181+0000","last_active":"2023-11-28T11:08:18.368410+0000","last_peered":"2023-11-28T11:08:18.368410+0000","last_clean":"2023-11-28T11:08:18.368410+0000","last_became_active":"2023-11-28T11:08:17.370931+0000","last_became_peered":"2023-11-28T11:08:17.370931+0000","last_unstale":"2023-11-28T11:08:18.368410+0000","last_undegraded":"2023-11-28T11:08:18.368410+0000","last_fullsized":"2023-11-28T11:08:18.368410+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.359510+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:08:17.359510+0000","last_peered":"2023-11-28T11:08:17.359510+0000","last_clean":"2023-11-28T11:08:17.359510+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:08:17.359510+0000","last_undegraded":"2023-11-28T11:08:17.359510+0000","last_fullsized":"2023-11-28T11:08:17.359510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362780+0000","last_change":"2023-11-28T11:08:16.972359+0000","last_active":"2023-11-28T11:08:17.362780+0000","last_peered":"2023-11-28T11:08:17.362780+0000","last_clean":"2023-11-28T11:08:17.362780+0000","last_became_active":"2023-11-28T11:08:16.972074+0000","last_became_peered":"2023-11-28T11:08:16.972074+0000","last_unstale":"2023-11-28T11:08:17.362780+0000","last_undegraded":"2023-11-28T11:08:17.362780+0000","last_fullsized":"2023-11-28T11:08:17.362780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368494+0000","last_change":"2023-11-28T11:08:17.369522+0000","last_active":"2023-11-28T11:08:18.368494+0000","last_peered":"2023-11-28T11:08:18.368494+0000","last_clean":"2023-11-28T11:08:18.368494+0000","last_became_active":"2023-11-28T11:08:17.368628+0000","last_became_peered":"2023-11-28T11:08:17.368628+0000","last_unstale":"2023-11-28T11:08:18.368494+0000","last_undegraded":"2023-11-28T11:08:18.368494+0000","last_fullsized":"2023-11-28T11:08:18.368494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357776+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:08:17.357776+0000","last_peered":"2023-11-28T11:08:17.357776+0000","last_clean":"2023-11-28T11:08:17.357776+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:08:17.357776+0000","last_undegraded":"2023-11-28T11:08:17.357776+0000","last_fullsized":"2023-11-28T11:08:17.357776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364014+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:08:17.364014+0000","last_peered":"2023-11-28T11:08:17.364014+0000","last_clean":"2023-11-28T11:08:17.364014+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:08:17.364014+0000","last_undegraded":"2023-11-28T11:08:17.364014+0000","last_fullsized":"2023-11-28T11:08:17.364014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.355796+0000","last_change":"2023-11-28T11:08:16.968819+0000","last_active":"2023-11-28T11:08:17.355796+0000","last_peered":"2023-11-28T11:08:17.355796+0000","last_clean":"2023-11-28T11:08:17.355796+0000","last_became_active":"2023-11-28T11:08:16.968536+0000","last_became_peered":"2023-11-28T11:08:16.968536+0000","last_unstale":"2023-11-28T11:08:17.355796+0000","last_undegraded":"2023-11-28T11:08:17.355796+0000","last_fullsized":"2023-11-28T11:08:17.355796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073714+0000","last_change":"2023-11-28T11:08:17.381098+0000","last_active":"2023-11-28T11:08:19.073714+0000","last_peered":"2023-11-28T11:08:19.073714+0000","last_clean":"2023-11-28T11:08:19.073714+0000","last_became_active":"2023-11-28T11:08:17.380823+0000","last_became_peered":"2023-11-28T11:08:17.380823+0000","last_unstale":"2023-11-28T11:08:19.073714+0000","last_undegraded":"2023-11-28T11:08:19.073714+0000","last_fullsized":"2023-11-28T11:08:19.073714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:56.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364256+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:08:17.364256+0000","last_peered":"2023-11-28T11:08:17.364256+0000","last_clean":"2023-11-28T11:08:17.364256+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:08:17.364256+0000","last_undegraded":"2023-11-28T11:08:17.364256+0000","last_fullsized":"2023-11-28T11:08:17.364256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.360038+0000","last_change":"2023-11-28T11:08:16.978195+0000","last_active":"2023-11-28T11:08:17.360038+0000","last_peered":"2023-11-28T11:08:17.360038+0000","last_clean":"2023-11-28T11:08:17.360038+0000","last_became_active":"2023-11-28T11:08:16.977906+0000","last_became_peered":"2023-11-28T11:08:16.977906+0000","last_unstale":"2023-11-28T11:08:17.360038+0000","last_undegraded":"2023-11-28T11:08:17.360038+0000","last_fullsized":"2023-11-28T11:08:17.360038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673777+0000","last_change":"2023-11-28T11:08:17.369849+0000","last_active":"2023-11-28T11:08:18.673777+0000","last_peered":"2023-11-28T11:08:18.673777+0000","last_clean":"2023-11-28T11:08:18.673777+0000","last_became_active":"2023-11-28T11:08:17.369614+0000","last_became_peered":"2023-11-28T11:08:17.369614+0000","last_unstale":"2023-11-28T11:08:18.673777+0000","last_undegraded":"2023-11-28T11:08:18.673777+0000","last_fullsized":"2023-11-28T11:08:18.673777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.368123+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:08:17.368123+0000","last_peered":"2023-11-28T11:08:17.368123+0000","last_clean":"2023-11-28T11:08:17.368123+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:08:17.368123+0000","last_undegraded":"2023-11-28T11:08:17.368123+0000","last_fullsized":"2023-11-28T11:08:17.368123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365891+0000","last_change":"2023-11-28T11:08:16.975131+0000","last_active":"2023-11-28T11:08:17.365891+0000","last_peered":"2023-11-28T11:08:17.365891+0000","last_clean":"2023-11-28T11:08:17.365891+0000","last_became_active":"2023-11-28T11:08:16.974846+0000","last_became_peered":"2023-11-28T11:08:16.974846+0000","last_unstale":"2023-11-28T11:08:17.365891+0000","last_undegraded":"2023-11-28T11:08:17.365891+0000","last_fullsized":"2023-11-28T11:08:17.365891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673513+0000","last_change":"2023-11-28T11:08:17.366819+0000","last_active":"2023-11-28T11:08:18.673513+0000","last_peered":"2023-11-28T11:08:18.673513+0000","last_clean":"2023-11-28T11:08:18.673513+0000","last_became_active":"2023-11-28T11:08:17.366576+0000","last_became_peered":"2023-11-28T11:08:17.366576+0000","last_unstale":"2023-11-28T11:08:18.673513+0000","last_undegraded":"2023-11-28T11:08:18.673513+0000","last_fullsized":"2023-11-28T11:08:18.673513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.379003+0000","last_change":"2023-11-28T11:08:16.973768+0000","last_active":"2023-11-28T11:08:17.379003+0000","last_peered":"2023-11-28T11:08:17.379003+0000","last_clean":"2023-11-28T11:08:17.379003+0000","last_became_active":"2023-11-28T11:08:16.973505+0000","last_became_peered":"2023-11-28T11:08:16.973505+0000","last_unstale":"2023-11-28T11:08:17.379003+0000","last_undegraded":"2023-11-28T11:08:17.379003+0000","last_fullsized":"2023-11-28T11:08:17.379003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.367956+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:08:17.367956+0000","last_peered":"2023-11-28T11:08:17.367956+0000","last_clean":"2023-11-28T11:08:17.367956+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:08:17.367956+0000","last_undegraded":"2023-11-28T11:08:17.367956+0000","last_fullsized":"2023-11-28T11:08:17.367956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365501+0000","last_change":"2023-11-28T11:08:17.375741+0000","last_active":"2023-11-28T11:08:18.365501+0000","last_peered":"2023-11-28T11:08:18.365501+0000","last_clean":"2023-11-28T11:08:18.365501+0000","last_became_active":"2023-11-28T11:08:17.375349+0000","last_became_peered":"2023-11-28T11:08:17.375349+0000","last_unstale":"2023-11-28T11:08:18.365501+0000","last_undegraded":"2023-11-28T11:08:18.365501+0000","last_fullsized":"2023-11-28T11:08:18.365501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.372744+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:08:17.372744+0000","last_peered":"2023-11-28T11:08:17.372744+0000","last_clean":"2023-11-28T11:08:17.372744+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:08:17.372744+0000","last_undegraded":"2023-11-28T11:08:17.372744+0000","last_fullsized":"2023-11-28T11:08:17.372744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.363328+0000","last_change":"2023-11-28T11:08:16.974692+0000","last_active":"2023-11-28T11:08:17.363328+0000","last_peered":"2023-11-28T11:08:17.363328+0000","last_clean":"2023-11-28T11:08:17.363328+0000","last_became_active":"2023-11-28T11:08:16.974255+0000","last_became_peered":"2023-11-28T11:08:16.974255+0000","last_unstale":"2023-11-28T11:08:17.363328+0000","last_undegraded":"2023-11-28T11:08:17.363328+0000","last_fullsized":"2023-11-28T11:08:17.363328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368842+0000","last_change":"2023-11-28T11:08:17.373312+0000","last_active":"2023-11-28T11:08:18.368842+0000","last_peered":"2023-11-28T11:08:18.368842+0000","last_clean":"2023-11-28T11:08:18.368842+0000","last_became_active":"2023-11-28T11:08:17.373033+0000","last_became_peered":"2023-11-28T11:08:17.373033+0000","last_unstale":"2023-11-28T11:08:18.368842+0000","last_undegraded":"2023-11-28T11:08:18.368842+0000","last_fullsized":"2023-11-28T11:08:18.368842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:25.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377530+0000","last_change":"2023-11-28T11:08:16.976409+0000","last_active":"2023-11-28T11:08:17.377530+0000","last_peered":"2023-11-28T11:08:17.377530+0000","last_clean":"2023-11-28T11:08:17.377530+0000","last_became_active":"2023-11-28T11:08:16.976127+0000","last_became_peered":"2023-11-28T11:08:16.976127+0000","last_unstale":"2023-11-28T11:08:17.377530+0000","last_undegraded":"2023-11-28T11:08:17.377530+0000","last_fullsized":"2023-11-28T11:08:17.377530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:11.421587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072815+0000","last_change":"2023-11-28T11:08:17.374219+0000","last_active":"2023-11-28T11:08:19.072815+0000","last_peered":"2023-11-28T11:08:19.072815+0000","last_clean":"2023-11-28T11:08:19.072815+0000","last_became_active":"2023-11-28T11:08:17.374010+0000","last_became_peered":"2023-11-28T11:08:17.374010+0000","last_unstale":"2023-11-28T11:08:19.072815+0000","last_undegraded":"2023-11-28T11:08:19.072815+0000","last_fullsized":"2023-11-28T11:08:19.072815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:50.027648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.375928+0000","last_change":"2023-11-28T11:08:16.978678+0000","last_active":"2023-11-28T11:08:17.375928+0000","last_peered":"2023-11-28T11:08:17.375928+0000","last_clean":"2023-11-28T11:08:17.375928+0000","last_became_active":"2023-11-28T11:08:16.978317+0000","last_became_peered":"2023-11-28T11:08:16.978317+0000","last_unstale":"2023-11-28T11:08:17.375928+0000","last_undegraded":"2023-11-28T11:08:17.375928+0000","last_fullsized":"2023-11-28T11:08:17.375928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368851+0000","last_change":"2023-11-28T11:08:17.373647+0000","last_active":"2023-11-28T11:08:18.368851+0000","last_peered":"2023-11-28T11:08:18.368851+0000","last_clean":"2023-11-28T11:08:18.368851+0000","last_became_active":"2023-11-28T11:08:17.373288+0000","last_became_peered":"2023-11-28T11:08:17.373288+0000","last_unstale":"2023-11-28T11:08:18.368851+0000","last_undegraded":"2023-11-28T11:08:18.368851+0000","last_fullsized":"2023-11-28T11:08:18.368851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:59.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.376913+0000","last_change":"2023-11-28T11:08:16.976048+0000","last_active":"2023-11-28T11:08:17.376913+0000","last_peered":"2023-11-28T11:08:17.376913+0000","last_clean":"2023-11-28T11:08:17.376913+0000","last_became_active":"2023-11-28T11:08:16.975600+0000","last_became_peered":"2023-11-28T11:08:16.975600+0000","last_unstale":"2023-11-28T11:08:17.376913+0000","last_undegraded":"2023-11-28T11:08:17.376913+0000","last_fullsized":"2023-11-28T11:08:17.376913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368940+0000","last_change":"2023-11-28T11:08:17.372086+0000","last_active":"2023-11-28T11:08:18.368940+0000","last_peered":"2023-11-28T11:08:18.368940+0000","last_clean":"2023-11-28T11:08:18.368940+0000","last_became_active":"2023-11-28T11:08:17.371833+0000","last_became_peered":"2023-11-28T11:08:17.371833+0000","last_unstale":"2023-11-28T11:08:18.368940+0000","last_undegraded":"2023-11-28T11:08:18.368940+0000","last_fullsized":"2023-11-28T11:08:18.368940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:46.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.380725+0000","last_change":"2023-11-28T11:08:16.964126+0000","last_active":"2023-11-28T11:08:17.380725+0000","last_peered":"2023-11-28T11:08:17.380725+0000","last_clean":"2023-11-28T11:08:17.380725+0000","last_became_active":"2023-11-28T11:08:16.963838+0000","last_became_peered":"2023-11-28T11:08:16.963838+0000","last_unstale":"2023-11-28T11:08:17.380725+0000","last_undegraded":"2023-11-28T11:08:17.380725+0000","last_fullsized":"2023-11-28T11:08:17.380725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365611+0000","last_change":"2023-11-28T11:08:17.361919+0000","last_active":"2023-11-28T11:08:18.365611+0000","last_peered":"2023-11-28T11:08:18.365611+0000","last_clean":"2023-11-28T11:08:18.365611+0000","last_became_active":"2023-11-28T11:08:17.361654+0000","last_became_peered":"2023-11-28T11:08:17.361654+0000","last_unstale":"2023-11-28T11:08:18.365611+0000","last_undegraded":"2023-11-28T11:08:18.365611+0000","last_fullsized":"2023-11-28T11:08:18.365611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:46.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.372346+0000","last_change":"2023-11-28T11:08:16.974232+0000","last_active":"2023-11-28T11:08:17.372346+0000","last_peered":"2023-11-28T11:08:17.372346+0000","last_clean":"2023-11-28T11:08:17.372346+0000","last_became_active":"2023-11-28T11:08:16.972983+0000","last_became_peered":"2023-11-28T11:08:16.972983+0000","last_unstale":"2023-11-28T11:08:17.372346+0000","last_undegraded":"2023-11-28T11:08:17.372346+0000","last_fullsized":"2023-11-28T11:08:17.372346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368650+0000","last_change":"2023-11-28T11:08:17.386923+0000","last_active":"2023-11-28T11:08:18.368650+0000","last_peered":"2023-11-28T11:08:18.368650+0000","last_clean":"2023-11-28T11:08:18.368650+0000","last_became_active":"2023-11-28T11:08:17.386690+0000","last_became_peered":"2023-11-28T11:08:17.386690+0000","last_unstale":"2023-11-28T11:08:18.368650+0000","last_undegraded":"2023-11-28T11:08:18.368650+0000","last_fullsized":"2023-11-28T11:08:18.368650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:35.096193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074332+0000","last_change":"2023-11-28T11:08:17.385964+0000","last_active":"2023-11-28T11:08:19.074332+0000","last_peered":"2023-11-28T11:08:19.074332+0000","last_clean":"2023-11-28T11:08:19.074332+0000","last_became_active":"2023-11-28T11:08:17.385673+0000","last_became_peered":"2023-11-28T11:08:17.385673+0000","last_unstale":"2023-11-28T11:08:19.074332+0000","last_undegraded":"2023-11-28T11:08:19.074332+0000","last_fullsized":"2023-11-28T11:08:19.074332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:37.983570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365540+0000","last_change":"2023-11-28T11:08:17.372667+0000","last_active":"2023-11-28T11:08:18.365540+0000","last_peered":"2023-11-28T11:08:18.365540+0000","last_clean":"2023-11-28T11:08:18.365540+0000","last_became_active":"2023-11-28T11:08:17.372363+0000","last_became_peered":"2023-11-28T11:08:17.372363+0000","last_unstale":"2023-11-28T11:08:18.365540+0000","last_undegraded":"2023-11-28T11:08:18.365540+0000","last_fullsized":"2023-11-28T11:08:18.365540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:25.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368769+0000","last_change":"2023-11-28T11:08:17.386764+0000","last_active":"2023-11-28T11:08:18.368769+0000","last_peered":"2023-11-28T11:08:18.368769+0000","last_clean":"2023-11-28T11:08:18.368769+0000","last_became_active":"2023-11-28T11:08:17.386525+0000","last_became_peered":"2023-11-28T11:08:17.386525+0000","last_unstale":"2023-11-28T11:08:18.368769+0000","last_undegraded":"2023-11-28T11:08:18.368769+0000","last_fullsized":"2023-11-28T11:08:18.368769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.366116+0000","last_change":"2023-11-28T11:08:17.378211+0000","last_active":"2023-11-28T11:08:18.366116+0000","last_peered":"2023-11-28T11:08:18.366116+0000","last_clean":"2023-11-28T11:08:18.366116+0000","last_became_active":"2023-11-28T11:08:17.376746+0000","last_became_peered":"2023-11-28T11:08:17.376746+0000","last_unstale":"2023-11-28T11:08:18.366116+0000","last_undegraded":"2023-11-28T11:08:18.366116+0000","last_fullsized":"2023-11-28T11:08:18.366116+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368303+0000","last_change":"2023-11-28T11:08:17.375172+0000","last_active":"2023-11-28T11:08:18.368303+0000","last_peered":"2023-11-28T11:08:18.368303+0000","last_clean":"2023-11-28T11:08:18.368303+0000","last_became_active":"2023-11-28T11:08:17.374911+0000","last_became_peered":"2023-11-28T11:08:17.374911+0000","last_unstale":"2023-11-28T11:08:18.368303+0000","last_undegraded":"2023-11-28T11:08:18.368303+0000","last_fullsized":"2023-11-28T11:08:18.368303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9108,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362732,"statfs":{"total":96636764160,"available":96627437568,"internally_reserved":0,"allocated":864256,"data_stored":681731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7988,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363852,"statfs":{"total":96636764160,"available":96628584448,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7988,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363852,"statfs":{"total":96636764160,"available":96628584448,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":680533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7924,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363916,"statfs":{"total":96636764160,"available":96628649984,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8056,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363784,"statfs":{"total":96636764160,"available":96628514816,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":91203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7996,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363844,"statfs":{"total":96636764160,"available":96628576256,"internally_reserved":0,"allocated":249856,"data_stored":90235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:08:26.702 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:08:26.703 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T11:08:26.703 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T11:08:26.703 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T11:08:26.703 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T11:08:26.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.861+0000 7fc1a4102700 1 Processor -- start 2023-11-28T11:08:26.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.861+0000 7fc1a4102700 1 -- start start 2023-11-28T11:08:26.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a4102700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c06c990 0x7fc19c06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:26.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a4102700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc19c0ca240 0x7fc19c0ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:26.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a4102700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19c06d320 con 0x7fc19c06e9f0 2023-11-28T11:08:26.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a4102700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19c06ae60 con 0x7fc19c0ca240 2023-11-28T11:08:26.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a4102700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c06a270 con 0x7fc19c06c990 2023-11-28T11:08:26.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a269f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc19c0ca240 0x7fc19c0ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:26.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a1e9e700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc19c06e9f0 0x7fc19c06b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46820/0 (socket says 172.21.15.156:46820) 2023-11-28T11:08:26.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a1e9e700 1 -- 172.21.15.156:0/3169108127 learned_addr learned my addr 172.21.15.156:0/3169108127 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:26.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a169d700 1 --2- 172.21.15.156:0/3169108127 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c06c990 0x7fc19c06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:26.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a269f700 1 -- 172.21.15.156:0/3169108127 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c06c990 msgr2=0x7fc19c06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:26.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a269f700 1 --2- 172.21.15.156:0/3169108127 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c06c990 0x7fc19c06cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:26.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a0e9c700 1 -- 172.21.15.156:0/3169108127 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037822567 0 0) 0x7fc19c06d320 con 0x7fc19c06e9f0 2023-11-28T11:08:26.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a269f700 1 -- 172.21.15.156:0/3169108127 >> v1:172.21.15.156:6789/0 conn(0x7fc19c06e9f0 legacy=0x7fc19c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:26.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.862+0000 7fc1a269f700 1 -- 172.21.15.156:0/3169108127 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c0cabe0 con 0x7fc19c0ca240 2023-11-28T11:08:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.863+0000 7fc1a269f700 1 --2- 172.21.15.156:0/3169108127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19c0ca240 0x7fc19c0ca6a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc194009d90 tx=0x7fc19400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846c7f6ecc94b4d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.863+0000 7fc1a0e9c700 1 -- 172.21.15.156:0/3169108127 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19400e070 con 0x7fc19c0ca240 2023-11-28T11:08:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.863+0000 7fc1a0e9c700 1 -- 172.21.15.156:0/3169108127 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19400bb20 con 0x7fc19c0ca240 2023-11-28T11:08:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.863+0000 7fc1a0e9c700 1 -- 172.21.15.156:0/3169108127 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194012430 con 0x7fc19c0ca240 2023-11-28T11:08:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.863+0000 7fc1a4102700 1 -- 172.21.15.156:0/3169108127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19c0ca240 msgr2=0x7fc19c0ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.863+0000 7fc1a4102700 1 --2- 172.21.15.156:0/3169108127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19c0ca240 0x7fc19c0ca6a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc194009d90 tx=0x7fc19400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.863+0000 7fc1a4102700 1 -- 172.21.15.156:0/3169108127 shutdown_connections 2023-11-28T11:08:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.863+0000 7fc1a4102700 1 --2- 172.21.15.156:0/3169108127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19c0ca240 0x7fc19c0ca6a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.863+0000 7fc1a4102700 1 --2- 172.21.15.156:0/3169108127 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c06c990 0x7fc19c06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.863+0000 7fc1a4102700 1 -- 172.21.15.156:0/3169108127 >> 172.21.15.156:0/3169108127 conn(0x7fc19c0b12b0 msgr2=0x7fc19c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.863+0000 7fc1a4102700 1 -- 172.21.15.156:0/3169108127 shutdown_connections 2023-11-28T11:08:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.863+0000 7fc1a4102700 1 -- 172.21.15.156:0/3169108127 wait complete. 2023-11-28T11:08:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.865+0000 7fc1a4102700 1 Processor -- start 2023-11-28T11:08:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.865+0000 7fc1a4102700 1 -- start start 2023-11-28T11:08:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.865+0000 7fc1a4102700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19c06c990 0x7fc19c159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.865+0000 7fc1a4102700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c06e9f0 0x7fc19c0c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.865+0000 7fc1a4102700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc19c0ca240 0x7fc19c0b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.865+0000 7fc1a4102700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c15a130 con 0x7fc19c06c990 2023-11-28T11:08:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.865+0000 7fc1a4102700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c1535d0 con 0x7fc19c0ca240 2023-11-28T11:08:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.865+0000 7fc1a4102700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0c7b80 con 0x7fc19c06e9f0 2023-11-28T11:08:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.865+0000 7fc1a1e9e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19c06c990 0x7fc19c159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.865+0000 7fc1a1e9e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19c06c990 0x7fc19c159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60182/0 (socket says 172.21.15.156:60182) 2023-11-28T11:08:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1a1e9e700 1 -- 172.21.15.156:0/179306348 learned_addr learned my addr 172.21.15.156:0/179306348 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1a269f700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc19c0ca240 0x7fc19c0b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1a169d700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c06e9f0 0x7fc19c0c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1a1e9e700 1 -- 172.21.15.156:0/179306348 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c06e9f0 msgr2=0x7fc19c0c2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1a1e9e700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c06e9f0 0x7fc19c0c2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1a1e9e700 1 -- 172.21.15.156:0/179306348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc19c0ca240 msgr2=0x7fc19c0b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1a1e9e700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc19c0ca240 0x7fc19c0b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1a1e9e700 1 -- 172.21.15.156:0/179306348 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc194009a70 con 0x7fc19c06c990 2023-11-28T11:08:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1a269f700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc19c0ca240 0x7fc19c0b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1a1e9e700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19c06c990 0x7fc19c159bf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00c580 tx=0x7fc18c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1927fc700 1 -- 172.21.15.156:0/179306348 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004c80 con 0x7fc19c06c990 2023-11-28T11:08:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1927fc700 1 -- 172.21.15.156:0/179306348 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c004e00 con 0x7fc19c06c990 2023-11-28T11:08:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1927fc700 1 -- 172.21.15.156:0/179306348 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c007310 con 0x7fc19c06c990 2023-11-28T11:08:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1a4102700 1 -- 172.21.15.156:0/179306348 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19c06a810 con 0x7fc19c06c990 2023-11-28T11:08:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.866+0000 7fc1a4102700 1 -- 172.21.15.156:0/179306348 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c06a530 con 0x7fc19c06c990 2023-11-28T11:08:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.867+0000 7fc1a4102700 1 -- 172.21.15.156:0/179306348 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19c06aaf0 con 0x7fc19c06c990 2023-11-28T11:08:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.868+0000 7fc1927fc700 1 -- 172.21.15.156:0/179306348 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc18c01a020 con 0x7fc19c06c990 2023-11-28T11:08:26.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.868+0000 7fc1927fc700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc1800682f0 0x7fc18006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:26.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.868+0000 7fc1927fc700 1 -- 172.21.15.156:0/179306348 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc18c0858f0 con 0x7fc19c06c990 2023-11-28T11:08:26.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.868+0000 7fc1a169d700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc1800682f0 0x7fc18006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:26.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.868+0000 7fc1a169d700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc1800682f0 0x7fc18006a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc19c160d40 tx=0x7fc198008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:26.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:26.872+0000 7fc1927fc700 1 -- 172.21.15.156:0/179306348 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc18c052150 con 0x7fc19c06c990 2023-11-28T11:08:27.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.106+0000 7fc1a4102700 1 -- 172.21.15.156:0/179306348 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc19c0cd5b0 con 0x7fc19c06c990 2023-11-28T11:08:27.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.109+0000 7fc1927fc700 1 -- 172.21.15.156:0/179306348 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+166386 (secure 0 0 0) 0x7fc18c055770 con 0x7fc19c06c990 2023-11-28T11:08:27.111 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:27.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.111+0000 7fc1a4102700 1 -- 172.21.15.156:0/179306348 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc1800682f0 msgr2=0x7fc18006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.111+0000 7fc1a4102700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc1800682f0 0x7fc18006a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc19c160d40 tx=0x7fc198008040 comp rx=0 tx=0).stop 2023-11-28T11:08:27.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.111+0000 7fc1a4102700 1 -- 172.21.15.156:0/179306348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19c06c990 msgr2=0x7fc19c159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.111+0000 7fc1a4102700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19c06c990 0x7fc19c159bf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00c580 tx=0x7fc18c00c940 comp rx=0 tx=0).stop 2023-11-28T11:08:27.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.111+0000 7fc1a4102700 1 -- 172.21.15.156:0/179306348 shutdown_connections 2023-11-28T11:08:27.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.111+0000 7fc1a4102700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc19c0ca240 0x7fc19c0b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.111+0000 7fc1a4102700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19c06c990 0x7fc19c159bf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.111+0000 7fc1a4102700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc1800682f0 0x7fc18006a7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.111+0000 7fc1a4102700 1 --2- 172.21.15.156:0/179306348 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c06e9f0 0x7fc19c0c2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.111+0000 7fc1a4102700 1 -- 172.21.15.156:0/179306348 >> 172.21.15.156:0/179306348 conn(0x7fc19c0b12b0 msgr2=0x7fc19c06b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:27.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.111+0000 7fc1a4102700 1 -- 172.21.15.156:0/179306348 shutdown_connections 2023-11-28T11:08:27.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.111+0000 7fc1a4102700 1 -- 172.21.15.156:0/179306348 wait complete. 2023-11-28T11:08:27.125 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":103293},{"type":"v1","addr":"172.21.15.156:6833","nonce":103293}]},"active_addr":"172.21.15.156:6833/103293","active_change":"2023-11-28T11:08:00.517815+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4113,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.156:0","nonce":2786427730}]},{"addrvec":[{"type":"v2","addr":"172.21.15.156:0","nonce":2535294241}]},{"addrvec":[{"type":"v2","addr":"172.21.15.156:0","nonce":1713473509}]},{"addrvec":[{"type":"v2","addr":"172.21.15.156:0","nonce":3247115136}]}]} 2023-11-28T11:08:27.130 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T11:08:27.131 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T11:08:27.131 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:27.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.250+0000 7fb6baa6c700 1 Processor -- start 2023-11-28T11:08:27.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.250+0000 7fb6baa6c700 1 -- start start 2023-11-28T11:08:27.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.251+0000 7fb6baa6c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb6b4071040 0x7fb6b4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.251+0000 7fb6baa6c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6b4160540 0x7fb6b41609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.251+0000 7fb6baa6c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b4071950 con 0x7fb6b406dee0 2023-11-28T11:08:27.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.251+0000 7fb6baa6c700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b415d060 con 0x7fb6b4071040 2023-11-28T11:08:27.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.251+0000 7fb6baa6c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b415d340 con 0x7fb6b4160540 2023-11-28T11:08:27.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.251+0000 7fb6b3fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb6b406dee0 0x7fb6b406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46834/0 (socket says 172.21.15.156:46834) 2023-11-28T11:08:27.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.251+0000 7fb6b3fff700 1 -- 172.21.15.156:0/792336516 learned_addr learned my addr 172.21.15.156:0/792336516 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:27.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b8808700 1 --2- 172.21.15.156:0/792336516 >> v2:172.21.15.186:3300/0 conn(0x7fb6b4071040 0x7fb6b4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b9009700 1 --2- 172.21.15.156:0/792336516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6b4160540 0x7fb6b41609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b37fe700 1 -- 172.21.15.156:0/792336516 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070150429 0 0) 0x7fb6b4071950 con 0x7fb6b406dee0 2023-11-28T11:08:27.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b37fe700 1 -- 172.21.15.156:0/792336516 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6a40035b0 con 0x7fb6b406dee0 2023-11-28T11:08:27.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b37fe700 1 -- 172.21.15.156:0/792336516 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 913515243 0 0) 0x7fb6a40035b0 con 0x7fb6b406dee0 2023-11-28T11:08:27.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b9009700 1 -- 172.21.15.156:0/792336516 >> v2:172.21.15.186:3300/0 conn(0x7fb6b4071040 msgr2=0x7fb6b4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:27.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b9009700 1 --2- 172.21.15.156:0/792336516 >> v2:172.21.15.186:3300/0 conn(0x7fb6b4071040 0x7fb6b4071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b9009700 1 -- 172.21.15.156:0/792336516 >> v1:172.21.15.156:6789/0 conn(0x7fb6b406dee0 legacy=0x7fb6b406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b9009700 1 -- 172.21.15.156:0/792336516 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b415d620 con 0x7fb6b4160540 2023-11-28T11:08:27.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b9009700 1 --2- 172.21.15.156:0/792336516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6b4160540 0x7fb6b41609c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009d90 tx=0x7fb6ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=556615386c5f6a07 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:27.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b37fe700 1 -- 172.21.15.156:0/792336516 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac00e070 con 0x7fb6b4160540 2023-11-28T11:08:27.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b37fe700 1 -- 172.21.15.156:0/792336516 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac003a20 con 0x7fb6b4160540 2023-11-28T11:08:27.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.252+0000 7fb6b37fe700 1 -- 172.21.15.156:0/792336516 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0001670 con 0x7fb6b4160540 2023-11-28T11:08:27.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.253+0000 7fb6baa6c700 1 -- 172.21.15.156:0/792336516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6b4160540 msgr2=0x7fb6b41609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.253+0000 7fb6baa6c700 1 --2- 172.21.15.156:0/792336516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6b4160540 0x7fb6b41609c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009d90 tx=0x7fb6ac00b330 comp rx=0 tx=0).stop 2023-11-28T11:08:27.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.253+0000 7fb6baa6c700 1 -- 172.21.15.156:0/792336516 shutdown_connections 2023-11-28T11:08:27.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.253+0000 7fb6baa6c700 1 --2- 172.21.15.156:0/792336516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6b4160540 0x7fb6b41609c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.253+0000 7fb6baa6c700 1 --2- 172.21.15.156:0/792336516 >> v2:172.21.15.186:3300/0 conn(0x7fb6b4071040 0x7fb6b4071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.253+0000 7fb6baa6c700 1 -- 172.21.15.156:0/792336516 >> 172.21.15.156:0/792336516 conn(0x7fb6b40d0000 msgr2=0x7fb6b40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:27.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.253+0000 7fb6baa6c700 1 -- 172.21.15.156:0/792336516 shutdown_connections 2023-11-28T11:08:27.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.253+0000 7fb6baa6c700 1 -- 172.21.15.156:0/792336516 wait complete. 2023-11-28T11:08:27.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.254+0000 7fb6baa6c700 1 Processor -- start 2023-11-28T11:08:27.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.254+0000 7fb6baa6c700 1 -- start start 2023-11-28T11:08:27.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6baa6c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6b406dee0 0x7fb6b4159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6baa6c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6b4071040 0x7fb6b4148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6baa6c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6b4160540 0x7fb6b414b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6baa6c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40b3280 con 0x7fb6b4160540 2023-11-28T11:08:27.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6baa6c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b415c0c0 con 0x7fb6b4071040 2023-11-28T11:08:27.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6baa6c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40b5c10 con 0x7fb6b406dee0 2023-11-28T11:08:27.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b9009700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6b4160540 0x7fb6b414b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b9009700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6b4160540 0x7fb6b414b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60198/0 (socket says 172.21.15.156:60198) 2023-11-28T11:08:27.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b9009700 1 -- 172.21.15.156:0/1939213981 learned_addr learned my addr 172.21.15.156:0/1939213981 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:27.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b3fff700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6b4071040 0x7fb6b4148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b8808700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6b406dee0 0x7fb6b4159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b9009700 1 -- 172.21.15.156:0/1939213981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6b406dee0 msgr2=0x7fb6b4159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b9009700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6b406dee0 0x7fb6b4159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b9009700 1 -- 172.21.15.156:0/1939213981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6b4071040 msgr2=0x7fb6b4148b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b9009700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6b4071040 0x7fb6b4148b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b9009700 1 -- 172.21.15.156:0/1939213981 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac009a70 con 0x7fb6b4160540 2023-11-28T11:08:27.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b3fff700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6b4071040 0x7fb6b4148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:27.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b9009700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6b4160540 0x7fb6b414b550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb6b415d480 tx=0x7fb6ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:27.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.255+0000 7fb6b17fa700 1 -- 172.21.15.156:0/1939213981 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0044c0 con 0x7fb6b4160540 2023-11-28T11:08:27.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.256+0000 7fb6b17fa700 1 -- 172.21.15.156:0/1939213981 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac004620 con 0x7fb6b4160540 2023-11-28T11:08:27.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.256+0000 7fb6b17fa700 1 -- 172.21.15.156:0/1939213981 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac012430 con 0x7fb6b4160540 2023-11-28T11:08:27.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.256+0000 7fb6baa6c700 1 -- 172.21.15.156:0/1939213981 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b415ccd0 con 0x7fb6b4160540 2023-11-28T11:08:27.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.256+0000 7fb6baa6c700 1 -- 172.21.15.156:0/1939213981 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b415d900 con 0x7fb6b4160540 2023-11-28T11:08:27.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.256+0000 7fb6baa6c700 1 -- 172.21.15.156:0/1939213981 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b406a270 con 0x7fb6b4160540 2023-11-28T11:08:27.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.257+0000 7fb6b17fa700 1 -- 172.21.15.156:0/1939213981 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb6ac02c070 con 0x7fb6b4160540 2023-11-28T11:08:27.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.257+0000 7fb6b17fa700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb6980682f0 0x7fb69806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.257+0000 7fb6b17fa700 1 -- 172.21.15.156:0/1939213981 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb6ac087750 con 0x7fb6b4160540 2023-11-28T11:08:27.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.257+0000 7fb6b8808700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb6980682f0 0x7fb69806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.258+0000 7fb6b8808700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb6980682f0 0x7fb69806a7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009b40 tx=0x7fb6a8009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:27.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.262+0000 7fb6b17fa700 1 -- 172.21.15.156:0/1939213981 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb6ac053fb0 con 0x7fb6b4160540 2023-11-28T11:08:27.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.444+0000 7fb6baa6c700 1 -- 172.21.15.156:0/1939213981 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6b4149080 con 0x7fb6b4160540 2023-11-28T11:08:27.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.446+0000 7fb6b17fa700 1 -- 172.21.15.156:0/1939213981 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17955 (secure 0 0 0) 0x7fb6ac085640 con 0x7fb6b4160540 2023-11-28T11:08:27.447 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:27.447 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:18.354009+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:27.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.448+0000 7fb6baa6c700 1 -- 172.21.15.156:0/1939213981 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb6980682f0 msgr2=0x7fb69806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.448+0000 7fb6baa6c700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb6980682f0 0x7fb69806a7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009b40 tx=0x7fb6a8009210 comp rx=0 tx=0).stop 2023-11-28T11:08:27.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.448+0000 7fb6baa6c700 1 -- 172.21.15.156:0/1939213981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6b4160540 msgr2=0x7fb6b414b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.448+0000 7fb6baa6c700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6b4160540 0x7fb6b414b550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb6b415d480 tx=0x7fb6ac004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.448+0000 7fb6baa6c700 1 -- 172.21.15.156:0/1939213981 shutdown_connections 2023-11-28T11:08:27.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.448+0000 7fb6baa6c700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6b4160540 0x7fb6b414b550 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.448+0000 7fb6baa6c700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb6980682f0 0x7fb69806a7a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.448+0000 7fb6baa6c700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6b406dee0 0x7fb6b4159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.448+0000 7fb6baa6c700 1 --2- 172.21.15.156:0/1939213981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6b4071040 0x7fb6b4148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.448+0000 7fb6baa6c700 1 -- 172.21.15.156:0/1939213981 >> 172.21.15.156:0/1939213981 conn(0x7fb6b40d0000 msgr2=0x7fb6b40cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:27.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.448+0000 7fb6baa6c700 1 -- 172.21.15.156:0/1939213981 shutdown_connections 2023-11-28T11:08:27.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.448+0000 7fb6baa6c700 1 -- 172.21.15.156:0/1939213981 wait complete. 2023-11-28T11:08:27.462 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T11:08:27.462 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:27.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.582+0000 7f2550971700 1 Processor -- start 2023-11-28T11:08:27.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.582+0000 7f2550971700 1 -- start start 2023-11-28T11:08:27.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2550971700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f254c0cd8c0 0x7f254c140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2550971700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f254c0cdc90 0x7f254c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2550971700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f254c0b2e70 0x7f254c06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2550971700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c13ff00 con 0x7f254c0cdc90 2023-11-28T11:08:27.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2550971700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c1404c0 con 0x7f254c0cd8c0 2023-11-28T11:08:27.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2550971700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c15aa80 con 0x7f254c0b2e70 2023-11-28T11:08:27.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2549d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f254c0cdc90 0x7f254c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2549d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f254c0cdc90 0x7f254c0b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60206/0 (socket says 172.21.15.156:60206) 2023-11-28T11:08:27.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2549d9b700 1 -- 172.21.15.156:0/1574484016 learned_addr learned my addr 172.21.15.156:0/1574484016 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f254a59c700 1 --2- 172.21.15.156:0/1574484016 >> v2:172.21.15.186:3300/0 conn(0x7f254c0cd8c0 0x7f254c140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f254ad9d700 1 --2- 172.21.15.156:0/1574484016 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f254c0b2e70 0x7f254c06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2549d9b700 1 -- 172.21.15.156:0/1574484016 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f254c0b2e70 msgr2=0x7f254c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2549d9b700 1 --2- 172.21.15.156:0/1574484016 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f254c0b2e70 0x7f254c06f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2549d9b700 1 -- 172.21.15.156:0/1574484016 >> v2:172.21.15.186:3300/0 conn(0x7f254c0cd8c0 msgr2=0x7f254c140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2549d9b700 1 --2- 172.21.15.156:0/1574484016 >> v2:172.21.15.186:3300/0 conn(0x7f254c0cd8c0 0x7f254c140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2549d9b700 1 -- 172.21.15.156:0/1574484016 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c0b3310 con 0x7f254c0cdc90 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f254ad9d700 1 --2- 172.21.15.156:0/1574484016 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f254c0b2e70 0x7f254c06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f2549d9b700 1 --2- 172.21.15.156:0/1574484016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f254c0cdc90 0x7f254c0b2930 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f254c070a20 tx=0x7f2540009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a32386dc58cf78 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f254959a700 1 -- 172.21.15.156:0/1574484016 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000b870 con 0x7f254c0cdc90 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f254959a700 1 -- 172.21.15.156:0/1574484016 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254000b9d0 con 0x7f254c0cdc90 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.583+0000 7f254959a700 1 -- 172.21.15.156:0/1574484016 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000bd60 con 0x7f254c0cdc90 2023-11-28T11:08:27.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.584+0000 7f2550971700 1 -- 172.21.15.156:0/1574484016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f254c0cdc90 msgr2=0x7f254c0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.584+0000 7f2550971700 1 --2- 172.21.15.156:0/1574484016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f254c0cdc90 0x7f254c0b2930 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f254c070a20 tx=0x7f2540009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.584+0000 7f2550971700 1 -- 172.21.15.156:0/1574484016 shutdown_connections 2023-11-28T11:08:27.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.584+0000 7f2550971700 1 --2- 172.21.15.156:0/1574484016 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f254c0b2e70 0x7f254c06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.584+0000 7f2550971700 1 --2- 172.21.15.156:0/1574484016 >> v2:172.21.15.186:3300/0 conn(0x7f254c0cd8c0 0x7f254c140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.584+0000 7f2550971700 1 --2- 172.21.15.156:0/1574484016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f254c0cdc90 0x7f254c0b2930 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.584+0000 7f2550971700 1 -- 172.21.15.156:0/1574484016 >> 172.21.15.156:0/1574484016 conn(0x7f254c06aed0 msgr2=0x7f254c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:27.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.584+0000 7f2550971700 1 -- 172.21.15.156:0/1574484016 shutdown_connections 2023-11-28T11:08:27.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.584+0000 7f2550971700 1 -- 172.21.15.156:0/1574484016 wait complete. 2023-11-28T11:08:27.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.585+0000 7f2550971700 1 Processor -- start 2023-11-28T11:08:27.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.585+0000 7f2550971700 1 -- start start 2023-11-28T11:08:27.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.586+0000 7f2550971700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f254c0b2e70 0x7f254c0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.586+0000 7f2550971700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f254c0cd8c0 0x7f254c1efca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.586+0000 7f2550971700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f254c0cdc90 0x7f254c0ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.586+0000 7f2550971700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0b53e0 con 0x7f254c0cd8c0 2023-11-28T11:08:27.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.586+0000 7f2550971700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0b56c0 con 0x7f254c0cdc90 2023-11-28T11:08:27.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.586+0000 7f2550971700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0b59a0 con 0x7f254c0b2e70 2023-11-28T11:08:27.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.586+0000 7f2549d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f254c0cd8c0 0x7f254c1efca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.586+0000 7f2549d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f254c0cd8c0 0x7f254c1efca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60210/0 (socket says 172.21.15.156:60210) 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.586+0000 7f2549d9b700 1 -- 172.21.15.156:0/1432324177 learned_addr learned my addr 172.21.15.156:0/1432324177 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.586+0000 7f254a59c700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f254c0b2e70 0x7f254c0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.586+0000 7f254ad9d700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f254c0cdc90 0x7f254c0ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.587+0000 7f2549d9b700 1 -- 172.21.15.156:0/1432324177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f254c0b2e70 msgr2=0x7f254c0b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.587+0000 7f2549d9b700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f254c0b2e70 0x7f254c0b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.587+0000 7f2549d9b700 1 -- 172.21.15.156:0/1432324177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f254c0cdc90 msgr2=0x7f254c0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.587+0000 7f2549d9b700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f254c0cdc90 0x7f254c0ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.587+0000 7f2549d9b700 1 -- 172.21.15.156:0/1432324177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2540009550 con 0x7f254c0cd8c0 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.587+0000 7f2549d9b700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f254c0cd8c0 0x7f254c1efca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2540000c00 tx=0x7f2540004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.587+0000 7f253affd700 1 -- 172.21.15.156:0/1432324177 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540004480 con 0x7f254c0cd8c0 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.587+0000 7f253affd700 1 -- 172.21.15.156:0/1432324177 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25400045e0 con 0x7f254c0cd8c0 2023-11-28T11:08:27.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.587+0000 7f2550971700 1 -- 172.21.15.156:0/1432324177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c140150 con 0x7f254c0cd8c0 2023-11-28T11:08:27.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.587+0000 7f253affd700 1 -- 172.21.15.156:0/1432324177 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25400038e0 con 0x7f254c0cd8c0 2023-11-28T11:08:27.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.587+0000 7f2550971700 1 -- 172.21.15.156:0/1432324177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c0b35f0 con 0x7f254c0cd8c0 2023-11-28T11:08:27.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.588+0000 7f2550971700 1 -- 172.21.15.156:0/1432324177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c0ce100 con 0x7f254c0cd8c0 2023-11-28T11:08:27.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.588+0000 7f253affd700 1 -- 172.21.15.156:0/1432324177 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f254000f050 con 0x7f254c0cd8c0 2023-11-28T11:08:27.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.588+0000 7f253affd700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f252c0682f0 0x7f252c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.589+0000 7f253affd700 1 -- 172.21.15.156:0/1432324177 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f25400873b0 con 0x7f254c0cd8c0 2023-11-28T11:08:27.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.589+0000 7f254a59c700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f252c0682f0 0x7f252c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.589+0000 7f254a59c700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f252c0682f0 0x7f252c06a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f253c006fd0 tx=0x7f253c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:27.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.593+0000 7f253affd700 1 -- 172.21.15.156:0/1432324177 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2540053c10 con 0x7f254c0cd8c0 2023-11-28T11:08:27.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.774+0000 7f2550971700 1 -- 172.21.15.156:0/1432324177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f254c0b5e90 con 0x7f254c0cd8c0 2023-11-28T11:08:27.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.776+0000 7f253affd700 1 -- 172.21.15.156:0/1432324177 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f2540057230 con 0x7f254c0cd8c0 2023-11-28T11:08:27.777 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:27.777 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:18.354009+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:27.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.778+0000 7f2550971700 1 -- 172.21.15.156:0/1432324177 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f252c0682f0 msgr2=0x7f252c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.778+0000 7f2550971700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f252c0682f0 0x7f252c06a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f253c006fd0 tx=0x7f253c008040 comp rx=0 tx=0).stop 2023-11-28T11:08:27.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.778+0000 7f2550971700 1 -- 172.21.15.156:0/1432324177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f254c0cd8c0 msgr2=0x7f254c1efca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.778+0000 7f2550971700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f254c0cd8c0 0x7f254c1efca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2540000c00 tx=0x7f2540004750 comp rx=0 tx=0).stop 2023-11-28T11:08:27.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.778+0000 7f2550971700 1 -- 172.21.15.156:0/1432324177 shutdown_connections 2023-11-28T11:08:27.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.778+0000 7f2550971700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f254c0cdc90 0x7f254c0ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.778+0000 7f2550971700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f252c0682f0 0x7f252c06a7a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.778+0000 7f2550971700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f254c0b2e70 0x7f254c0b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.778+0000 7f2550971700 1 --2- 172.21.15.156:0/1432324177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f254c0cd8c0 0x7f254c1efca0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.778+0000 7f2550971700 1 -- 172.21.15.156:0/1432324177 >> 172.21.15.156:0/1432324177 conn(0x7f254c06aed0 msgr2=0x7f254c0cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:27.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.778+0000 7f2550971700 1 -- 172.21.15.156:0/1432324177 shutdown_connections 2023-11-28T11:08:27.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.778+0000 7f2550971700 1 -- 172.21.15.156:0/1432324177 wait complete. 2023-11-28T11:08:27.791 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T11:08:27.792 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T11:08:27.792 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T11:08:27.793 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T11:08:27.793 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T11:08:27.793 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T11:08:27.794 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T11:08:27.794 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T11:08:27.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.996+0000 7f1ac22d3700 1 Processor -- start 2023-11-28T11:08:27.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.996+0000 7f1ac22d3700 1 -- start start 2023-11-28T11:08:27.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.997+0000 7f1ac22d3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1abc160b30 0x7f1abc0c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.997+0000 7f1ac22d3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1abc160f00 0x7f1abc164840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.997+0000 7f1ac22d3700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1abc0cc730 0x7f1abc0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.997+0000 7f1ac22d3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1abc164d80 con 0x7f1abc160b30 2023-11-28T11:08:27.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.997+0000 7f1ac22d3700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1abc06ba50 con 0x7f1abc0cc730 2023-11-28T11:08:27.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.997+0000 7f1ac22d3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc164f90 con 0x7f1abc160f00 2023-11-28T11:08:28.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.998+0000 7f1abbfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1abc160b30 0x7f1abc0c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.998+0000 7f1abbfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1abc160b30 0x7f1abc0c1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60216/0 (socket says 172.21.15.156:60216) 2023-11-28T11:08:28.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.998+0000 7f1abbfff700 1 -- 172.21.15.156:0/3494424843 learned_addr learned my addr 172.21.15.156:0/3494424843 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.998+0000 7f1ac0870700 1 --2- 172.21.15.156:0/3494424843 >> v2:172.21.15.186:3300/0 conn(0x7f1abc0cc730 0x7f1abc0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.998+0000 7f1abb7fe700 1 --2- 172.21.15.156:0/3494424843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1abc160f00 0x7f1abc164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.998+0000 7f1ac0870700 1 -- 172.21.15.156:0/3494424843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1abc160f00 msgr2=0x7f1abc164840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.998+0000 7f1ac0870700 1 --2- 172.21.15.156:0/3494424843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1abc160f00 0x7f1abc164840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.998+0000 7f1ac0870700 1 -- 172.21.15.156:0/3494424843 >> v2:172.21.15.156:3300/0 conn(0x7f1abc160b30 msgr2=0x7f1abc0c1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.998+0000 7f1ac0870700 1 --2- 172.21.15.156:0/3494424843 >> v2:172.21.15.156:3300/0 conn(0x7f1abc160b30 0x7f1abc0c1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.998+0000 7f1ac0870700 1 -- 172.21.15.156:0/3494424843 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc165270 con 0x7f1abc0cc730 2023-11-28T11:08:28.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.998+0000 7f1ac0870700 1 --2- 172.21.15.156:0/3494424843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc0cc730 0x7f1abc0b0670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0c9140 tx=0x7f1aac009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=323e4c366702a217 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.999+0000 7f1abaffd700 1 -- 172.21.15.156:0/3494424843 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac011830 con 0x7f1abc0cc730 2023-11-28T11:08:28.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.999+0000 7f1abaffd700 1 -- 172.21.15.156:0/3494424843 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac011990 con 0x7f1abc0cc730 2023-11-28T11:08:28.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:27.999+0000 7f1abaffd700 1 -- 172.21.15.156:0/3494424843 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac011d20 con 0x7f1abc0cc730 2023-11-28T11:08:28.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.000+0000 7f1ac22d3700 1 -- 172.21.15.156:0/3494424843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc0cc730 msgr2=0x7f1abc0b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.000+0000 7f1ac22d3700 1 --2- 172.21.15.156:0/3494424843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc0cc730 0x7f1abc0b0670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0c9140 tx=0x7f1aac009b20 comp rx=0 tx=0).stop 2023-11-28T11:08:28.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.000+0000 7f1ac22d3700 1 -- 172.21.15.156:0/3494424843 shutdown_connections 2023-11-28T11:08:28.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.000+0000 7f1ac22d3700 1 --2- 172.21.15.156:0/3494424843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc0cc730 0x7f1abc0b0670 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.000+0000 7f1ac22d3700 1 --2- 172.21.15.156:0/3494424843 >> v2:172.21.15.156:3300/0 conn(0x7f1abc160b30 0x7f1abc0c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.000+0000 7f1ac22d3700 1 --2- 172.21.15.156:0/3494424843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1abc160f00 0x7f1abc164840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.000+0000 7f1ac22d3700 1 -- 172.21.15.156:0/3494424843 >> 172.21.15.156:0/3494424843 conn(0x7f1abc0cb670 msgr2=0x7f1abc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.000+0000 7f1ac22d3700 1 -- 172.21.15.156:0/3494424843 shutdown_connections 2023-11-28T11:08:28.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.000+0000 7f1ac22d3700 1 -- 172.21.15.156:0/3494424843 wait complete. 2023-11-28T11:08:28.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.000+0000 7f7d42152700 1 Processor -- start 2023-11-28T11:08:28.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.000+0000 7f7d42152700 1 -- start start 2023-11-28T11:08:28.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.002+0000 7f7d42152700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d3c158890 0x7f7d3c0ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.002+0000 7f7d42152700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3c15c490 con 0x7f7d3c06d4f0 2023-11-28T11:08:28.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.002+0000 7f7d42152700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3c0ce650 con 0x7f7d3c0b0d80 2023-11-28T11:08:28.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.002+0000 7f7d42152700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c15c730 con 0x7f7d3c158890 2023-11-28T11:08:28.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.002+0000 7f1ac22d3700 1 Processor -- start 2023-11-28T11:08:28.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.002+0000 7f1ac22d3700 1 -- start start 2023-11-28T11:08:28.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.002+0000 7f7d3b7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d3c158890 0x7f7d3c0ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.002+0000 7f7d3b7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d3c158890 0x7f7d3c0ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:35856/0 (socket says 172.21.15.156:35856) 2023-11-28T11:08:28.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.002+0000 7f7d3b7fe700 1 -- 172.21.15.156:0/2836709570 learned_addr learned my addr 172.21.15.156:0/2836709570 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.002+0000 7f7d3b7fe700 1 -- 172.21.15.156:0/2836709570 >> v1:172.21.15.186:6789/0 conn(0x7f7d3c0b0d80 legacy=0x7f7d3c06b820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:28.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.002+0000 7f7d3b7fe700 1 -- 172.21.15.156:0/2836709570 >> v1:172.21.15.156:6789/0 conn(0x7f7d3c06d4f0 legacy=0x7f7d3c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.002+0000 7f7d3b7fe700 1 -- 172.21.15.156:0/2836709570 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c15ca10 con 0x7f7d3c158890 2023-11-28T11:08:28.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f7d3b7fe700 1 --2- 172.21.15.156:0/2836709570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d3c158890 0x7f7d3c0ce110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7d30007390 tx=0x7f7d300076a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec453eabbb19215 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f7d3a7fc700 1 -- 172.21.15.156:0/2836709570 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000f070 con 0x7f7d3c158890 2023-11-28T11:08:28.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f7d3a7fc700 1 -- 172.21.15.156:0/2836709570 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d30008bf0 con 0x7f7d3c158890 2023-11-28T11:08:28.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f7d3a7fc700 1 -- 172.21.15.156:0/2836709570 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30004030 con 0x7f7d3c158890 2023-11-28T11:08:28.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.004+0000 7f7d42152700 1 -- 172.21.15.156:0/2836709570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d3c158890 msgr2=0x7f7d3c0ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.004+0000 7f7d42152700 1 --2- 172.21.15.156:0/2836709570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d3c158890 0x7f7d3c0ce110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7d30007390 tx=0x7f7d300076a0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f1ac22d3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc0cc730 0x7f1abc152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f1ac22d3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1abc160b30 0x7f1abc151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f1ac22d3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1abc160f00 0x7f1abc0c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f1ac22d3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc06fb30 con 0x7f1abc160f00 2023-11-28T11:08:28.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f1ac22d3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc0717a0 con 0x7f1abc0cc730 2023-11-28T11:08:28.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f1ac22d3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc071a40 con 0x7f1abc160b30 2023-11-28T11:08:28.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f1ac0870700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1abc160f00 0x7f1abc0c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f1ac0870700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1abc160f00 0x7f1abc0c08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60228/0 (socket says 172.21.15.156:60228) 2023-11-28T11:08:28.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.003+0000 7f1ac0870700 1 -- 172.21.15.156:0/2500768150 learned_addr learned my addr 172.21.15.156:0/2500768150 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.004+0000 7f1ac0870700 1 -- 172.21.15.156:0/2500768150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1abc160b30 msgr2=0x7f1abc151c90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:28.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.004+0000 7f1ac0870700 1 --2- 172.21.15.156:0/2500768150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1abc160b30 0x7f1abc151c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.004+0000 7f1ac0870700 1 -- 172.21.15.156:0/2500768150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc0cc730 msgr2=0x7f1abc152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.004+0000 7f1ac0870700 1 --2- 172.21.15.156:0/2500768150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc0cc730 0x7f1abc152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.004+0000 7f1ac0870700 1 -- 172.21.15.156:0/2500768150 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac0094a0 con 0x7f1abc160f00 2023-11-28T11:08:28.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.004+0000 7f1ac0870700 1 --2- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1abc160f00 0x7f1abc0c08c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009b60 tx=0x7f1aac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.004+0000 7f1ab8ff9700 1 -- 172.21.15.156:0/2500768150 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac004450 con 0x7f1abc160f00 2023-11-28T11:08:28.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.004+0000 7f1ab8ff9700 1 -- 172.21.15.156:0/2500768150 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac01f070 con 0x7f1abc160f00 2023-11-28T11:08:28.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.004+0000 7f1ab8ff9700 1 -- 172.21.15.156:0/2500768150 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0038e0 con 0x7f1abc160f00 2023-11-28T11:08:28.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.005+0000 7f1ac22d3700 1 -- 172.21.15.156:0/2500768150 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc0ccbd0 con 0x7f1abc160f00 2023-11-28T11:08:28.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.005+0000 7f1ac22d3700 1 -- 172.21.15.156:0/2500768150 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc165550 con 0x7f1abc160f00 2023-11-28T11:08:28.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.006+0000 7f1ac22d3700 1 -- 172.21.15.156:0/2500768150 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1abc0ccd60 con 0x7f1abc160f00 2023-11-28T11:08:28.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.006+0000 7f7d42152700 1 -- 172.21.15.156:0/2836709570 shutdown_connections 2023-11-28T11:08:28.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.006+0000 7f7d42152700 1 --2- 172.21.15.156:0/2836709570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d3c158890 0x7f7d3c0ce110 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.006+0000 7f7d42152700 1 -- 172.21.15.156:0/2836709570 >> 172.21.15.156:0/2836709570 conn(0x7f7d3c0cb670 msgr2=0x7f7d3c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.007+0000 7f1ab8ff9700 1 -- 172.21.15.156:0/2500768150 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1aac00f050 con 0x7f1abc160f00 2023-11-28T11:08:28.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.007+0000 7f1ab8ff9700 1 --2- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1aa80682f0 0x7f1aa806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.007+0000 7f7d42152700 1 -- 172.21.15.156:0/2836709570 shutdown_connections 2023-11-28T11:08:28.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.007+0000 7f7d42152700 1 -- 172.21.15.156:0/2836709570 wait complete. 2023-11-28T11:08:28.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.007+0000 7f9e70ba5700 1 Processor -- start 2023-11-28T11:08:28.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.007+0000 7fb06ec14700 1 Processor -- start 2023-11-28T11:08:28.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.007+0000 7f1ab8ff9700 1 -- 172.21.15.156:0/2500768150 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1aac021070 con 0x7f1abc160f00 2023-11-28T11:08:28.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.008+0000 7f1ab8ff9700 1 --2- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] conn(0x7f1aa8070b20 0x7f1aa8072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.008+0000 7f1ab8ff9700 1 -- 172.21.15.156:0/2500768150 --> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa80735e0 con 0x7f1aa8070b20 2023-11-28T11:08:28.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.008+0000 7f1ab8ff9700 1 -- 172.21.15.156:0/2500768150 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aac0897b0 con 0x7f1abc160f00 2023-11-28T11:08:28.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.008+0000 7f1abb7fe700 1 --2- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] conn(0x7f1aa8070b20 0x7f1aa8072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.008+0000 7fb06ec14700 1 -- start start 2023-11-28T11:08:28.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f1abb7fe700 1 --2- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] conn(0x7f1aa8070b20 0x7f1aa8072f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f1abbfff700 1 --2- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1aa80682f0 0x7f1aa806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f9b84b0b700 1 Processor -- start 2023-11-28T11:08:28.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7fb06ec14700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb0680c1fd0 0x7fb068161220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7fb06ec14700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb068162210 con 0x7fb068164e60 2023-11-28T11:08:28.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7fb06ec14700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0681649d0 con 0x7fb0680ce480 2023-11-28T11:08:28.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.008+0000 7f9e70ba5700 1 -- start start 2023-11-28T11:08:28.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7fb06ec14700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb068161760 con 0x7fb0680c1fd0 2023-11-28T11:08:28.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f9e70ba5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e6c0ce480 0x7f9e6c0c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f9e70ba5700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9e6c164e60 0x7f9e6c1652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f9e70ba5700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6c162210 con 0x7f9e6c161220 2023-11-28T11:08:28.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f9e70ba5700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6c1649d0 con 0x7f9e6c164e60 2023-11-28T11:08:28.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f9e70ba5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c06c020 con 0x7f9e6c0ce480 2023-11-28T11:08:28.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7f1ab8ff9700 1 -- 172.21.15.156:0/2500768150 <== osd.0 v2:172.21.15.156:6802/103563 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1aa80735e0 con 0x7f1aa8070b20 2023-11-28T11:08:28.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f7d42152700 1 Processor -- start 2023-11-28T11:08:28.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f7d42152700 1 -- start start 2023-11-28T11:08:28.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d42152700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d3c06d4f0 0x7f7d3c0c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d42152700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d3c0b0d80 0x7f7d3c0c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d42152700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d3c158890 0x7f7d3c0c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d42152700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0c8c30 con 0x7f7d3c06d4f0 2023-11-28T11:08:28.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d42152700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0c8020 con 0x7f7d3c0b0d80 2023-11-28T11:08:28.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d42152700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0c8300 con 0x7f7d3c158890 2023-11-28T11:08:28.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7fb067fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb0680c1fd0 0x7fb068161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7fb067fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb0680c1fd0 0x7fb068161220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:43028/0 (socket says 172.21.15.156:43028) 2023-11-28T11:08:28.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7fb067fff700 1 -- 172.21.15.156:0/3267927731 learned_addr learned my addr 172.21.15.156:0/3267927731 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7fb0677fe700 1 -- 172.21.15.156:0/3267927731 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728302237 0 0) 0x7fb0681649d0 con 0x7fb0680ce480 2023-11-28T11:08:28.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7fb0677fe700 1 -- 172.21.15.156:0/3267927731 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0500035b0 con 0x7fb0680ce480 2023-11-28T11:08:28.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7fb067fff700 1 -- 172.21.15.156:0/3267927731 >> v1:172.21.15.186:6789/0 conn(0x7fb0680ce480 legacy=0x7fb06806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7fb067fff700 1 -- 172.21.15.156:0/3267927731 >> v1:172.21.15.156:6789/0 conn(0x7fb068164e60 legacy=0x7fb0680b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7fb067fff700 1 -- 172.21.15.156:0/3267927731 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0680c2450 con 0x7fb0680c1fd0 2023-11-28T11:08:28.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f9e6a59c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e6c0ce480 0x7f9e6c0c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f9e6ad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9e6c164e60 0x7f9e6c1652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f9e6a59c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e6c0ce480 0x7f9e6c0c1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:35870/0 (socket says 172.21.15.156:35870) 2023-11-28T11:08:28.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f9e6a59c700 1 -- 172.21.15.156:0/2087330721 learned_addr learned my addr 172.21.15.156:0/2087330721 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7f9e6ad9d700 1 -- 172.21.15.156:0/2087330721 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e6c0ce480 msgr2=0x7f9e6c0c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7f9e6ad9d700 1 --2- 172.21.15.156:0/2087330721 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e6c0ce480 0x7f9e6c0c1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7f9e6ad9d700 1 -- 172.21.15.156:0/2087330721 >> v1:172.21.15.186:6789/0 conn(0x7f9e6c161220 legacy=0x7f9e6c06b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7f9e6ad9d700 1 -- 172.21.15.156:0/2087330721 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c161970 con 0x7f9e6c164e60 2023-11-28T11:08:28.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.010+0000 7f9e6ad9d700 1 --2- 172.21.15.156:0/2087330721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e6c164e60 0x7f9e6c1652e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009d90 tx=0x7f9e6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dfea98c7c727da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.009+0000 7f9b84b0b700 1 -- start start 2023-11-28T11:08:28.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f9b84b0b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b800ce480 0x7f9b800c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f9b84b0b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9b80161220 0x7f9b8006b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f9b84b0b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b80162210 con 0x7f9b80164e60 2023-11-28T11:08:28.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f9b84b0b700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b801649d0 con 0x7f9b80161220 2023-11-28T11:08:28.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f9b84b0b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8006c020 con 0x7f9b800ce480 2023-11-28T11:08:28.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d3b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d3c06d4f0 0x7f7d3c0c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d3b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d3c06d4f0 0x7f7d3c0c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60250/0 (socket says 172.21.15.156:60250) 2023-11-28T11:08:28.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d3b7fe700 1 -- 172.21.15.156:0/3272548205 learned_addr learned my addr 172.21.15.156:0/3272548205 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d3affd700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d3c0b0d80 0x7f7d3c0c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d3bfff700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d3c158890 0x7f7d3c0c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d3affd700 1 -- 172.21.15.156:0/3272548205 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d3c158890 msgr2=0x7f7d3c0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d3affd700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d3c158890 0x7f7d3c0c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d3affd700 1 -- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d3c06d4f0 msgr2=0x7f7d3c0c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d3affd700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d3c06d4f0 0x7f7d3c0c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d3affd700 1 -- 172.21.15.156:0/3272548205 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d30007040 con 0x7f7d3c0b0d80 2023-11-28T11:08:28.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f7d3b7fe700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d3c06d4f0 0x7f7d3c0c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:28.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f9b7e59c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b800ce480 0x7f9b800c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f9b7e59c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b800ce480 0x7f9b800c1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:35892/0 (socket says 172.21.15.156:35892) 2023-11-28T11:08:28.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f9b7e59c700 1 -- 172.21.15.156:0/2157980652 learned_addr learned my addr 172.21.15.156:0/2157980652 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f9e6959a700 1 -- 172.21.15.156:0/2087330721 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6400e070 con 0x7f9e6c164e60 2023-11-28T11:08:28.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f9e6959a700 1 -- 172.21.15.156:0/2087330721 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6400bb20 con 0x7f9e6c164e60 2023-11-28T11:08:28.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f9e6959a700 1 -- 172.21.15.156:0/2087330721 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64012430 con 0x7f9e6c164e60 2023-11-28T11:08:28.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.011+0000 7f335b95f700 1 Processor -- start 2023-11-28T11:08:28.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.012+0000 7fb067fff700 1 --2- 172.21.15.156:0/3267927731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0680c1fd0 0x7fb068161220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb0580098f0 tx=0x7fb058009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=313c05a5723e0824 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.012+0000 7f7d3affd700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d3c0b0d80 0x7f7d3c0c7480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c000f60 tx=0x7f7d2c00c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.012+0000 7fb0677fe700 1 -- 172.21.15.156:0/3267927731 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058011830 con 0x7fb0680c1fd0 2023-11-28T11:08:28.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.012+0000 7fb0677fe700 1 -- 172.21.15.156:0/3267927731 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb058011990 con 0x7fb0680c1fd0 2023-11-28T11:08:28.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.012+0000 7fb0677fe700 1 -- 172.21.15.156:0/3267927731 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058011d20 con 0x7fb0680c1fd0 2023-11-28T11:08:28.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.012+0000 7f335b95f700 1 -- start start 2023-11-28T11:08:28.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.012+0000 7f9e70ba5700 1 -- 172.21.15.156:0/2087330721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e6c164e60 msgr2=0x7f9e6c1652e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.012+0000 7f9e70ba5700 1 --2- 172.21.15.156:0/2087330721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e6c164e60 0x7f9e6c1652e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009d90 tx=0x7f9e6400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:28.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7fb06ec14700 1 -- 172.21.15.156:0/3267927731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0680c1fd0 msgr2=0x7fb068161220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7fb06ec14700 1 --2- 172.21.15.156:0/3267927731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0680c1fd0 0x7fb068161220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb0580098f0 tx=0x7fb058009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:28.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7fb06ec14700 1 -- 172.21.15.156:0/3267927731 shutdown_connections 2023-11-28T11:08:28.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7fb06ec14700 1 --2- 172.21.15.156:0/3267927731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0680c1fd0 0x7fb068161220 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7fb06ec14700 1 -- 172.21.15.156:0/3267927731 >> 172.21.15.156:0/3267927731 conn(0x7fb0680cb670 msgr2=0x7fb0680cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f9e70ba5700 1 -- 172.21.15.156:0/2087330721 shutdown_connections 2023-11-28T11:08:28.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f9e70ba5700 1 --2- 172.21.15.156:0/2087330721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e6c164e60 0x7f9e6c1652e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f9e70ba5700 1 --2- 172.21.15.156:0/2087330721 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e6c0ce480 0x7f9e6c0c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f9e70ba5700 1 -- 172.21.15.156:0/2087330721 >> 172.21.15.156:0/2087330721 conn(0x7f9e6c0cb670 msgr2=0x7f9e6c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f7d23fff700 1 -- 172.21.15.156:0/3272548205 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0052d0 con 0x7f7d3c0b0d80 2023-11-28T11:08:28.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f7d23fff700 1 -- 172.21.15.156:0/3272548205 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c005430 con 0x7f7d3c0b0d80 2023-11-28T11:08:28.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f7d23fff700 1 -- 172.21.15.156:0/3272548205 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0038c0 con 0x7f7d3c0b0d80 2023-11-28T11:08:28.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f7d42152700 1 -- 172.21.15.156:0/3272548205 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3c0c9370 con 0x7f7d3c0b0d80 2023-11-28T11:08:28.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f7d42152700 1 -- 172.21.15.156:0/3272548205 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c15ccf0 con 0x7f7d3c0b0d80 2023-11-28T11:08:28.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f9b7dd9b700 1 --2- 172.21.15.156:0/2157980652 >> v2:172.21.15.186:3300/0 conn(0x7f9b80161220 0x7f9b8006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f335b95f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33540b0670 0x7f33540cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f335b95f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f335406d4d0 0x7f33540b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f335b95f700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33540ccc70 con 0x7f33541603a0 2023-11-28T11:08:28.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f335b95f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33540ad6e0 con 0x7f335406d4d0 2023-11-28T11:08:28.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f335b95f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33540b0f80 con 0x7f33540b0670 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f9e70ba5700 1 -- 172.21.15.156:0/2087330721 shutdown_connections 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f9e70ba5700 1 -- 172.21.15.156:0/2087330721 wait complete. 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f3359efc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f33541603a0 0x7f33541607f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46912/0 (socket says 172.21.15.156:46912) 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.013+0000 7f3359efc700 1 -- 172.21.15.156:0/2683460890 learned_addr learned my addr 172.21.15.156:0/2683460890 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f9b7d59a700 1 -- 172.21.15.156:0/2157980652 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376546991 0 0) 0x7f9b80162210 con 0x7f9b80164e60 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f9b7d59a700 1 -- 172.21.15.156:0/2157980652 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b680035b0 con 0x7f9b80164e60 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f9b7dd9b700 1 -- 172.21.15.156:0/2157980652 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b800ce480 msgr2=0x7f9b800c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f9b7dd9b700 1 --2- 172.21.15.156:0/2157980652 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b800ce480 0x7f9b800c1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f9b7dd9b700 1 -- 172.21.15.156:0/2157980652 >> v1:172.21.15.156:6789/0 conn(0x7f9b80164e60 legacy=0x7f9b800b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f9b7dd9b700 1 -- 172.21.15.156:0/2157980652 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8006bd60 con 0x7f9b80161220 2023-11-28T11:08:28.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7fb06ec14700 1 -- 172.21.15.156:0/3267927731 shutdown_connections 2023-11-28T11:08:28.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f7d42152700 1 -- 172.21.15.156:0/3272548205 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d1c000fc0 con 0x7f7d3c0b0d80 2023-11-28T11:08:28.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f1abbfff700 1 --2- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1aa80682f0 0x7f1aa806a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400e190 tx=0x7f1ab400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f9b7dd9b700 1 --2- 172.21.15.156:0/2157980652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b80161220 0x7f9b8006b820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009840 tx=0x7f9b70009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=463c17330d3f1c62 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.015+0000 7f9b7d59a700 1 -- 172.21.15.156:0/2157980652 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70004230 con 0x7f9b80161220 2023-11-28T11:08:28.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7fb06ec14700 1 -- 172.21.15.156:0/3267927731 wait complete. 2023-11-28T11:08:28.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.015+0000 7f9b7d59a700 1 -- 172.21.15.156:0/2157980652 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b70004390 con 0x7f9b80161220 2023-11-28T11:08:28.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.015+0000 7f9b7d59a700 1 -- 172.21.15.156:0/2157980652 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b700036a0 con 0x7f9b80161220 2023-11-28T11:08:28.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.016+0000 7fb06ec14700 1 Processor -- start 2023-11-28T11:08:28.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.015+0000 7f9e70ba5700 1 Processor -- start 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f33596fb700 1 --2- 172.21.15.156:0/2683460890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33540b0670 0x7f33540cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f3358efa700 1 --2- 172.21.15.156:0/2683460890 >> v2:172.21.15.156:3300/0 conn(0x7f335406d4d0 0x7f33540b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f334bfff700 1 -- 172.21.15.156:0/2683460890 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875056062 0 0) 0x7f33540ccc70 con 0x7f33541603a0 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f334bfff700 1 -- 172.21.15.156:0/2683460890 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33400035b0 con 0x7f33541603a0 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f33596fb700 1 -- 172.21.15.156:0/2683460890 >> v2:172.21.15.156:3300/0 conn(0x7f335406d4d0 msgr2=0x7f33540b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f334bfff700 1 -- 172.21.15.156:0/2683460890 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3794197476 0 0) 0x7f33400035b0 con 0x7f33541603a0 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f33596fb700 1 --2- 172.21.15.156:0/2683460890 >> v2:172.21.15.156:3300/0 conn(0x7f335406d4d0 0x7f33540b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f33596fb700 1 -- 172.21.15.156:0/2683460890 >> v1:172.21.15.156:6789/0 conn(0x7f33541603a0 legacy=0x7f33541607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.014+0000 7f33596fb700 1 -- 172.21.15.156:0/2683460890 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33540aca60 con 0x7f33540b0670 2023-11-28T11:08:28.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.016+0000 7f9b84b0b700 1 -- 172.21.15.156:0/2157980652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b80161220 msgr2=0x7f9b8006b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.016+0000 7f9b84b0b700 1 --2- 172.21.15.156:0/2157980652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b80161220 0x7f9b8006b820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009840 tx=0x7f9b70009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:28.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f33596fb700 1 --2- 172.21.15.156:0/2683460890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33540b0670 0x7f33540cc730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3344009840 tx=0x7f3344009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=128d7f89c52ad093 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9b84b0b700 1 -- 172.21.15.156:0/2157980652 shutdown_connections 2023-11-28T11:08:28.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9b84b0b700 1 --2- 172.21.15.156:0/2157980652 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b800ce480 0x7f9b800c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9b84b0b700 1 --2- 172.21.15.156:0/2157980652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b80161220 0x7f9b8006b820 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9b84b0b700 1 -- 172.21.15.156:0/2157980652 >> 172.21.15.156:0/2157980652 conn(0x7f9b800cb670 msgr2=0x7f9b800cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.016+0000 7f9e70ba5700 1 -- start start 2023-11-28T11:08:28.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9e70ba5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e6c0ce480 0x7f9e6c0add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9e70ba5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e6c161220 0x7f9e6c0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9e70ba5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e6c164e60 0x7f9e6c154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9e70ba5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0c5c30 con 0x7f9e6c161220 2023-11-28T11:08:28.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9e70ba5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0b2e10 con 0x7f9e6c0ce480 2023-11-28T11:08:28.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9e70ba5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0b30f0 con 0x7f9e6c164e60 2023-11-28T11:08:28.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9e6ad9d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e6c164e60 0x7f9e6c154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9e6ad9d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e6c164e60 0x7f9e6c154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:35900/0 (socket says 172.21.15.156:35900) 2023-11-28T11:08:28.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f9e6ad9d700 1 -- 172.21.15.156:0/3033842034 learned_addr learned my addr 172.21.15.156:0/3033842034 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f7d23fff700 1 -- 172.21.15.156:0/3272548205 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d2c019030 con 0x7f7d3c0b0d80 2023-11-28T11:08:28.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f7d23fff700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7d280682f0 0x7f7d2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f7d23fff700 1 -- 172.21.15.156:0/3272548205 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d2c084d10 con 0x7f7d3c0b0d80 2023-11-28T11:08:28.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f7d3b7fe700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7d280682f0 0x7f7d2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f7d23fff700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x7f7d28070aa0 0x7f7d28072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f7d23fff700 1 -- 172.21.15.156:0/3272548205 --> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d28073560 con 0x7f7d28070aa0 2023-11-28T11:08:28.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f334bfff700 1 -- 172.21.15.156:0/2683460890 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c0019f0 con 0x7f33540b0670 2023-11-28T11:08:28.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7f7d3b7fe700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7d280682f0 0x7f7d2806a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d300076e0 tx=0x7f7d3000a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7f9e6ad9d700 1 -- 172.21.15.156:0/3033842034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e6c0ce480 msgr2=0x7f9e6c0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7f9e6ad9d700 1 --2- 172.21.15.156:0/3033842034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e6c0ce480 0x7f9e6c0add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7f9e6ad9d700 1 -- 172.21.15.156:0/3033842034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e6c161220 msgr2=0x7f9e6c0b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7f9e6ad9d700 1 --2- 172.21.15.156:0/3033842034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e6c161220 0x7f9e6c0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7f9e6ad9d700 1 -- 172.21.15.156:0/3033842034 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e64009a70 con 0x7f9e6c164e60 2023-11-28T11:08:28.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7f9e6ad9d700 1 --2- 172.21.15.156:0/3033842034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e6c164e60 0x7f9e6c154ce0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9e64014040 tx=0x7f9e64004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.016+0000 7fb06ec14700 1 -- start start 2023-11-28T11:08:28.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7fb06ec14700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0680c1fd0 0x7fb0680c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7fb06ec14700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0680ce480 0x7fb0680ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7fb06ec14700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb068164e60 0x7fb0681ec180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7fb06ec14700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680c2e30 con 0x7fb068164e60 2023-11-28T11:08:28.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7fb06ec14700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680af6b0 con 0x7fb0680c1fd0 2023-11-28T11:08:28.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7fb06ec14700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680af3d0 con 0x7fb0680ce480 2023-11-28T11:08:28.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7fb067fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0680ce480 0x7fb0680ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7fb067fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0680ce480 0x7fb0680ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:35906/0 (socket says 172.21.15.156:35906) 2023-11-28T11:08:28.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7fb067fff700 1 -- 172.21.15.156:0/3823981899 learned_addr learned my addr 172.21.15.156:0/3823981899 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7f7d3bfff700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x7f7d28070aa0 0x7f7d28072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7fb067fff700 1 -- 172.21.15.156:0/3823981899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0680c1fd0 msgr2=0x7fb0680c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7fb067fff700 1 --2- 172.21.15.156:0/3823981899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0680c1fd0 0x7fb0680c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7fb067fff700 1 -- 172.21.15.156:0/3823981899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb068164e60 msgr2=0x7fb0681ec180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7fb067fff700 1 --2- 172.21.15.156:0/3823981899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb068164e60 0x7fb0681ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7fb067fff700 1 -- 172.21.15.156:0/3823981899 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb058009550 con 0x7fb0680ce480 2023-11-28T11:08:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7f7d3bfff700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x7f7d28070aa0 0x7f7d28072eb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7fb067fff700 1 --2- 172.21.15.156:0/3823981899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0680ce480 0x7fb0680ae9f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb058009c40 tx=0x7fb058004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f334bfff700 1 -- 172.21.15.156:0/2683460890 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3344004030 con 0x7f33540b0670 2023-11-28T11:08:28.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.017+0000 7f334bfff700 1 -- 172.21.15.156:0/2683460890 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33440043a0 con 0x7f33540b0670 2023-11-28T11:08:28.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7f335b95f700 1 -- 172.21.15.156:0/2683460890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33540b0670 msgr2=0x7f33540cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.018+0000 7f335b95f700 1 --2- 172.21.15.156:0/2683460890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33540b0670 0x7f33540cc730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3344009840 tx=0x7f3344009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:28.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.019+0000 7f335b95f700 1 -- 172.21.15.156:0/2683460890 shutdown_connections 2023-11-28T11:08:28.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.019+0000 7f335b95f700 1 --2- 172.21.15.156:0/2683460890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33540b0670 0x7f33540cc730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.019+0000 7f335b95f700 1 --2- 172.21.15.156:0/2683460890 >> v2:172.21.15.156:3300/0 conn(0x7f335406d4d0 0x7f33540b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.019+0000 7f335b95f700 1 -- 172.21.15.156:0/2683460890 >> 172.21.15.156:0/2683460890 conn(0x7f33540cb670 msgr2=0x7f33540cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.019+0000 7f7d23fff700 1 -- 172.21.15.156:0/3272548205 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d2c001140 con 0x7f7d3c0b0d80 2023-11-28T11:08:28.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.019+0000 7fb0657fa700 1 -- 172.21.15.156:0/3823981899 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0580036a0 con 0x7fb0680ce480 2023-11-28T11:08:28.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.019+0000 7fb0657fa700 1 -- 172.21.15.156:0/3823981899 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05801e070 con 0x7fb0680ce480 2023-11-28T11:08:28.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.019+0000 7fb0657fa700 1 -- 172.21.15.156:0/3823981899 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058003a00 con 0x7fb0680ce480 2023-11-28T11:08:28.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.019+0000 7f335b95f700 1 -- 172.21.15.156:0/2683460890 shutdown_connections 2023-11-28T11:08:28.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.020+0000 7f7d23fff700 1 -- 172.21.15.156:0/3272548205 <== osd.1 v2:172.21.15.156:6820/103566 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7d28073560 con 0x7f7d28070aa0 2023-11-28T11:08:28.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.020+0000 7f9e5affd700 1 -- 172.21.15.156:0/3033842034 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e640043f0 con 0x7f9e6c164e60 2023-11-28T11:08:28.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.020+0000 7f9e5affd700 1 -- 172.21.15.156:0/3033842034 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e64021070 con 0x7f9e6c164e60 2023-11-28T11:08:28.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.020+0000 7f9e5affd700 1 -- 172.21.15.156:0/3033842034 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e640047b0 con 0x7f9e6c164e60 2023-11-28T11:08:28.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.020+0000 7f9e70ba5700 1 -- 172.21.15.156:0/3033842034 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6c161f30 con 0x7f9e6c164e60 2023-11-28T11:08:28.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.020+0000 7f9e70ba5700 1 -- 172.21.15.156:0/3033842034 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c161c50 con 0x7f9e6c164e60 2023-11-28T11:08:28.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.019+0000 7f335b95f700 1 -- 172.21.15.156:0/2683460890 wait complete. 2023-11-28T11:08:28.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.021+0000 7f335b95f700 1 Processor -- start 2023-11-28T11:08:28.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.021+0000 7f335b95f700 1 -- start start 2023-11-28T11:08:28.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.021+0000 7f335b95f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f335406d4d0 0x7f33540c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.021+0000 7f335b95f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33540b0670 0x7f335415ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.021+0000 7f335b95f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33541603a0 0x7f33541feb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.021+0000 7f335b95f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33540c3960 con 0x7f335406d4d0 2023-11-28T11:08:28.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.021+0000 7f335b95f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33540c3680 con 0x7f33540b0670 2023-11-28T11:08:28.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.021+0000 7f335b95f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335415cb90 con 0x7f33541603a0 2023-11-28T11:08:28.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f33596fb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f335406d4d0 0x7f33540c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f33596fb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f335406d4d0 0x7f33540c30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60284/0 (socket says 172.21.15.156:60284) 2023-11-28T11:08:28.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f33596fb700 1 -- 172.21.15.156:0/1873244604 learned_addr learned my addr 172.21.15.156:0/1873244604 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f3358efa700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33540b0670 0x7f335415ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.019+0000 7fb06ec14700 1 -- 172.21.15.156:0/3823981899 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb068161970 con 0x7fb0680ce480 2023-11-28T11:08:28.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.019+0000 7fb06ec14700 1 -- 172.21.15.156:0/3823981899 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06806c020 con 0x7fb0680ce480 2023-11-28T11:08:28.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.020+0000 7fb06ec14700 1 -- 172.21.15.156:0/3823981899 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb068161c50 con 0x7fb0680ce480 2023-11-28T11:08:28.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f33596fb700 1 -- 172.21.15.156:0/1873244604 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33541603a0 msgr2=0x7f33541feb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f33596fb700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33541603a0 0x7f33541feb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f33596fb700 1 -- 172.21.15.156:0/1873244604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33540b0670 msgr2=0x7f335415ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f33596fb700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33540b0670 0x7f335415ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f33596fb700 1 -- 172.21.15.156:0/1873244604 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33440094a0 con 0x7f335406d4d0 2023-11-28T11:08:28.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f9e5affd700 1 -- 172.21.15.156:0/3033842034 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9e6402c020 con 0x7f9e6c164e60 2023-11-28T11:08:28.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f33596fb700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f335406d4d0 0x7f33540c30a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f33440090b0 tx=0x7f3344004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f9e70ba5700 1 -- 172.21.15.156:0/3033842034 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9e6c161690 con 0x7f9e6c164e60 2023-11-28T11:08:28.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.023+0000 7f9e5affd700 1 --2- 172.21.15.156:0/3033842034 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9e540684a0 0x7f9e5406a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.023+0000 7f9e5affd700 1 -- 172.21.15.156:0/3033842034 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e64087c20 con 0x7f9e6c164e60 2023-11-28T11:08:28.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.023+0000 7f9e5affd700 1 --2- 172.21.15.156:0/3033842034 >> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] conn(0x7f9e54070cd0 0x7f9e540730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.023+0000 7f9e5affd700 1 -- 172.21.15.156:0/3033842034 --> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e54073790 con 0x7f9e54070cd0 2023-11-28T11:08:28.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.023+0000 7f9e5affd700 1 -- 172.21.15.156:0/3033842034 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e64054350 con 0x7f9e6c164e60 2023-11-28T11:08:28.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f3349ffb700 1 -- 172.21.15.156:0/1873244604 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344003df0 con 0x7f335406d4d0 2023-11-28T11:08:28.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f3349ffb700 1 -- 172.21.15.156:0/1873244604 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334401f070 con 0x7f335406d4d0 2023-11-28T11:08:28.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f3349ffb700 1 -- 172.21.15.156:0/1873244604 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344011870 con 0x7f335406d4d0 2023-11-28T11:08:28.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f335b95f700 1 -- 172.21.15.156:0/1873244604 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33540ad020 con 0x7f335406d4d0 2023-11-28T11:08:28.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.023+0000 7f335b95f700 1 -- 172.21.15.156:0/1873244604 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33540acd40 con 0x7f335406d4d0 2023-11-28T11:08:28.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f9b84b0b700 1 -- 172.21.15.156:0/2157980652 shutdown_connections 2023-11-28T11:08:28.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.022+0000 7f9b84b0b700 1 -- 172.21.15.156:0/2157980652 wait complete. 2023-11-28T11:08:28.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.024+0000 7f9e6a59c700 1 --2- 172.21.15.156:0/3033842034 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9e540684a0 0x7f9e5406a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.024+0000 7f9e69d9b700 1 --2- 172.21.15.156:0/3033842034 >> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] conn(0x7f9e54070cd0 0x7f9e540730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.024+0000 7f335b95f700 1 -- 172.21.15.156:0/1873244604 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3334000fc0 con 0x7f335406d4d0 2023-11-28T11:08:28.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.024+0000 7f9e6a59c700 1 --2- 172.21.15.156:0/3033842034 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9e540684a0 0x7f9e5406a950 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c006b20 tx=0x7f9e5c006ab0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.024+0000 7f9e69d9b700 1 --2- 172.21.15.156:0/3033842034 >> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] conn(0x7f9e54070cd0 0x7f9e540730e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.025+0000 7f9b84b0b700 1 Processor -- start 2023-11-28T11:08:28.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.025+0000 7f9b84b0b700 1 -- start start 2023-11-28T11:08:28.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.025+0000 7f9b84b0b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b800ce480 0x7f9b800b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.025+0000 7f9b84b0b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b80161220 0x7f9b80151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.025+0000 7f9b84b0b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b80164e60 0x7f9b8014e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.025+0000 7f9b84b0b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b800af020 con 0x7f9b80164e60 2023-11-28T11:08:28.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.025+0000 7f9b84b0b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b800af300 con 0x7f9b800ce480 2023-11-28T11:08:28.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.025+0000 7f9b84b0b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b800af5e0 con 0x7f9b80161220 2023-11-28T11:08:28.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.025+0000 7f9b7e59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b800ce480 0x7f9b800b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.026+0000 7f9b7dd9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b80161220 0x7f9b80151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.026+0000 7f9b7e59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b800ce480 0x7f9b800b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:43076/0 (socket says 172.21.15.156:43076) 2023-11-28T11:08:28.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.026+0000 7f9b7e59c700 1 -- 172.21.15.156:0/572590795 learned_addr learned my addr 172.21.15.156:0/572590795 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.026+0000 7f9b7ed9d700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b80164e60 0x7f9b8014e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.026+0000 7f9b7dd9b700 1 -- 172.21.15.156:0/572590795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b800ce480 msgr2=0x7f9b800b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.026+0000 7f9b7dd9b700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b800ce480 0x7f9b800b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.026+0000 7f9b7dd9b700 1 -- 172.21.15.156:0/572590795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b80164e60 msgr2=0x7f9b8014e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.026+0000 7f9b7dd9b700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b80164e60 0x7f9b8014e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.026+0000 7f9b7dd9b700 1 -- 172.21.15.156:0/572590795 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b700094a0 con 0x7f9b80161220 2023-11-28T11:08:28.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.026+0000 7f9b7dd9b700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b80161220 0x7f9b80151650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9b70013000 tx=0x7f9b70004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.027+0000 7f9b6effd700 1 -- 172.21.15.156:0/572590795 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70003c70 con 0x7f9b80161220 2023-11-28T11:08:28.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.027+0000 7f9b6effd700 1 -- 172.21.15.156:0/572590795 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b70003df0 con 0x7f9b80161220 2023-11-28T11:08:28.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.027+0000 7f9b84b0b700 1 -- 172.21.15.156:0/572590795 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b80161c50 con 0x7f9b80161220 2023-11-28T11:08:28.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.027+0000 7f9b6effd700 1 -- 172.21.15.156:0/572590795 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70011870 con 0x7f9b80161220 2023-11-28T11:08:28.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.027+0000 7f9b84b0b700 1 -- 172.21.15.156:0/572590795 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b80161970 con 0x7f9b80161220 2023-11-28T11:08:28.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.028+0000 7f9b84b0b700 1 -- 172.21.15.156:0/572590795 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9b80161f30 con 0x7f9b80161220 2023-11-28T11:08:28.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.028+0000 7fb0657fa700 1 -- 172.21.15.156:0/3823981899 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb05800f050 con 0x7fb0680ce480 2023-11-28T11:08:28.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.028+0000 7fb0657fa700 1 --2- 172.21.15.156:0/3823981899 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb0540682f0 0x7fb05406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.028+0000 7fb0657fa700 1 -- 172.21.15.156:0/3823981899 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0580872a0 con 0x7fb0680ce480 2023-11-28T11:08:28.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.028+0000 7fb06c9b0700 1 --2- 172.21.15.156:0/3823981899 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb0540682f0 0x7fb05406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.028+0000 7fb0657fa700 1 --2- 172.21.15.156:0/3823981899 >> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] conn(0x7fb054070b20 0x7fb054072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.028+0000 7fb0657fa700 1 -- 172.21.15.156:0/3823981899 --> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0540735e0 con 0x7fb054070b20 2023-11-28T11:08:28.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.028+0000 7fb0657fa700 1 -- 172.21.15.156:0/3823981899 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0580876a0 con 0x7fb0680ce480 2023-11-28T11:08:28.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.028+0000 7fb06d1b1700 1 --2- 172.21.15.156:0/3823981899 >> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] conn(0x7fb054070b20 0x7fb054072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.029+0000 7fb06c9b0700 1 --2- 172.21.15.156:0/3823981899 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb0540682f0 0x7fb05406a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb05c007fd0 tx=0x7fb05c0079a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.029+0000 7fb06d1b1700 1 --2- 172.21.15.156:0/3823981899 >> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] conn(0x7fb054070b20 0x7fb054072f30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.030+0000 7fb0657fa700 1 -- 172.21.15.156:0/3823981899 <== osd.2 v2:172.21.15.156:6800/103564 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb0540735e0 con 0x7fb054070b20 2023-11-28T11:08:28.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.030+0000 7f9e5affd700 1 -- 172.21.15.156:0/3033842034 <== osd.3 v2:172.21.15.156:6816/103567 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9e54073790 con 0x7f9e54070cd0 2023-11-28T11:08:28.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.031+0000 7f3349ffb700 1 -- 172.21.15.156:0/1873244604 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f334402a020 con 0x7f335406d4d0 2023-11-28T11:08:28.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.032+0000 7f3349ffb700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3338068340 0x7f333806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.032+0000 7f3349ffb700 1 -- 172.21.15.156:0/1873244604 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f33440876a0 con 0x7f335406d4d0 2023-11-28T11:08:28.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.032+0000 7f3349ffb700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x7f3338070bf0 0x7f3338073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.032+0000 7f3349ffb700 1 -- 172.21.15.156:0/1873244604 --> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33380736b0 con 0x7f3338070bf0 2023-11-28T11:08:28.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.032+0000 7f3349ffb700 1 -- 172.21.15.156:0/1873244604 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3344087b20 con 0x7f335406d4d0 2023-11-28T11:08:28.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.032+0000 7f3358efa700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3338068340 0x7f333806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.032+0000 7f3359efc700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x7f3338070bf0 0x7f3338073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.033+0000 7f3359efc700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x7f3338070bf0 0x7f3338073000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.033+0000 7f3358efa700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3338068340 0x7f333806a7f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f335000b9b0 tx=0x7f3350008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.034+0000 7f96e085f700 1 Processor -- start 2023-11-28T11:08:28.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.034+0000 7f3349ffb700 1 -- 172.21.15.156:0/1873244604 <== osd.5 v2:172.21.15.186:6808/103778 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f33380736b0 con 0x7f3338070bf0 2023-11-28T11:08:28.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.034+0000 7f96e085f700 1 -- start start 2023-11-28T11:08:28.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.035+0000 7f96e085f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc158890 0x7f96dc0ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.035+0000 7f96e085f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f96dc06d4f0 0x7f96dc0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.035+0000 7f96e085f700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96dc15c490 con 0x7f96dc0b0bb0 2023-11-28T11:08:28.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.035+0000 7f96e085f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96dc0b1030 con 0x7f96dc06d4f0 2023-11-28T11:08:28.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.035+0000 7f96e085f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0ce650 con 0x7f96dc158890 2023-11-28T11:08:28.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.035+0000 7f96d9d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f96dc06d4f0 0x7f96dc0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.035+0000 7f96d9d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f96dc06d4f0 0x7f96dc0b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60310/0 (socket says 172.21.15.156:60310) 2023-11-28T11:08:28.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.035+0000 7f96d9d9b700 1 -- 172.21.15.156:0/3610369186 learned_addr learned my addr 172.21.15.156:0/3610369186 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.036+0000 7f96da59c700 1 --2- 172.21.15.156:0/3610369186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc158890 0x7f96dc0ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.036+0000 7f96da59c700 1 -- 172.21.15.156:0/3610369186 >> v2:172.21.15.156:3300/0 conn(0x7f96dc06d4f0 msgr2=0x7f96dc0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.036+0000 7f96da59c700 1 --2- 172.21.15.156:0/3610369186 >> v2:172.21.15.156:3300/0 conn(0x7f96dc06d4f0 0x7f96dc0b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.036+0000 7f96da59c700 1 -- 172.21.15.156:0/3610369186 >> v1:172.21.15.186:6789/0 conn(0x7f96dc0b0bb0 legacy=0x7f96dc0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.036+0000 7f96da59c700 1 -- 172.21.15.156:0/3610369186 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc15c730 con 0x7f96dc158890 2023-11-28T11:08:28.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.036+0000 7f96da59c700 1 --2- 172.21.15.156:0/3610369186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc158890 0x7f96dc0ce110 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009d90 tx=0x7f96d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7fb0fbea7bd35b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.037+0000 7f96d959a700 1 -- 172.21.15.156:0/3610369186 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d400e070 con 0x7f96dc158890 2023-11-28T11:08:28.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.037+0000 7f96d959a700 1 -- 172.21.15.156:0/3610369186 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d4003a20 con 0x7f96dc158890 2023-11-28T11:08:28.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.037+0000 7f96d959a700 1 -- 172.21.15.156:0/3610369186 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4003dd0 con 0x7f96dc158890 2023-11-28T11:08:28.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.037+0000 7f96e085f700 1 -- 172.21.15.156:0/3610369186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc158890 msgr2=0x7f96dc0ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.037+0000 7f96e085f700 1 --2- 172.21.15.156:0/3610369186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc158890 0x7f96dc0ce110 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009d90 tx=0x7f96d400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:28.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.038+0000 7f96e085f700 1 -- 172.21.15.156:0/3610369186 shutdown_connections 2023-11-28T11:08:28.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.038+0000 7f96e085f700 1 --2- 172.21.15.156:0/3610369186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc158890 0x7f96dc0ce110 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.038+0000 7f96e085f700 1 --2- 172.21.15.156:0/3610369186 >> v2:172.21.15.156:3300/0 conn(0x7f96dc06d4f0 0x7f96dc0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.038+0000 7f96e085f700 1 -- 172.21.15.156:0/3610369186 >> 172.21.15.156:0/3610369186 conn(0x7f96dc0cb670 msgr2=0x7f96dc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.038+0000 7f9b6effd700 1 -- 172.21.15.156:0/572590795 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9b7002d070 con 0x7f9b80161220 2023-11-28T11:08:28.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.038+0000 7f9b6effd700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9b60068340 0x7f9b6006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.038+0000 7f9b6effd700 1 -- 172.21.15.156:0/572590795 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9b70086e10 con 0x7f9b80161220 2023-11-28T11:08:28.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.038+0000 7f9b6effd700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x7f9b60070bf0 0x7f9b60073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.038+0000 7f9b6effd700 1 -- 172.21.15.156:0/572590795 --> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b600736b0 con 0x7f9b60070bf0 2023-11-28T11:08:28.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.038+0000 7f9b6effd700 1 -- 172.21.15.156:0/572590795 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b70087240 con 0x7f9b80161220 2023-11-28T11:08:28.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.038+0000 7f9b7e59c700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9b60068340 0x7f9b6006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.039+0000 7f9b7e59c700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9b60068340 0x7f9b6006a7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9b800b08e0 tx=0x7f9b74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.040+0000 7f96e085f700 1 -- 172.21.15.156:0/3610369186 shutdown_connections 2023-11-28T11:08:28.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.040+0000 7f96e085f700 1 -- 172.21.15.156:0/3610369186 wait complete. 2023-11-28T11:08:28.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.041+0000 7f96e085f700 1 Processor -- start 2023-11-28T11:08:28.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.041+0000 7f96e085f700 1 -- start start 2023-11-28T11:08:28.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.042+0000 7f96e085f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96dc0b0bb0 0x7f96dc150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.042+0000 7f96e085f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc158890 0x7f96dc1e28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.042+0000 7f96e085f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc1e2e20 0x7f96dc15a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.042+0000 7f96e085f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1e3e70 con 0x7f96dc0b0bb0 2023-11-28T11:08:28.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.042+0000 7f96e085f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc150650 con 0x7f96dc158890 2023-11-28T11:08:28.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.042+0000 7f96e085f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1e32f0 con 0x7f96dc1e2e20 2023-11-28T11:08:28.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.042+0000 7f96da59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96dc0b0bb0 0x7f96dc150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.043+0000 7f96da59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96dc0b0bb0 0x7f96dc150110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60318/0 (socket says 172.21.15.156:60318) 2023-11-28T11:08:28.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.043+0000 7f96da59c700 1 -- 172.21.15.156:0/1018973743 learned_addr learned my addr 172.21.15.156:0/1018973743 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.043+0000 7f96dad9d700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc1e2e20 0x7f96dc15a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.043+0000 7f96d9d9b700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc158890 0x7f96dc1e28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.043+0000 7f96da59c700 1 -- 172.21.15.156:0/1018973743 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc1e2e20 msgr2=0x7f96dc15a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.043+0000 7f96da59c700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc1e2e20 0x7f96dc15a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.043+0000 7f96da59c700 1 -- 172.21.15.156:0/1018973743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc158890 msgr2=0x7f96dc1e28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.043+0000 7f96da59c700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc158890 0x7f96dc1e28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.043+0000 7f96da59c700 1 -- 172.21.15.156:0/1018973743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d4009a70 con 0x7f96dc0b0bb0 2023-11-28T11:08:28.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.043+0000 7f96d9d9b700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc158890 0x7f96dc1e28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:28.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.043+0000 7f96da59c700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96dc0b0bb0 0x7f96dc150110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f96d400b370 tx=0x7f96d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.044+0000 7f96caffd700 1 -- 172.21.15.156:0/1018973743 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d40044c0 con 0x7f96dc0b0bb0 2023-11-28T11:08:28.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.044+0000 7f96caffd700 1 -- 172.21.15.156:0/1018973743 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d4004620 con 0x7f96dc0b0bb0 2023-11-28T11:08:28.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.044+0000 7f96caffd700 1 -- 172.21.15.156:0/1018973743 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d401e430 con 0x7f96dc0b0bb0 2023-11-28T11:08:28.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.044+0000 7f9b7ed9d700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x7f9b60070bf0 0x7f9b60073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.044+0000 7f1ac22d3700 1 -- 172.21.15.156:0/2500768150 --> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1abc071f30 con 0x7f1aa8070b20 2023-11-28T11:08:28.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.044+0000 7f9b7ed9d700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x7f9b60070bf0 0x7f9b60073000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.044+0000 7fea239cd700 1 Processor -- start 2023-11-28T11:08:28.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.045+0000 7f96e085f700 1 -- 172.21.15.156:0/1018973743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc15ccf0 con 0x7f96dc0b0bb0 2023-11-28T11:08:28.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.045+0000 7f96e085f700 1 -- 172.21.15.156:0/1018973743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc15ca10 con 0x7f96dc0b0bb0 2023-11-28T11:08:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.046+0000 7f9b6effd700 1 -- 172.21.15.156:0/572590795 <== osd.6 v2:172.21.15.186:6800/103779 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9b600736b0 con 0x7f9b60070bf0 2023-11-28T11:08:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.045+0000 7fea239cd700 1 -- start start 2023-11-28T11:08:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.046+0000 7fea239cd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fea1c158890 0x7fea1c0ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.046+0000 7fea239cd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fea1c0b0d80 0x7fea1c06b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.046+0000 7fea239cd700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1c15c490 con 0x7fea1c06d4f0 2023-11-28T11:08:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.046+0000 7fea239cd700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c0ce650 con 0x7fea1c158890 2023-11-28T11:08:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.046+0000 7fea239cd700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c15c730 con 0x7fea1c0b0d80 2023-11-28T11:08:28.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.047+0000 7f1ab8ff9700 1 -- 172.21.15.156:0/2500768150 <== osd.0 v2:172.21.15.156:6802/103563 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1abc071f30 con 0x7f1aa8070b20 2023-11-28T11:08:28.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.046+0000 7f96caffd700 1 -- 172.21.15.156:0/1018973743 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f96d401e590 con 0x7f96dc0b0bb0 2023-11-28T11:08:28.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.047+0000 7f96e085f700 1 -- 172.21.15.156:0/1018973743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96dc0c9370 con 0x7f96dc0b0bb0 2023-11-28T11:08:28.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.047+0000 7fea21769700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fea1c158890 0x7fea1c0ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.047+0000 7fea21769700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fea1c158890 0x7fea1c0ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60324/0 (socket says 172.21.15.156:60324) 2023-11-28T11:08:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.047+0000 7fea21769700 1 -- 172.21.15.156:0/1803020967 learned_addr learned my addr 172.21.15.156:0/1803020967 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.047+0000 7f96caffd700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f96bc068310 0x7f96bc06a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.047+0000 7f96caffd700 1 -- 172.21.15.156:0/1018973743 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f96d4088460 con 0x7f96dc0b0bb0 2023-11-28T11:08:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.047+0000 7f96caffd700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] conn(0x7f96bc070b40 0x7f96bc072f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.047+0000 7f96caffd700 1 -- 172.21.15.156:0/1018973743 --> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f96bc073600 con 0x7f96bc070b40 2023-11-28T11:08:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.048+0000 7fea21f6a700 1 --2- 172.21.15.156:0/1803020967 >> v2:172.21.15.186:3300/0 conn(0x7fea1c0b0d80 0x7fea1c06b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.048+0000 7fea21769700 1 -- 172.21.15.156:0/1803020967 >> v2:172.21.15.186:3300/0 conn(0x7fea1c0b0d80 msgr2=0x7fea1c06b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.048+0000 7fea21769700 1 --2- 172.21.15.156:0/1803020967 >> v2:172.21.15.186:3300/0 conn(0x7fea1c0b0d80 0x7fea1c06b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.048+0000 7fea21769700 1 -- 172.21.15.156:0/1803020967 >> v1:172.21.15.156:6789/0 conn(0x7fea1c06d4f0 legacy=0x7fea1c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.048+0000 7fea21769700 1 -- 172.21.15.156:0/1803020967 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c15ca10 con 0x7fea1c158890 2023-11-28T11:08:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.048+0000 7f96d9d9b700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f96bc068310 0x7f96bc06a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.048+0000 7fea21769700 1 --2- 172.21.15.156:0/1803020967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c158890 0x7fea1c0ce110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0ca0d0 tx=0x7fea1800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1111618553edd440 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.048+0000 7f96d9d9b700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f96bc068310 0x7f96bc06a7c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f96dc157f30 tx=0x7f96cc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.049+0000 7f96dad9d700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] conn(0x7f96bc070b40 0x7f96bc072f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.049+0000 7fea13fff700 1 -- 172.21.15.156:0/1803020967 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18010070 con 0x7fea1c158890 2023-11-28T11:08:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.049+0000 7fea13fff700 1 -- 172.21.15.156:0/1803020967 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1800da80 con 0x7fea1c158890 2023-11-28T11:08:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.049+0000 7fea13fff700 1 -- 172.21.15.156:0/1803020967 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800de10 con 0x7fea1c158890 2023-11-28T11:08:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.050+0000 7fea239cd700 1 -- 172.21.15.156:0/1803020967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c158890 msgr2=0x7fea1c0ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.050+0000 7fea239cd700 1 --2- 172.21.15.156:0/1803020967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c158890 0x7fea1c0ce110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0ca0d0 tx=0x7fea1800d330 comp rx=0 tx=0).stop 2023-11-28T11:08:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.050+0000 7fea239cd700 1 -- 172.21.15.156:0/1803020967 shutdown_connections 2023-11-28T11:08:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.050+0000 7fea239cd700 1 --2- 172.21.15.156:0/1803020967 >> v2:172.21.15.186:3300/0 conn(0x7fea1c0b0d80 0x7fea1c06b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.050+0000 7fea239cd700 1 --2- 172.21.15.156:0/1803020967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c158890 0x7fea1c0ce110 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.050+0000 7fea239cd700 1 -- 172.21.15.156:0/1803020967 >> 172.21.15.156:0/1803020967 conn(0x7fea1c0cb670 msgr2=0x7fea1c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.050+0000 7f96dad9d700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] conn(0x7f96bc070b40 0x7f96bc072f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.050+0000 7f96caffd700 1 -- 172.21.15.156:0/1018973743 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96d4054d60 con 0x7f96dc0b0bb0 2023-11-28T11:08:28.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.050+0000 7fea239cd700 1 -- 172.21.15.156:0/1803020967 shutdown_connections 2023-11-28T11:08:28.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.050+0000 7fea239cd700 1 -- 172.21.15.156:0/1803020967 wait complete. 2023-11-28T11:08:28.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.051+0000 7f96caffd700 1 -- 172.21.15.156:0/1018973743 <== osd.4 v2:172.21.15.186:6824/103781 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f96bc073600 con 0x7f96bc070b40 2023-11-28T11:08:28.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.052+0000 7fea239cd700 1 Processor -- start 2023-11-28T11:08:28.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.053+0000 7f7d42152700 1 -- 172.21.15.156:0/3272548205 --> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7d1c002d40 con 0x7f7d28070aa0 2023-11-28T11:08:28.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.054+0000 7f7d23fff700 1 -- 172.21.15.156:0/3272548205 <== osd.1 v2:172.21.15.156:6820/103566 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7d1c002d40 con 0x7f7d28070aa0 2023-11-28T11:08:28.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.054+0000 7f7d42152700 1 -- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x7f7d28070aa0 msgr2=0x7f7d28072eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.054+0000 7f7d42152700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x7f7d28070aa0 0x7f7d28072eb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.054+0000 7f7d42152700 1 -- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7d280682f0 msgr2=0x7f7d2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.054+0000 7f7d42152700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7d280682f0 0x7f7d2806a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d300076e0 tx=0x7f7d3000a000 comp rx=0 tx=0).stop 2023-11-28T11:08:28.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.054+0000 7f7d42152700 1 -- 172.21.15.156:0/3272548205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d3c0b0d80 msgr2=0x7f7d3c0c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.054+0000 7f7d42152700 1 --2- 172.21.15.156:0/3272548205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d3c0b0d80 0x7f7d3c0c7480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c000f60 tx=0x7f7d2c00c6e0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.054+0000 7f7d3bfff700 1 -- 172.21.15.156:0/3272548205 reap_dead start 2023-11-28T11:08:28.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.053+0000 7fea239cd700 1 -- start start 2023-11-28T11:08:28.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.054+0000 7fea239cd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea1c06d4f0 0x7fea1c1f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.054+0000 7fea239cd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea1c0b0d80 0x7fea1c153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.054+0000 7fea239cd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c158890 0x7fea1c153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.054+0000 7fea239cd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c15a7e0 con 0x7fea1c158890 2023-11-28T11:08:28.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea239cd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c1f7990 con 0x7fea1c0b0d80 2023-11-28T11:08:28.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea239cd700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c159ed0 con 0x7fea1c06d4f0 2023-11-28T11:08:28.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7f7d42152700 1 -- 172.21.15.156:0/3272548205 shutdown_connections 2023-11-28T11:08:28.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7f7d42152700 1 -- 172.21.15.156:0/3272548205 >> 172.21.15.156:0/3272548205 conn(0x7f7d3c0cb670 msgr2=0x7f7d3c070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea21769700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea1c06d4f0 0x7fea1c1f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea21f6a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c158890 0x7fea1c153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea21769700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea1c06d4f0 0x7fea1c1f06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:35966/0 (socket says 172.21.15.156:35966) 2023-11-28T11:08:28.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea21769700 1 -- 172.21.15.156:0/2259240176 learned_addr learned my addr 172.21.15.156:0/2259240176 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea21769700 1 -- 172.21.15.156:0/2259240176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea1c0b0d80 msgr2=0x7fea1c153210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:28.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea21769700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea1c0b0d80 0x7fea1c153210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea21769700 1 -- 172.21.15.156:0/2259240176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c158890 msgr2=0x7fea1c153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea21769700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c158890 0x7fea1c153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea21769700 1 -- 172.21.15.156:0/2259240176 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea180085d0 con 0x7fea1c06d4f0 2023-11-28T11:08:28.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea21f6a700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1c158890 0x7fea1c153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:28.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.055+0000 7fea21769700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea1c06d4f0 0x7fea1c1f06a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fea18000c00 tx=0x7fea18004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.056+0000 7fea11ffb700 1 -- 172.21.15.156:0/2259240176 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18008e20 con 0x7fea1c06d4f0 2023-11-28T11:08:28.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.056+0000 7fea11ffb700 1 -- 172.21.15.156:0/2259240176 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea18003720 con 0x7fea1c06d4f0 2023-11-28T11:08:28.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.056+0000 7fea11ffb700 1 -- 172.21.15.156:0/2259240176 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18003a90 con 0x7fea1c06d4f0 2023-11-28T11:08:28.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.056+0000 7fea239cd700 1 -- 172.21.15.156:0/2259240176 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1c0c9370 con 0x7fea1c06d4f0 2023-11-28T11:08:28.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.056+0000 7fea239cd700 1 -- 172.21.15.156:0/2259240176 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c15ccf0 con 0x7fea1c06d4f0 2023-11-28T11:08:28.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.057+0000 7fea239cd700 1 -- 172.21.15.156:0/2259240176 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea1c158c60 con 0x7fea1c06d4f0 2023-11-28T11:08:28.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.057+0000 7fea11ffb700 1 -- 172.21.15.156:0/2259240176 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea18014070 con 0x7fea1c06d4f0 2023-11-28T11:08:28.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.058+0000 7fea11ffb700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea00068340 0x7fea0006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.058+0000 7fea20f68700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea00068340 0x7fea0006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.058+0000 7fea11ffb700 1 -- 172.21.15.156:0/2259240176 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea180871b0 con 0x7fea1c06d4f0 2023-11-28T11:08:28.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.058+0000 7fea11ffb700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x7fea00070b70 0x7fea00072f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.058+0000 7fea11ffb700 1 -- 172.21.15.156:0/2259240176 --> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea00073630 con 0x7fea00070b70 2023-11-28T11:08:28.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.058+0000 7fea11ffb700 1 -- 172.21.15.156:0/2259240176 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea18089790 con 0x7fea1c06d4f0 2023-11-28T11:08:28.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.058+0000 7fea21f6a700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x7fea00070b70 0x7fea00072f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.059+0000 7fea20f68700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea00068340 0x7fea0006a7f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000f60 tx=0x7fea0c007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.059+0000 7fea21f6a700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x7fea00070b70 0x7fea00072f80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.059+0000 7f7d42152700 1 -- 172.21.15.156:0/3272548205 shutdown_connections 2023-11-28T11:08:28.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.059+0000 7fb06ec14700 1 -- 172.21.15.156:0/3823981899 --> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb0680c2640 con 0x7fb054070b20 2023-11-28T11:08:28.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.059+0000 7f7d42152700 1 -- 172.21.15.156:0/3272548205 wait complete. 2023-11-28T11:08:28.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.059+0000 7fb0657fa700 1 -- 172.21.15.156:0/3823981899 <== osd.2 v2:172.21.15.156:6800/103564 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb0680c2640 con 0x7fb054070b20 2023-11-28T11:08:28.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.059+0000 7f9e70ba5700 1 -- 172.21.15.156:0/3033842034 --> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9e6c0b12d0 con 0x7f9e54070cd0 2023-11-28T11:08:28.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.060+0000 7f9e5affd700 1 -- 172.21.15.156:0/3033842034 <== osd.3 v2:172.21.15.156:6816/103567 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9e6c0b12d0 con 0x7f9e54070cd0 2023-11-28T11:08:28.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.060+0000 7fea11ffb700 1 -- 172.21.15.156:0/2259240176 <== osd.7 v2:172.21.15.186:6816/103780 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fea00073630 con 0x7fea00070b70 2023-11-28T11:08:28.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.060+0000 7f9e70ba5700 1 -- 172.21.15.156:0/3033842034 >> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] conn(0x7f9e54070cd0 msgr2=0x7f9e540730e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.060+0000 7f9e70ba5700 1 --2- 172.21.15.156:0/3033842034 >> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] conn(0x7f9e54070cd0 0x7f9e540730e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.061+0000 7f9e70ba5700 1 -- 172.21.15.156:0/3033842034 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9e540684a0 msgr2=0x7f9e5406a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.061+0000 7f9e70ba5700 1 --2- 172.21.15.156:0/3033842034 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9e540684a0 0x7f9e5406a950 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c006b20 tx=0x7f9e5c006ab0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.061+0000 7f9e70ba5700 1 -- 172.21.15.156:0/3033842034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e6c164e60 msgr2=0x7f9e6c154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.061+0000 7f9e70ba5700 1 --2- 172.21.15.156:0/3033842034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e6c164e60 0x7f9e6c154ce0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9e64014040 tx=0x7f9e64004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.061+0000 7f9e6ad9d700 1 -- 172.21.15.156:0/3033842034 reap_dead start 2023-11-28T11:08:28.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.061+0000 7f9e70ba5700 1 -- 172.21.15.156:0/3033842034 shutdown_connections 2023-11-28T11:08:28.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.061+0000 7f9e70ba5700 1 -- 172.21.15.156:0/3033842034 >> 172.21.15.156:0/3033842034 conn(0x7f9e6c0cb670 msgr2=0x7f9e6c0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.062+0000 7f1ac22d3700 1 -- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] conn(0x7f1aa8070b20 msgr2=0x7f1aa8072f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.062+0000 7f1ac22d3700 1 --2- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] conn(0x7f1aa8070b20 0x7f1aa8072f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.063+0000 7f1ac22d3700 1 -- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1aa80682f0 msgr2=0x7f1aa806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.063+0000 7f1ac22d3700 1 --2- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1aa80682f0 0x7f1aa806a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400e190 tx=0x7f1ab400b040 comp rx=0 tx=0).stop 2023-11-28T11:08:28.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.063+0000 7f1ac22d3700 1 -- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1abc160f00 msgr2=0x7f1abc0c08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.063+0000 7f1ac22d3700 1 --2- 172.21.15.156:0/2500768150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1abc160f00 0x7f1abc0c08c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009b60 tx=0x7f1aac004750 comp rx=0 tx=0).stop 2023-11-28T11:08:28.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.063+0000 7f1ac0870700 1 -- 172.21.15.156:0/2500768150 reap_dead start 2023-11-28T11:08:28.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.063+0000 7f1ac22d3700 1 -- 172.21.15.156:0/2500768150 shutdown_connections 2023-11-28T11:08:28.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.063+0000 7f1ac22d3700 1 -- 172.21.15.156:0/2500768150 >> 172.21.15.156:0/2500768150 conn(0x7f1abc0cb670 msgr2=0x7f1abc14ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.063+0000 7f1ac22d3700 1 -- 172.21.15.156:0/2500768150 shutdown_connections 2023-11-28T11:08:28.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.063+0000 7f1ac22d3700 1 -- 172.21.15.156:0/2500768150 wait complete. 2023-11-28T11:08:28.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.065+0000 7f335b95f700 1 -- 172.21.15.156:0/1873244604 --> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3334002d40 con 0x7f3338070bf0 2023-11-28T11:08:28.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.066+0000 7f3349ffb700 1 -- 172.21.15.156:0/1873244604 <== osd.5 v2:172.21.15.186:6808/103778 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3334002d40 con 0x7f3338070bf0 2023-11-28T11:08:28.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.066+0000 7f335b95f700 1 -- 172.21.15.156:0/1873244604 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x7f3338070bf0 msgr2=0x7f3338073000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.066+0000 7f335b95f700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x7f3338070bf0 0x7f3338073000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.069+0000 7f335b95f700 1 -- 172.21.15.156:0/1873244604 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3338068340 msgr2=0x7f333806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.069+0000 7f335b95f700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3338068340 0x7f333806a7f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f335000b9b0 tx=0x7f3350008040 comp rx=0 tx=0).stop 2023-11-28T11:08:28.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.069+0000 7f335b95f700 1 -- 172.21.15.156:0/1873244604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f335406d4d0 msgr2=0x7f33540c30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.069+0000 7f335b95f700 1 --2- 172.21.15.156:0/1873244604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f335406d4d0 0x7f33540c30a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f33440090b0 tx=0x7f3344004750 comp rx=0 tx=0).stop 2023-11-28T11:08:28.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.069+0000 7f3359efc700 1 -- 172.21.15.156:0/1873244604 reap_dead start 2023-11-28T11:08:28.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.071+0000 7f335b95f700 1 -- 172.21.15.156:0/1873244604 shutdown_connections 2023-11-28T11:08:28.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.071+0000 7f335b95f700 1 -- 172.21.15.156:0/1873244604 >> 172.21.15.156:0/1873244604 conn(0x7f33540cb670 msgr2=0x7f33540aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.071+0000 7f335b95f700 1 -- 172.21.15.156:0/1873244604 shutdown_connections 2023-11-28T11:08:28.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.071+0000 7f335b95f700 1 -- 172.21.15.156:0/1873244604 wait complete. 2023-11-28T11:08:28.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.072+0000 7f9e70ba5700 1 -- 172.21.15.156:0/3033842034 shutdown_connections 2023-11-28T11:08:28.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.075+0000 7f9e70ba5700 1 -- 172.21.15.156:0/3033842034 wait complete. 2023-11-28T11:08:28.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.075+0000 7f9b84b0b700 1 -- 172.21.15.156:0/572590795 --> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9b800745c0 con 0x7f9b60070bf0 2023-11-28T11:08:28.106 INFO:teuthology.orchestra.run.smithi156.stdout:55834574855 2023-11-28T11:08:28.106 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T11:08:28.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.079+0000 7f9b6effd700 1 -- 172.21.15.156:0/572590795 <== osd.6 v2:172.21.15.186:6800/103779 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9b800745c0 con 0x7f9b60070bf0 2023-11-28T11:08:28.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.079+0000 7fb06ec14700 1 -- 172.21.15.156:0/3823981899 >> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] conn(0x7fb054070b20 msgr2=0x7fb054072f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.079+0000 7fb06ec14700 1 --2- 172.21.15.156:0/3823981899 >> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] conn(0x7fb054070b20 0x7fb054072f30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.079+0000 7fb06ec14700 1 -- 172.21.15.156:0/3823981899 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb0540682f0 msgr2=0x7fb05406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.079+0000 7fb06ec14700 1 --2- 172.21.15.156:0/3823981899 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb0540682f0 0x7fb05406a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb05c007fd0 tx=0x7fb05c0079a0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.079+0000 7fb06ec14700 1 -- 172.21.15.156:0/3823981899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0680ce480 msgr2=0x7fb0680ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.079+0000 7fb06ec14700 1 --2- 172.21.15.156:0/3823981899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0680ce480 0x7fb0680ae9f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb058009c40 tx=0x7fb058004750 comp rx=0 tx=0).stop 2023-11-28T11:08:28.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.079+0000 7fb06d1b1700 1 -- 172.21.15.156:0/3823981899 reap_dead start 2023-11-28T11:08:28.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.079+0000 7fb06ec14700 1 -- 172.21.15.156:0/3823981899 shutdown_connections 2023-11-28T11:08:28.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.079+0000 7fb06ec14700 1 -- 172.21.15.156:0/3823981899 >> 172.21.15.156:0/3823981899 conn(0x7fb0680cb670 msgr2=0x7fb06815e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.109 INFO:teuthology.orchestra.run.smithi156.stdout:55834574855 2023-11-28T11:08:28.109 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T11:08:28.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.080+0000 7f96e085f700 1 -- 172.21.15.156:0/1018973743 --> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96dc0745c0 con 0x7f96bc070b40 2023-11-28T11:08:28.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.080+0000 7f96caffd700 1 -- 172.21.15.156:0/1018973743 <== osd.4 v2:172.21.15.186:6824/103781 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f96dc0745c0 con 0x7f96bc070b40 2023-11-28T11:08:28.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7fb06ec14700 1 -- 172.21.15.156:0/3823981899 shutdown_connections 2023-11-28T11:08:28.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7fb06ec14700 1 -- 172.21.15.156:0/3823981899 wait complete. 2023-11-28T11:08:28.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f96e085f700 1 -- 172.21.15.156:0/1018973743 >> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] conn(0x7f96bc070b40 msgr2=0x7f96bc072f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f96e085f700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] conn(0x7f96bc070b40 0x7f96bc072f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f9b84b0b700 1 -- 172.21.15.156:0/572590795 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x7f9b60070bf0 msgr2=0x7f9b60073000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f9b84b0b700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x7f9b60070bf0 0x7f9b60073000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f96e085f700 1 -- 172.21.15.156:0/1018973743 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f96bc068310 msgr2=0x7f96bc06a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f96e085f700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f96bc068310 0x7f96bc06a7c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f96dc157f30 tx=0x7f96cc00b040 comp rx=0 tx=0).stop 2023-11-28T11:08:28.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f96e085f700 1 -- 172.21.15.156:0/1018973743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96dc0b0bb0 msgr2=0x7f96dc150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f96e085f700 1 --2- 172.21.15.156:0/1018973743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96dc0b0bb0 0x7f96dc150110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f96d400b370 tx=0x7f96d4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f96dad9d700 1 -- 172.21.15.156:0/1018973743 reap_dead start 2023-11-28T11:08:28.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f96e085f700 1 -- 172.21.15.156:0/1018973743 shutdown_connections 2023-11-28T11:08:28.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f96e085f700 1 -- 172.21.15.156:0/1018973743 >> 172.21.15.156:0/1018973743 conn(0x7f96dc0cb670 msgr2=0x7f96dc070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f9b84b0b700 1 -- 172.21.15.156:0/572590795 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9b60068340 msgr2=0x7f9b6006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f9b84b0b700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9b60068340 0x7f9b6006a7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9b800b08e0 tx=0x7f9b74008040 comp rx=0 tx=0).stop 2023-11-28T11:08:28.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f9b84b0b700 1 -- 172.21.15.156:0/572590795 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b80161220 msgr2=0x7f9b80151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f9b84b0b700 1 --2- 172.21.15.156:0/572590795 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b80161220 0x7f9b80151650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9b70013000 tx=0x7f9b70004750 comp rx=0 tx=0).stop 2023-11-28T11:08:28.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.081+0000 7f9b7ed9d700 1 -- 172.21.15.156:0/572590795 reap_dead start 2023-11-28T11:08:28.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.082+0000 7f96e085f700 1 -- 172.21.15.156:0/1018973743 shutdown_connections 2023-11-28T11:08:28.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.082+0000 7f96e085f700 1 -- 172.21.15.156:0/1018973743 wait complete. 2023-11-28T11:08:28.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.083+0000 7f9b84b0b700 1 -- 172.21.15.156:0/572590795 shutdown_connections 2023-11-28T11:08:28.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.083+0000 7f9b84b0b700 1 -- 172.21.15.156:0/572590795 >> 172.21.15.156:0/572590795 conn(0x7f9b800cb670 msgr2=0x7f9b8015ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.115 INFO:teuthology.orchestra.run.smithi156.stdout:55834574856 2023-11-28T11:08:28.115 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T11:08:28.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.088+0000 7f9b84b0b700 1 -- 172.21.15.156:0/572590795 shutdown_connections 2023-11-28T11:08:28.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.088+0000 7fea239cd700 1 -- 172.21.15.156:0/2259240176 --> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea1c0745c0 con 0x7fea00070b70 2023-11-28T11:08:28.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.089+0000 7f9b84b0b700 1 -- 172.21.15.156:0/572590795 wait complete. 2023-11-28T11:08:28.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.089+0000 7fea11ffb700 1 -- 172.21.15.156:0/2259240176 <== osd.7 v2:172.21.15.186:6816/103780 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fea1c0745c0 con 0x7fea00070b70 2023-11-28T11:08:28.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.089+0000 7fea239cd700 1 -- 172.21.15.156:0/2259240176 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x7fea00070b70 msgr2=0x7fea00072f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.089+0000 7fea239cd700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x7fea00070b70 0x7fea00072f80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.089+0000 7fea239cd700 1 -- 172.21.15.156:0/2259240176 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea00068340 msgr2=0x7fea0006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.089+0000 7fea239cd700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea00068340 0x7fea0006a7f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000f60 tx=0x7fea0c007a40 comp rx=0 tx=0).stop 2023-11-28T11:08:28.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.089+0000 7fea239cd700 1 -- 172.21.15.156:0/2259240176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea1c06d4f0 msgr2=0x7fea1c1f06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.089+0000 7fea239cd700 1 --2- 172.21.15.156:0/2259240176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea1c06d4f0 0x7fea1c1f06a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fea18000c00 tx=0x7fea18004750 comp rx=0 tx=0).stop 2023-11-28T11:08:28.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.089+0000 7fea21f6a700 1 -- 172.21.15.156:0/2259240176 reap_dead start 2023-11-28T11:08:28.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.090+0000 7fea239cd700 1 -- 172.21.15.156:0/2259240176 shutdown_connections 2023-11-28T11:08:28.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.090+0000 7fea239cd700 1 -- 172.21.15.156:0/2259240176 >> 172.21.15.156:0/2259240176 conn(0x7fea1c0cb670 msgr2=0x7fea1c0b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.090+0000 7fea239cd700 1 -- 172.21.15.156:0/2259240176 shutdown_connections 2023-11-28T11:08:28.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.090+0000 7fea239cd700 1 -- 172.21.15.156:0/2259240176 wait complete. 2023-11-28T11:08:28.119 INFO:teuthology.orchestra.run.smithi156.stdout:55834574855 2023-11-28T11:08:28.120 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T11:08:28.120 INFO:teuthology.orchestra.run.smithi156.stdout:55834574855 2023-11-28T11:08:28.121 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T11:08:28.121 INFO:teuthology.orchestra.run.smithi156.stdout:55834574857 2023-11-28T11:08:28.122 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T11:08:28.123 INFO:teuthology.orchestra.run.smithi156.stdout:55834574856 2023-11-28T11:08:28.123 INFO:teuthology.orchestra.run.smithi156.stdout:55834574855 2023-11-28T11:08:28.124 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T11:08:28.124 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T11:08:28.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.357+0000 7fbac7a56700 1 Processor -- start 2023-11-28T11:08:28.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.357+0000 7fbac7a56700 1 -- start start 2023-11-28T11:08:28.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.358+0000 7fbac7a56700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbac00d01b0 0x7fbac00d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.358+0000 7fbac7a56700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac00b0230 con 0x7fbac00b16d0 2023-11-28T11:08:28.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.358+0000 7fbac7a56700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac00b37d0 con 0x7fbac00b20b0 2023-11-28T11:08:28.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.358+0000 7fbac7a56700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac015cec0 con 0x7fbac00d01b0 2023-11-28T11:08:28.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.358+0000 7f2ea2053700 1 Processor -- start 2023-11-28T11:08:28.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.358+0000 7f2ea2053700 1 -- start start 2023-11-28T11:08:28.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.359+0000 7f2ea2053700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2e9c160b30 0x7f2e9c0c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.359+0000 7f2ea2053700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2e9c160f00 0x7f2e9c164840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.359+0000 7f2ea2053700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c0cc730 0x7f2e9c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.359+0000 7f2ea2053700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c164d80 con 0x7f2e9c160f00 2023-11-28T11:08:28.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.359+0000 7f2ea2053700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c06ba50 con 0x7f2e9c160b30 2023-11-28T11:08:28.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.359+0000 7f2ea2053700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c164f90 con 0x7f2e9c0cc730 2023-11-28T11:08:28.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.359+0000 7f2e9bfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c0cc730 0x7f2e9c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.359+0000 7f2e9bfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c0cc730 0x7f2e9c0b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:35982/0 (socket says 172.21.15.156:35982) 2023-11-28T11:08:28.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.359+0000 7f2e9bfff700 1 -- 172.21.15.156:0/3093714411 learned_addr learned my addr 172.21.15.156:0/3093714411 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7f2e9bfff700 1 -- 172.21.15.156:0/3093714411 >> v2:172.21.15.186:3300/0 conn(0x7f2e9c160b30 msgr2=0x7f2e9c0c1e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:28.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7f2e9bfff700 1 --2- 172.21.15.156:0/3093714411 >> v2:172.21.15.186:3300/0 conn(0x7f2e9c160b30 0x7f2e9c0c1e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7f2e9bfff700 1 -- 172.21.15.156:0/3093714411 >> v2:172.21.15.156:3300/0 conn(0x7f2e9c160f00 msgr2=0x7f2e9c164840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7f2e9bfff700 1 --2- 172.21.15.156:0/3093714411 >> v2:172.21.15.156:3300/0 conn(0x7f2e9c160f00 0x7f2e9c164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7f2e9bfff700 1 -- 172.21.15.156:0/3093714411 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c165270 con 0x7f2e9c0cc730 2023-11-28T11:08:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.359+0000 7fbac57f2700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbac00d01b0 0x7fbac00d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.359+0000 7fbac57f2700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbac00d01b0 0x7fbac00d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:35976/0 (socket says 172.21.15.156:35976) 2023-11-28T11:08:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.359+0000 7fbac57f2700 1 -- 172.21.15.156:0/1639715518 learned_addr learned my addr 172.21.15.156:0/1639715518 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612535111 0 0) 0x7fbac00b37d0 con 0x7fbac00b20b0 2023-11-28T11:08:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaac0035b0 con 0x7fbac00b20b0 2023-11-28T11:08:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3230903756 0 0) 0x7fbaac0035b0 con 0x7fbac00b20b0 2023-11-28T11:08:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbac00d01b0 msgr2=0x7fbac00d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7fbab7fff700 1 --2- 172.21.15.156:0/1639715518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbac00d01b0 0x7fbac00d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 >> v1:172.21.15.156:6789/0 conn(0x7fbac00b16d0 legacy=0x7fbac00b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac00b4570 con 0x7fbac00b20b0 2023-11-28T11:08:28.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fbab0001670 con 0x7fbac00b20b0 2023-11-28T11:08:28.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 >> v1:172.21.15.186:6789/0 conn(0x7fbac00b20b0 legacy=0x7fbac00b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbab7fff700 1 --2- 172.21.15.156:0/1639715518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaac003a30 0x7fbaac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbab7fff700 1 --2- 172.21.15.156:0/1639715518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaac0043f0 0x7fbaac0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbab7fff700 1 --2- 172.21.15.156:0/1639715518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaac0050b0 0x7fbaac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac015cec0 con 0x7fbaac0050b0 2023-11-28T11:08:28.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac00bfb0 con 0x7fbaac0043f0 2023-11-28T11:08:28.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac014980 con 0x7fbaac003a30 2023-11-28T11:08:28.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.360+0000 7f2e9bfff700 1 --2- 172.21.15.156:0/3093714411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c0cc730 0x7f2e9c0b0670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c068870 tx=0x7f2e9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9c575741e321175 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7f2e9a7fc700 1 -- 172.21.15.156:0/3093714411 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400e070 con 0x7f2e9c0cc730 2023-11-28T11:08:28.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7f2e9a7fc700 1 -- 172.21.15.156:0/3093714411 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9400b9c0 con 0x7f2e9c0cc730 2023-11-28T11:08:28.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7f2e9a7fc700 1 -- 172.21.15.156:0/3093714411 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400bd70 con 0x7f2e9c0cc730 2023-11-28T11:08:28.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbac4ff1700 1 --2- 172.21.15.156:0/1639715518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaac0043f0 0x7fbaac0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbac5ff3700 1 --2- 172.21.15.156:0/1639715518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaac0050b0 0x7fbaac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbac4ff1700 1 -- 172.21.15.156:0/1639715518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaac003a30 msgr2=0x7fbaac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbac4ff1700 1 --2- 172.21.15.156:0/1639715518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaac003a30 0x7fbaac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbac4ff1700 1 -- 172.21.15.156:0/1639715518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaac0050b0 msgr2=0x7fbaac00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbac4ff1700 1 --2- 172.21.15.156:0/1639715518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaac0050b0 0x7fbaac00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbac4ff1700 1 -- 172.21.15.156:0/1639715518 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac00b4570 con 0x7fbaac0043f0 2023-11-28T11:08:28.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbac5ff3700 1 --2- 172.21.15.156:0/1639715518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaac0050b0 0x7fbaac00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:28.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.361+0000 7fbac4ff1700 1 --2- 172.21.15.156:0/1639715518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaac0043f0 0x7fbaac0048b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbab000c290 tx=0x7fbaac005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7474f8f7856372b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab000cdd0 con 0x7fbaac0043f0 2023-11-28T11:08:28.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7f2ea2053700 1 -- 172.21.15.156:0/3093714411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c0cc730 msgr2=0x7f2e9c0b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7f2ea2053700 1 --2- 172.21.15.156:0/3093714411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c0cc730 0x7f2e9c0b0670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c068870 tx=0x7f2e9400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:28.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7f2ea2053700 1 -- 172.21.15.156:0/3093714411 shutdown_connections 2023-11-28T11:08:28.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7f2ea2053700 1 --2- 172.21.15.156:0/3093714411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c0cc730 0x7f2e9c0b0670 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7f2ea2053700 1 --2- 172.21.15.156:0/3093714411 >> v2:172.21.15.186:3300/0 conn(0x7f2e9c160b30 0x7f2e9c0c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7f2ea2053700 1 --2- 172.21.15.156:0/3093714411 >> v2:172.21.15.156:3300/0 conn(0x7f2e9c160f00 0x7f2e9c164840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7f2ea2053700 1 -- 172.21.15.156:0/3093714411 >> 172.21.15.156:0/3093714411 conn(0x7f2e9c0cb670 msgr2=0x7f2e9c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7f860ea07700 1 Processor -- start 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7feb46fdc700 1 Processor -- start 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab0006860 con 0x7fbaac0043f0 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7fbab7fff700 1 -- 172.21.15.156:0/1639715518 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab0006c10 con 0x7fbaac0043f0 2023-11-28T11:08:28.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7f2ea2053700 1 -- 172.21.15.156:0/3093714411 shutdown_connections 2023-11-28T11:08:28.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7f2ea2053700 1 -- 172.21.15.156:0/3093714411 wait complete. 2023-11-28T11:08:28.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7fbac7a56700 1 -- 172.21.15.156:0/1639715518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaac0043f0 msgr2=0x7fbaac0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7fbac7a56700 1 --2- 172.21.15.156:0/1639715518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaac0043f0 0x7fbaac0048b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbab000c290 tx=0x7fbaac005880 comp rx=0 tx=0).stop 2023-11-28T11:08:28.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7fbac5ff3700 1 -- 172.21.15.156:0/1639715518 reap_dead start 2023-11-28T11:08:28.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7feb46fdc700 1 -- start start 2023-11-28T11:08:28.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7feb46fdc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb4006d4f0 0x7feb400b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7fbac7a56700 1 -- 172.21.15.156:0/1639715518 shutdown_connections 2023-11-28T11:08:28.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7fbac7a56700 1 -- 172.21.15.156:0/1639715518 >> 172.21.15.156:0/1639715518 conn(0x7fbac009c3f0 msgr2=0x7fbac00d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7feb46fdc700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7feb400b0bb0 0x7feb4006b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7feb46fdc700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb400b1000 con 0x7feb40158890 2023-11-28T11:08:28.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7feb46fdc700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4015c490 con 0x7feb400b0bb0 2023-11-28T11:08:28.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7feb46fdc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4015c730 con 0x7feb4006d4f0 2023-11-28T11:08:28.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7fbac7a56700 1 -- 172.21.15.156:0/1639715518 shutdown_connections 2023-11-28T11:08:28.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.362+0000 7f860ea07700 1 -- start start 2023-11-28T11:08:28.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7f860ea07700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8608158890 0x7f86080ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7f860ea07700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860815c490 con 0x7f86080b0d80 2023-11-28T11:08:28.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7f860ea07700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86080ce650 con 0x7f860806d4f0 2023-11-28T11:08:28.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7f860ea07700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f860815c730 con 0x7f8608158890 2023-11-28T11:08:28.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7feb3ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb4006d4f0 0x7feb400b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7feb3ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb4006d4f0 0x7feb400b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:35994/0 (socket says 172.21.15.156:35994) 2023-11-28T11:08:28.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7feb3ffff700 1 -- 172.21.15.156:0/1507068025 learned_addr learned my addr 172.21.15.156:0/1507068025 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7feb45579700 1 --2- 172.21.15.156:0/1507068025 >> v2:172.21.15.186:3300/0 conn(0x7feb400b0bb0 0x7feb4006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7feb3ffff700 1 -- 172.21.15.156:0/1507068025 >> v2:172.21.15.186:3300/0 conn(0x7feb400b0bb0 msgr2=0x7feb4006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7feb3ffff700 1 --2- 172.21.15.156:0/1507068025 >> v2:172.21.15.186:3300/0 conn(0x7feb400b0bb0 0x7feb4006b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7feb3ffff700 1 -- 172.21.15.156:0/1507068025 >> v1:172.21.15.186:6789/0 conn(0x7feb40158890 legacy=0x7feb400ce110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:28.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7feb3ffff700 1 -- 172.21.15.156:0/1507068025 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4015ca10 con 0x7feb4006d4f0 2023-11-28T11:08:28.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7f2ea2053700 1 Processor -- start 2023-11-28T11:08:28.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7f8607fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8608158890 0x7f86080ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7f8607fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8608158890 0x7f86080ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60366/0 (socket says 172.21.15.156:60366) 2023-11-28T11:08:28.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7f8607fff700 1 -- 172.21.15.156:0/3760109447 learned_addr learned my addr 172.21.15.156:0/3760109447 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7f39b03ce700 1 Processor -- start 2023-11-28T11:08:28.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7f2ea2053700 1 -- start start 2023-11-28T11:08:28.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998892777 0 0) 0x7f86080ce650 con 0x7f860806d4f0 2023-11-28T11:08:28.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85f00035b0 con 0x7f860806d4f0 2023-11-28T11:08:28.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017029881 0 0) 0x7f860815c490 con 0x7f86080b0d80 2023-11-28T11:08:28.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86080ce650 con 0x7f86080b0d80 2023-11-28T11:08:28.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7feb3ffff700 1 --2- 172.21.15.156:0/1507068025 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb4006d4f0 0x7feb400b0670 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb3400a990 tx=0x7feb3400a2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5a290ed312b3b40 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7f39b03ce700 1 -- start start 2023-11-28T11:08:28.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.364+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 623820467 0 0) 0x7f86080ce650 con 0x7f86080b0d80 2023-11-28T11:08:28.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 >> v2:172.21.15.156:3300/0 conn(0x7f8608158890 msgr2=0x7f86080ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f8606ffd700 1 --2- 172.21.15.156:0/3760109447 >> v2:172.21.15.156:3300/0 conn(0x7f8608158890 0x7f86080ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 >> v1:172.21.15.186:6789/0 conn(0x7f860806d4f0 legacy=0x7f86080b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860815ca10 con 0x7f86080b0d80 2023-11-28T11:08:28.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f85f8001670 con 0x7f86080b0d80 2023-11-28T11:08:28.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 >> v1:172.21.15.156:6789/0 conn(0x7f86080b0d80 legacy=0x7f860806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f8606ffd700 1 --2- 172.21.15.156:0/3760109447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85f0004bf0 0x7f85f0004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f8606ffd700 1 --2- 172.21.15.156:0/3760109447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f00037f0 0x7f85f0003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f8606ffd700 1 --2- 172.21.15.156:0/3760109447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f0009a00 0x7f85f00041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860815c730 con 0x7f85f0009a00 2023-11-28T11:08:28.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0014870 con 0x7f85f00037f0 2023-11-28T11:08:28.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0014a30 con 0x7f85f0004bf0 2023-11-28T11:08:28.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f86077fe700 1 --2- 172.21.15.156:0/3760109447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f00037f0 0x7f85f0003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f2ea2053700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c0cc730 0x7f2e9c0bcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f2ea2053700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c160b30 0x7f2e9c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.363+0000 7fbac7a56700 1 -- 172.21.15.156:0/1639715518 wait complete. 2023-11-28T11:08:28.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7fbac7a56700 1 Processor -- start 2023-11-28T11:08:28.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f860cfa4700 1 --2- 172.21.15.156:0/3760109447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f0009a00 0x7f85f00041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f2ea2053700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c160f00 0x7f2e9c201890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f2ea2053700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0bd4f0 con 0x7f2e9c160f00 2023-11-28T11:08:28.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f2ea2053700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c14cc70 con 0x7f2e9c160b30 2023-11-28T11:08:28.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f2ea2053700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c14cf50 con 0x7f2e9c0cc730 2023-11-28T11:08:28.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f2e9bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c160f00 0x7f2e9c201890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f2e9bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c160f00 0x7f2e9c201890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60380/0 (socket says 172.21.15.156:60380) 2023-11-28T11:08:28.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f2e9bfff700 1 -- 172.21.15.156:0/2750177424 learned_addr learned my addr 172.21.15.156:0/2750177424 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f860cfa4700 1 -- 172.21.15.156:0/3760109447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85f0004bf0 msgr2=0x7f85f0004fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:28.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f860cfa4700 1 --2- 172.21.15.156:0/3760109447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85f0004bf0 0x7f85f0004fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f860cfa4700 1 -- 172.21.15.156:0/3760109447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f00037f0 msgr2=0x7f85f0003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f860cfa4700 1 --2- 172.21.15.156:0/3760109447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f00037f0 0x7f85f0003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f860cfa4700 1 -- 172.21.15.156:0/3760109447 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860815ca10 con 0x7f85f0009a00 2023-11-28T11:08:28.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7f860cfa4700 1 -- 172.21.15.156:0/3760109447 reap_dead start 2023-11-28T11:08:28.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39b03ce700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f39a80b0d80 0x7f39a80c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39b03ce700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a806ba50 con 0x7f39a8160b30 2023-11-28T11:08:28.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39b03ce700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a8164f90 con 0x7f39a8160f00 2023-11-28T11:08:28.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39b03ce700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a8165270 con 0x7f39a80b0d80 2023-11-28T11:08:28.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ae16a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f39a8160b30 0x7f39a80cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47028/0 (socket says 172.21.15.156:47028) 2023-11-28T11:08:28.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ae16a700 1 -- 172.21.15.156:0/886059987 learned_addr learned my addr 172.21.15.156:0/886059987 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:28.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f2e9bfff700 1 -- 172.21.15.156:0/2750177424 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c0cc730 msgr2=0x7f2e9c0bcfb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:28.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f2e9bfff700 1 --2- 172.21.15.156:0/2750177424 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c0cc730 0x7f2e9c0bcfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f2e9bfff700 1 -- 172.21.15.156:0/2750177424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c160b30 msgr2=0x7f2e9c0b4320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:28.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f2e9bfff700 1 --2- 172.21.15.156:0/2750177424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c160b30 0x7f2e9c0b4320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f2e9bfff700 1 -- 172.21.15.156:0/2750177424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e94009b30 con 0x7f2e9c160f00 2023-11-28T11:08:28.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f860cfa4700 1 --2- 172.21.15.156:0/3760109447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f0009a00 0x7f85f00041a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c260 tx=0x7f85f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bfb251e1aae042 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f86077fe700 1 --2- 172.21.15.156:0/3760109447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85f00037f0 0x7f85f0003c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:28.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f2e9bfff700 1 --2- 172.21.15.156:0/2750177424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c160f00 0x7f2e9c201890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400b730 tx=0x7f2e94004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ae96b700 1 --2- 172.21.15.156:0/886059987 >> v2:172.21.15.186:3300/0 conn(0x7f39a80b0d80 0x7f39a80c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8007b80 con 0x7f85f0009a00 2023-11-28T11:08:28.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285512293 0 0) 0x7f39a8164f90 con 0x7f39a8160f00 2023-11-28T11:08:28.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39940035b0 con 0x7f39a8160f00 2023-11-28T11:08:28.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284859055 0 0) 0x7f39a806ba50 con 0x7f39a8160b30 2023-11-28T11:08:28.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f800cd20 con 0x7f85f0009a00 2023-11-28T11:08:28.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39a8164f90 con 0x7f39a8160b30 2023-11-28T11:08:28.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.365+0000 7fbac7a56700 1 -- start start 2023-11-28T11:08:28.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7fbac7a56700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaac0043f0 0x7fbac0202f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7fbac7a56700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbac00b6930 0x7fbac015c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7fbac7a56700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbac00b6da0 0x7fbac01ff420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7fbac7a56700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac0203460 con 0x7fbac00b6da0 2023-11-28T11:08:28.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7fbac7a56700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac015d610 con 0x7fbaac0043f0 2023-11-28T11:08:28.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7fbac7a56700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac015d330 con 0x7fbac00b6930 2023-11-28T11:08:28.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2980033609 0 0) 0x7f39a8164f90 con 0x7f39a8160b30 2023-11-28T11:08:28.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 >> v2:172.21.15.186:3300/0 conn(0x7f39a80b0d80 msgr2=0x7f39a80c1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 --2- 172.21.15.156:0/886059987 >> v2:172.21.15.186:3300/0 conn(0x7f39a80b0d80 0x7f39a80c1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 >> v1:172.21.15.186:6789/0 conn(0x7f39a8160f00 legacy=0x7f39a80b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a8165550 con 0x7f39a8160b30 2023-11-28T11:08:28.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f39a40015a0 con 0x7f39a8160b30 2023-11-28T11:08:28.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 >> v1:172.21.15.156:6789/0 conn(0x7f39a8160b30 legacy=0x7f39a80cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7fbac4ff1700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbac00b6930 0x7fbac015c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7fbac4ff1700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbac00b6930 0x7fbac015c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36022/0 (socket says 172.21.15.156:36022) 2023-11-28T11:08:28.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7fbac4ff1700 1 -- 172.21.15.156:0/1568806790 learned_addr learned my addr 172.21.15.156:0/1568806790 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 --2- 172.21.15.156:0/886059987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3994004c20 0x7f3994004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 --2- 172.21.15.156:0/886059987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 0x7f3994003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f39ad168700 1 --2- 172.21.15.156:0/886059987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39940041a0 0x7f399400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3998001230 con 0x7f39940037f0 2023-11-28T11:08:28.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a8165270 con 0x7f3994004c20 2023-11-28T11:08:28.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399400c100 con 0x7f39940041a0 2023-11-28T11:08:28.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ad969700 1 --2- 172.21.15.156:0/886059987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 0x7f3994003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ae16a700 1 --2- 172.21.15.156:0/886059987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3994004c20 0x7f3994004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ae96b700 1 --2- 172.21.15.156:0/886059987 >> v2:172.21.15.186:3300/0 conn(0x7f39a80b0d80 0x7f39a80c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:28.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f860ea07700 1 -- 172.21.15.156:0/3760109447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f0009a00 msgr2=0x7f85f00041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f860ea07700 1 --2- 172.21.15.156:0/3760109447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f0009a00 0x7f85f00041a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c260 tx=0x7f85f0005880 comp rx=0 tx=0).stop 2023-11-28T11:08:28.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ad969700 1 -- 172.21.15.156:0/886059987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39940041a0 msgr2=0x7f399400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ad969700 1 --2- 172.21.15.156:0/886059987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39940041a0 0x7f399400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ad969700 1 -- 172.21.15.156:0/886059987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3994004c20 msgr2=0x7f3994004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ad969700 1 --2- 172.21.15.156:0/886059987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3994004c20 0x7f3994004ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ad969700 1 -- 172.21.15.156:0/886059987 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a8164f90 con 0x7f39940037f0 2023-11-28T11:08:28.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ae96b700 1 -- 172.21.15.156:0/886059987 reap_dead start 2023-11-28T11:08:28.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ad969700 1 --2- 172.21.15.156:0/886059987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 0x7f3994003c60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3998007fd0 tx=0x7f3994005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d014aa4f39230dc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.367+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399800cca0 con 0x7f39940037f0 2023-11-28T11:08:28.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.368+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399800ce00 con 0x7f39940037f0 2023-11-28T11:08:28.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.368+0000 7f39ad168700 1 -- 172.21.15.156:0/886059987 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3998005310 con 0x7f39940037f0 2023-11-28T11:08:28.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.368+0000 7fbac57f2700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaac0043f0 0x7fbac0202f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.368+0000 7f2e93fff700 1 -- 172.21.15.156:0/2750177424 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94004390 con 0x7f2e9c160f00 2023-11-28T11:08:28.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7feb3f7fe700 1 -- 172.21.15.156:0/1507068025 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3400acf0 con 0x7feb4006d4f0 2023-11-28T11:08:28.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7feb3f7fe700 1 -- 172.21.15.156:0/1507068025 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3400ae50 con 0x7feb4006d4f0 2023-11-28T11:08:28.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7feb3f7fe700 1 -- 172.21.15.156:0/1507068025 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34004250 con 0x7feb4006d4f0 2023-11-28T11:08:28.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.368+0000 7feb46fdc700 1 -- 172.21.15.156:0/1507068025 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb4006d4f0 msgr2=0x7feb400b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.368+0000 7feb46fdc700 1 --2- 172.21.15.156:0/1507068025 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb4006d4f0 0x7feb400b0670 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb3400a990 tx=0x7feb3400a2d0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.366+0000 7f8606ffd700 1 -- 172.21.15.156:0/3760109447 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8006aa0 con 0x7f85f0009a00 2023-11-28T11:08:28.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.368+0000 7f39b03ce700 1 -- 172.21.15.156:0/886059987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 msgr2=0x7f3994003c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.368+0000 7f39b03ce700 1 --2- 172.21.15.156:0/886059987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 0x7f3994003c60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3998007fd0 tx=0x7f3994005880 comp rx=0 tx=0).stop 2023-11-28T11:08:28.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7f39b03ce700 1 -- 172.21.15.156:0/886059987 shutdown_connections 2023-11-28T11:08:28.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7f39b03ce700 1 --2- 172.21.15.156:0/886059987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 0x7f3994003c60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7f39b03ce700 1 -- 172.21.15.156:0/886059987 >> 172.21.15.156:0/886059987 conn(0x7f39a80cb670 msgr2=0x7f39a80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7fbac5ff3700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbac00b6da0 0x7fbac01ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7fbac57f2700 1 -- 172.21.15.156:0/1568806790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbac00b6930 msgr2=0x7fbac015c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7fbac57f2700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbac00b6930 0x7fbac015c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7fbac57f2700 1 -- 172.21.15.156:0/1568806790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbac00b6da0 msgr2=0x7fbac01ff420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7f860ea07700 1 -- 172.21.15.156:0/3760109447 shutdown_connections 2023-11-28T11:08:28.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7f860ea07700 1 --2- 172.21.15.156:0/3760109447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f0009a00 0x7f85f00041a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7f860ea07700 1 -- 172.21.15.156:0/3760109447 >> 172.21.15.156:0/3760109447 conn(0x7f86080cb670 msgr2=0x7f86080cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7fbac57f2700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbac00b6da0 0x7fbac01ff420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7fbac57f2700 1 -- 172.21.15.156:0/1568806790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac00afbe0 con 0x7fbaac0043f0 2023-11-28T11:08:28.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7f39b03ce700 1 -- 172.21.15.156:0/886059987 shutdown_connections 2023-11-28T11:08:28.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7fbac5ff3700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbac00b6da0 0x7fbac01ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:28.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7feb46fdc700 1 -- 172.21.15.156:0/1507068025 shutdown_connections 2023-11-28T11:08:28.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7feb46fdc700 1 --2- 172.21.15.156:0/1507068025 >> v2:172.21.15.186:3300/0 conn(0x7feb400b0bb0 0x7feb4006b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7feb46fdc700 1 --2- 172.21.15.156:0/1507068025 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb4006d4f0 0x7feb400b0670 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7feb46fdc700 1 -- 172.21.15.156:0/1507068025 >> 172.21.15.156:0/1507068025 conn(0x7feb400cb670 msgr2=0x7feb400cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.368+0000 7f2e93fff700 1 -- 172.21.15.156:0/2750177424 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e940044f0 con 0x7f2e9c160f00 2023-11-28T11:08:28.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.368+0000 7f2e93fff700 1 -- 172.21.15.156:0/2750177424 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e940048a0 con 0x7f2e9c160f00 2023-11-28T11:08:28.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.368+0000 7f2ea2053700 1 -- 172.21.15.156:0/2750177424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c0ccbb0 con 0x7f2e9c160f00 2023-11-28T11:08:28.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.368+0000 7f2ea2053700 1 -- 172.21.15.156:0/2750177424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c165550 con 0x7f2e9c160f00 2023-11-28T11:08:28.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7fbac57f2700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaac0043f0 0x7fbac0202f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbac00b5bb0 tx=0x7fbabc00eac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7f2ea2053700 1 -- 172.21.15.156:0/2750177424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c0ccd40 con 0x7f2e9c160f00 2023-11-28T11:08:28.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7f39b03ce700 1 -- 172.21.15.156:0/886059987 wait complete. 2023-11-28T11:08:28.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7fbab5ffb700 1 -- 172.21.15.156:0/1568806790 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc013680 con 0x7fbaac0043f0 2023-11-28T11:08:28.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7fbab5ffb700 1 -- 172.21.15.156:0/1568806790 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc013800 con 0x7fbaac0043f0 2023-11-28T11:08:28.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7fbab5ffb700 1 -- 172.21.15.156:0/1568806790 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc013bb0 con 0x7fbaac0043f0 2023-11-28T11:08:28.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7fbac7a56700 1 -- 172.21.15.156:0/1568806790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac00b4850 con 0x7fbaac0043f0 2023-11-28T11:08:28.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7f39b03ce700 1 Processor -- start 2023-11-28T11:08:28.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.371+0000 7fbac7a56700 1 -- 172.21.15.156:0/1568806790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac00afec0 con 0x7fbaac0043f0 2023-11-28T11:08:28.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7fec82840700 1 Processor -- start 2023-11-28T11:08:28.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.369+0000 7fec82840700 1 -- start start 2023-11-28T11:08:28.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7fec82840700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fec7c160b30 0x7fec7c0c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7fec82840700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fec7c160f00 0x7fec7c164840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7fec82840700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7c06ba50 con 0x7fec7c0cc730 2023-11-28T11:08:28.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7fec82840700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c164d80 con 0x7fec7c160b30 2023-11-28T11:08:28.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7fec82840700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c164f90 con 0x7fec7c160f00 2023-11-28T11:08:28.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7fec80ddd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fec7c0cc730 0x7fec7c0b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47048/0 (socket says 172.21.15.156:47048) 2023-11-28T11:08:28.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7fec80ddd700 1 -- 172.21.15.156:0/3840022023 learned_addr learned my addr 172.21.15.156:0/3840022023 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:28.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7fec7b7fe700 1 --2- 172.21.15.156:0/3840022023 >> v2:172.21.15.186:3300/0 conn(0x7fec7c160f00 0x7fec7c164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.371+0000 7fec7b7fe700 1 -- 172.21.15.156:0/3840022023 >> v2:172.21.15.156:3300/0 conn(0x7fec7c160b30 msgr2=0x7fec7c0c1e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:28.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.371+0000 7fec7b7fe700 1 --2- 172.21.15.156:0/3840022023 >> v2:172.21.15.156:3300/0 conn(0x7fec7c160b30 0x7fec7c0c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.371+0000 7fec7b7fe700 1 -- 172.21.15.156:0/3840022023 >> v1:172.21.15.156:6789/0 conn(0x7fec7c0cc730 legacy=0x7fec7c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.371+0000 7fec7b7fe700 1 -- 172.21.15.156:0/3840022023 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c165270 con 0x7fec7c160f00 2023-11-28T11:08:28.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.371+0000 7fb0ef262700 1 Processor -- start 2023-11-28T11:08:28.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.371+0000 7f860ea07700 1 -- 172.21.15.156:0/3760109447 shutdown_connections 2023-11-28T11:08:28.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.371+0000 7feb46fdc700 1 -- 172.21.15.156:0/1507068025 shutdown_connections 2023-11-28T11:08:28.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.370+0000 7f39b03ce700 1 -- start start 2023-11-28T11:08:28.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f39b03ce700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 0x7f39a8155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f39b03ce700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39a80c6f50 0x7f39a80af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f39b03ce700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a80c73c0 0x7f39a80afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f39b03ce700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80c4b40 con 0x7f39940037f0 2023-11-28T11:08:28.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f39b03ce700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80c85b0 con 0x7f39a80c73c0 2023-11-28T11:08:28.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f39b03ce700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a8155850 con 0x7f39a80c6f50 2023-11-28T11:08:28.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7fec7b7fe700 1 --2- 172.21.15.156:0/3840022023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec7c160f00 0x7fec7c164840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0c9be0 tx=0x7fec6c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99ff238b12f2d0af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f39ad969700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39a80c6f50 0x7f39a80af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f39ae96b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a80c73c0 0x7f39a80afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7fbac7a56700 1 -- 172.21.15.156:0/1568806790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac00b5fa0 con 0x7fbaac0043f0 2023-11-28T11:08:28.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f39ae16a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 0x7f39a8155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f39ae16a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 0x7f39a8155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60412/0 (socket says 172.21.15.156:60412) 2023-11-28T11:08:28.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f39ae16a700 1 -- 172.21.15.156:0/1491311414 learned_addr learned my addr 172.21.15.156:0/1491311414 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7fec7affd700 1 -- 172.21.15.156:0/3840022023 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c011830 con 0x7fec7c160f00 2023-11-28T11:08:28.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7fec7affd700 1 -- 172.21.15.156:0/3840022023 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c011990 con 0x7fec7c160f00 2023-11-28T11:08:28.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7fec7affd700 1 -- 172.21.15.156:0/3840022023 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c011d20 con 0x7fec7c160f00 2023-11-28T11:08:28.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f860ea07700 1 -- 172.21.15.156:0/3760109447 wait complete. 2023-11-28T11:08:28.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7f61362e0700 1 Processor -- start 2023-11-28T11:08:28.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7fbab5ffb700 1 -- 172.21.15.156:0/1568806790 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbabc019030 con 0x7fbaac0043f0 2023-11-28T11:08:28.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7f61362e0700 1 -- start start 2023-11-28T11:08:28.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7f39ae16a700 1 -- 172.21.15.156:0/1491311414 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39a80c6f50 msgr2=0x7f39a80af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7f39ae16a700 1 --2- 172.21.15.156:0/1491311414 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39a80c6f50 0x7f39a80af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7f39ae16a700 1 -- 172.21.15.156:0/1491311414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a80c73c0 msgr2=0x7f39a80afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7f39ae16a700 1 --2- 172.21.15.156:0/1491311414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a80c73c0 0x7f39a80afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7f39ae16a700 1 -- 172.21.15.156:0/1491311414 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80c1fc0 con 0x7f39940037f0 2023-11-28T11:08:28.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7fbab5ffb700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbaa40683c0 0x7fbaa406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7fec82840700 1 -- 172.21.15.156:0/3840022023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec7c160f00 msgr2=0x7fec7c164840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7fec82840700 1 --2- 172.21.15.156:0/3840022023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec7c160f00 0x7fec7c164840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0c9be0 tx=0x7fec6c009b20 comp rx=0 tx=0).stop 2023-11-28T11:08:28.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7fec82840700 1 -- 172.21.15.156:0/3840022023 shutdown_connections 2023-11-28T11:08:28.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7fec82840700 1 --2- 172.21.15.156:0/3840022023 >> v2:172.21.15.156:3300/0 conn(0x7fec7c160b30 0x7fec7c0c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7fec82840700 1 --2- 172.21.15.156:0/3840022023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec7c160f00 0x7fec7c164840 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7fec82840700 1 -- 172.21.15.156:0/3840022023 >> 172.21.15.156:0/3840022023 conn(0x7fec7c0cb670 msgr2=0x7fec7c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7fbab5ffb700 1 -- 172.21.15.156:0/1568806790 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbabc0859e0 con 0x7fbaac0043f0 2023-11-28T11:08:28.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7fbac4ff1700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbaa40683c0 0x7fbaa406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7f39ae16a700 1 --2- 172.21.15.156:0/1491311414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 0x7f39a8155310 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f39a4007370 tx=0x7f39a4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7f860ea07700 1 Processor -- start 2023-11-28T11:08:28.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7fbac4ff1700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbaa40683c0 0x7fbaa406a870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbab0000fb0 tx=0x7fbab000a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.371+0000 7f2e93fff700 1 -- 172.21.15.156:0/2750177424 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2e9401c440 con 0x7f2e9c160f00 2023-11-28T11:08:28.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.371+0000 7f2e93fff700 1 --2- 172.21.15.156:0/2750177424 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e7c0683c0 0x7f2e7c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.371+0000 7f2e93fff700 1 -- 172.21.15.156:0/2750177424 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e940879d0 con 0x7f2e9c160f00 2023-11-28T11:08:28.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.372+0000 7f2e9b7fe700 1 --2- 172.21.15.156:0/2750177424 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e7c0683c0 0x7f2e7c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.373+0000 7f2e9b7fe700 1 --2- 172.21.15.156:0/2750177424 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e7c0683c0 0x7f2e7c06a870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c202870 tx=0x7f2e8c006c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7f61362e0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f613006e500 0x7f613015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7f61362e0700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f61300ae9d0 0x7f613006e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7f61362e0700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613006ee10 con 0x7f613015e540 2023-11-28T11:08:28.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7f61362e0700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613015bf10 con 0x7f61300ae9d0 2023-11-28T11:08:28.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7f61362e0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61300aee40 con 0x7f613006e500 2023-11-28T11:08:28.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7fb0ef262700 1 -- start start 2023-11-28T11:08:28.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7fec82840700 1 -- 172.21.15.156:0/3840022023 shutdown_connections 2023-11-28T11:08:28.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7feb46fdc700 1 -- 172.21.15.156:0/1507068025 wait complete. 2023-11-28T11:08:28.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7f612f7fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f61300ae9d0 0x7f613006e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f612f7fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f61300ae9d0 0x7f613006e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:43198/0 (socket says 172.21.15.156:43198) 2023-11-28T11:08:28.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f613487d700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f613015e540 0x7f613015e960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:57424/0 (socket says 172.21.15.156:57424) 2023-11-28T11:08:28.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f612f7fe700 1 -- 172.21.15.156:0/2962501942 learned_addr learned my addr 172.21.15.156:0/2962501942 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f612ffff700 1 --2- 172.21.15.156:0/2962501942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f613006e500 0x7f613015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7f860ea07700 1 -- start start 2023-11-28T11:08:28.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f612f7fe700 1 -- 172.21.15.156:0/2962501942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f613006e500 msgr2=0x7f613015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f612f7fe700 1 --2- 172.21.15.156:0/2962501942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f613006e500 0x7f613015b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f612f7fe700 1 -- 172.21.15.156:0/2962501942 >> v1:172.21.15.186:6789/0 conn(0x7f613015e540 legacy=0x7f613015e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f612f7fe700 1 -- 172.21.15.156:0/2962501942 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613006c930 con 0x7f61300ae9d0 2023-11-28T11:08:28.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f612ffff700 1 --2- 172.21.15.156:0/2962501942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f613006e500 0x7f613015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:28.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f860ea07700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f0009a00 0x7f8608147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f860ea07700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86080c4a20 0x7f86080c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f860ea07700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86080c2680 0x7f86080c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f860ea07700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86080c53d0 con 0x7f85f0009a00 2023-11-28T11:08:28.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f860ea07700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86080c33b0 con 0x7f86080c2680 2023-11-28T11:08:28.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f860ea07700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86080c3690 con 0x7f86080c4a20 2023-11-28T11:08:28.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f612f7fe700 1 --2- 172.21.15.156:0/2962501942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61300ae9d0 0x7f613006e8d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f61240098f0 tx=0x7f6124009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6edd790fe0f9e897 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7fb0ef262700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb0e806d4f0 0x7fb0e80b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7fb0ef262700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb0e80b0bb0 0x7fb0e806b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f86077fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86080c4a20 0x7f86080c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f86077fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86080c4a20 0x7f86080c4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36056/0 (socket says 172.21.15.156:36056) 2023-11-28T11:08:28.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f86077fe700 1 -- 172.21.15.156:0/393958933 learned_addr learned my addr 172.21.15.156:0/393958933 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.375+0000 7f860cfa4700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86080c2680 0x7f86080c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7fb0ef262700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e80b1010 con 0x7fb0e8158890 2023-11-28T11:08:28.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7fb0ef262700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e815c490 con 0x7fb0e80b0bb0 2023-11-28T11:08:28.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7fb0ef262700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e815c730 con 0x7fb0e806d4f0 2023-11-28T11:08:28.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7f86077fe700 1 -- 172.21.15.156:0/393958933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86080c2680 msgr2=0x7f86080c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7f86077fe700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86080c2680 0x7f86080c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7f86077fe700 1 -- 172.21.15.156:0/393958933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f0009a00 msgr2=0x7f8608147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7f86077fe700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f0009a00 0x7f8608147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7f86077fe700 1 -- 172.21.15.156:0/393958933 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86080c9370 con 0x7f86080c4a20 2023-11-28T11:08:28.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7feb46fdc700 1 Processor -- start 2023-11-28T11:08:28.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7fb0ecffe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb0e8158890 0x7fb0e80ce110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47074/0 (socket says 172.21.15.156:47074) 2023-11-28T11:08:28.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7fb0ecffe700 1 -- 172.21.15.156:0/1104289066 learned_addr learned my addr 172.21.15.156:0/1104289066 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:28.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7feb46fdc700 1 -- start start 2023-11-28T11:08:28.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.374+0000 7fec82840700 1 -- 172.21.15.156:0/3840022023 wait complete. 2023-11-28T11:08:28.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7fb0ed7ff700 1 --2- 172.21.15.156:0/1104289066 >> v2:172.21.15.156:3300/0 conn(0x7fb0e80b0bb0 0x7fb0e806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7fec82840700 1 Processor -- start 2023-11-28T11:08:28.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7fec82840700 1 -- start start 2023-11-28T11:08:28.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7f86077fe700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86080c4a20 0x7f86080c4e90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f860000d1b0 tx=0x7f860000d4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7f8604ff9700 1 -- 172.21.15.156:0/393958933 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860000de00 con 0x7f86080c4a20 2023-11-28T11:08:28.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7f8604ff9700 1 -- 172.21.15.156:0/393958933 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8600003a00 con 0x7f86080c4a20 2023-11-28T11:08:28.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7f860ea07700 1 -- 172.21.15.156:0/393958933 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860815ccf0 con 0x7f86080c4a20 2023-11-28T11:08:28.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7f8604ff9700 1 -- 172.21.15.156:0/393958933 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600003d90 con 0x7f86080c4a20 2023-11-28T11:08:28.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7f860ea07700 1 -- 172.21.15.156:0/393958933 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8608158c60 con 0x7f86080c4a20 2023-11-28T11:08:28.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7fb0ed7ff700 1 -- 172.21.15.156:0/1104289066 >> v2:172.21.15.186:3300/0 conn(0x7fb0e806d4f0 msgr2=0x7fb0e80b0670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:28.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7fb0ed7ff700 1 --2- 172.21.15.156:0/1104289066 >> v2:172.21.15.186:3300/0 conn(0x7fb0e806d4f0 0x7fb0e80b0670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7fb0ed7ff700 1 -- 172.21.15.156:0/1104289066 >> v1:172.21.15.156:6789/0 conn(0x7fb0e8158890 legacy=0x7fb0e80ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.376+0000 7fb0ed7ff700 1 -- 172.21.15.156:0/1104289066 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e815ca10 con 0x7fb0e80b0bb0 2023-11-28T11:08:28.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fb0ed7ff700 1 --2- 172.21.15.156:0/1104289066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0e80b0bb0 0x7fb0e806b820 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80669d0 tx=0x7fb0e0009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40782c97a18fb305 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec82840700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec7c0cc730 0x7fec7c142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec82840700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec7c160b30 0x7fec7c147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec82840700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec7c160f00 0x7fec7c0c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec82840700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c146920 con 0x7fec7c160b30 2023-11-28T11:08:28.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec82840700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c147130 con 0x7fec7c160f00 2023-11-28T11:08:28.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec82840700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c0c3450 con 0x7fec7c0cc730 2023-11-28T11:08:28.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec7bfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec7c0cc730 0x7fec7c142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec7bfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec7c0cc730 0x7fec7c142990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36064/0 (socket says 172.21.15.156:36064) 2023-11-28T11:08:28.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec7bfff700 1 -- 172.21.15.156:0/3989325461 learned_addr learned my addr 172.21.15.156:0/3989325461 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7feb46fdc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb4006d4f0 0x7feb4015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7feb46fdc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb400b0bb0 0x7feb401f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7feb46fdc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb40158890 0x7feb400b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7feb46fdc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb400c3780 con 0x7feb400b0bb0 2023-11-28T11:08:28.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7feb46fdc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb400c2b90 con 0x7feb40158890 2023-11-28T11:08:28.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7feb46fdc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4015bcf0 con 0x7feb4006d4f0 2023-11-28T11:08:28.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7feb3ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb400b0bb0 0x7feb401f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7feb3ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb400b0bb0 0x7feb401f64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60436/0 (socket says 172.21.15.156:60436) 2023-11-28T11:08:28.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7feb3ffff700 1 -- 172.21.15.156:0/4174260695 learned_addr learned my addr 172.21.15.156:0/4174260695 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec7b7fe700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec7c160b30 0x7fec7c147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec7bfff700 1 -- 172.21.15.156:0/3989325461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec7c160f00 msgr2=0x7fec7c0c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec7bfff700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec7c160f00 0x7fec7c0c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec7bfff700 1 -- 172.21.15.156:0/3989325461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec7c160b30 msgr2=0x7fec7c147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec7bfff700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec7c160b30 0x7fec7c147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.377+0000 7fec7bfff700 1 -- 172.21.15.156:0/3989325461 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0094a0 con 0x7fec7c0cc730 2023-11-28T11:08:28.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7feb3ffff700 1 -- 172.21.15.156:0/4174260695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb4006d4f0 msgr2=0x7feb4015b7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:08:28.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7feb3ffff700 1 --2- 172.21.15.156:0/4174260695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb4006d4f0 0x7feb4015b7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7feb3ffff700 1 -- 172.21.15.156:0/4174260695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb40158890 msgr2=0x7feb400b1ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:08:28.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7feb3ffff700 1 --2- 172.21.15.156:0/4174260695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb40158890 0x7feb400b1ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7feb3ffff700 1 -- 172.21.15.156:0/4174260695 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3400a5f0 con 0x7feb400b0bb0 2023-11-28T11:08:28.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7fec7bfff700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec7c0cc730 0x7fec7c142990 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec7000b530 tx=0x7fec7000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7f2e93fff700 1 -- 172.21.15.156:0/2750177424 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2e94054160 con 0x7f2e9c160f00 2023-11-28T11:08:28.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7feb3ffff700 1 --2- 172.21.15.156:0/4174260695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb400b0bb0 0x7feb401f64e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feb34004000 tx=0x7feb34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7f399effd700 1 -- 172.21.15.156:0/1491311414 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a40077a0 con 0x7f39940037f0 2023-11-28T11:08:28.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7f860ea07700 1 -- 172.21.15.156:0/393958933 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86080745c0 con 0x7f86080c4a20 2023-11-28T11:08:28.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7f399effd700 1 -- 172.21.15.156:0/1491311414 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a400f8d0 con 0x7f39940037f0 2023-11-28T11:08:28.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7f399effd700 1 -- 172.21.15.156:0/1491311414 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a400fc60 con 0x7f39940037f0 2023-11-28T11:08:28.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7f39b03ce700 1 -- 172.21.15.156:0/1491311414 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a80c1e30 con 0x7f39940037f0 2023-11-28T11:08:28.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7f39b03ce700 1 -- 172.21.15.156:0/1491311414 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a80c2150 con 0x7f39940037f0 2023-11-28T11:08:28.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7fb0e77fe700 1 -- 172.21.15.156:0/1104289066 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000ca40 con 0x7fb0e80b0bb0 2023-11-28T11:08:28.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7fb0e77fe700 1 -- 172.21.15.156:0/1104289066 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e000cba0 con 0x7fb0e80b0bb0 2023-11-28T11:08:28.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.378+0000 7fb0e77fe700 1 -- 172.21.15.156:0/1104289066 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0010570 con 0x7fb0e80b0bb0 2023-11-28T11:08:28.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7feb3d7fa700 1 -- 172.21.15.156:0/4174260695 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3400acf0 con 0x7feb400b0bb0 2023-11-28T11:08:28.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7feb46fdc700 1 -- 172.21.15.156:0/4174260695 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb400c9370 con 0x7feb400b0bb0 2023-11-28T11:08:28.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7feb46fdc700 1 -- 172.21.15.156:0/4174260695 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4015ccf0 con 0x7feb400b0bb0 2023-11-28T11:08:28.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7f612effd700 1 -- 172.21.15.156:0/2962501942 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124011830 con 0x7f61300ae9d0 2023-11-28T11:08:28.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7f612effd700 1 -- 172.21.15.156:0/2962501942 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6124011990 con 0x7f61300ae9d0 2023-11-28T11:08:28.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7f612effd700 1 -- 172.21.15.156:0/2962501942 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124011d20 con 0x7f61300ae9d0 2023-11-28T11:08:28.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7fb0ef262700 1 -- 172.21.15.156:0/1104289066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0e80b0bb0 msgr2=0x7fb0e806b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7fb0ef262700 1 --2- 172.21.15.156:0/1104289066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0e80b0bb0 0x7fb0e806b820 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80669d0 tx=0x7fb0e0009950 comp rx=0 tx=0).stop 2023-11-28T11:08:28.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7fbab5ffb700 1 -- 172.21.15.156:0/1568806790 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbac00b5fa0 con 0x7fbaac0043f0 2023-11-28T11:08:28.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7feb3d7fa700 1 -- 172.21.15.156:0/4174260695 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb340038a0 con 0x7feb400b0bb0 2023-11-28T11:08:28.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7feb3d7fa700 1 -- 172.21.15.156:0/4174260695 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34003c10 con 0x7feb400b0bb0 2023-11-28T11:08:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7f61362e0700 1 -- 172.21.15.156:0/2962501942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61300ae9d0 msgr2=0x7f613006e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7f61362e0700 1 --2- 172.21.15.156:0/2962501942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61300ae9d0 0x7f613006e8d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f61240098f0 tx=0x7f6124009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7f61362e0700 1 -- 172.21.15.156:0/2962501942 shutdown_connections 2023-11-28T11:08:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7f61362e0700 1 --2- 172.21.15.156:0/2962501942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f613006e500 0x7f613015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7f61362e0700 1 --2- 172.21.15.156:0/2962501942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61300ae9d0 0x7f613006e8d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7f61362e0700 1 -- 172.21.15.156:0/2962501942 >> 172.21.15.156:0/2962501942 conn(0x7f61300b12b0 msgr2=0x7f61300b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7f61362e0700 1 -- 172.21.15.156:0/2962501942 shutdown_connections 2023-11-28T11:08:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7f399effd700 1 -- 172.21.15.156:0/1491311414 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f39a401c030 con 0x7f39940037f0 2023-11-28T11:08:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7f39b03ce700 1 -- 172.21.15.156:0/1491311414 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a80c8820 con 0x7f39940037f0 2023-11-28T11:08:28.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7f399effd700 1 --2- 172.21.15.156:0/1491311414 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f398c068340 0x7f398c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7fb0ef262700 1 -- 172.21.15.156:0/1104289066 shutdown_connections 2023-11-28T11:08:28.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7fb0ef262700 1 --2- 172.21.15.156:0/1104289066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0e80b0bb0 0x7fb0e806b820 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7fb0ef262700 1 --2- 172.21.15.156:0/1104289066 >> v2:172.21.15.186:3300/0 conn(0x7fb0e806d4f0 0x7fb0e80b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7fb0ef262700 1 -- 172.21.15.156:0/1104289066 >> 172.21.15.156:0/1104289066 conn(0x7fb0e80cb670 msgr2=0x7fb0e80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7fec78ff9700 1 -- 172.21.15.156:0/3989325461 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70011830 con 0x7fec7c0cc730 2023-11-28T11:08:28.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7fec78ff9700 1 -- 172.21.15.156:0/3989325461 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec700119b0 con 0x7fec7c0cc730 2023-11-28T11:08:28.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.379+0000 7fec78ff9700 1 -- 172.21.15.156:0/3989325461 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70011d60 con 0x7fec7c0cc730 2023-11-28T11:08:28.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.381+0000 7f399effd700 1 -- 172.21.15.156:0/1491311414 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39a4085570 con 0x7f39940037f0 2023-11-28T11:08:28.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.381+0000 7fb0ef262700 1 -- 172.21.15.156:0/1104289066 shutdown_connections 2023-11-28T11:08:28.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.382+0000 7f8604ff9700 1 -- 172.21.15.156:0/393958933 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f86000050d0 con 0x7f86080c4a20 2023-11-28T11:08:28.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.382+0000 7f8604ff9700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f85f40682f0 0x7f85f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.380+0000 7f61362e0700 1 -- 172.21.15.156:0/2962501942 wait complete. 2023-11-28T11:08:28.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.382+0000 7f61362e0700 1 Processor -- start 2023-11-28T11:08:28.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.382+0000 7f8607fff700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f85f40682f0 0x7f85f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.383+0000 7f8607fff700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f85f40682f0 0x7f85f406a7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0085a0 tx=0x7f85fc00f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.381+0000 7fb0ef262700 1 -- 172.21.15.156:0/1104289066 wait complete. 2023-11-28T11:08:28.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.383+0000 7fb0ef262700 1 Processor -- start 2023-11-28T11:08:28.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.382+0000 7f61362e0700 1 -- start start 2023-11-28T11:08:28.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.383+0000 7f61362e0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f613006e500 0x7f61301597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.383+0000 7f61362e0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f613015e540 0x7f61300ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.383+0000 7f61362e0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61300cedf0 0x7f61301f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.383+0000 7f61362e0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61300cf270 con 0x7f613015e540 2023-11-28T11:08:28.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.383+0000 7f61362e0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61300c6930 con 0x7f61300cedf0 2023-11-28T11:08:28.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.383+0000 7f61362e0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61300c6c10 con 0x7f613006e500 2023-11-28T11:08:28.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.383+0000 7f612f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f613015e540 0x7f61300ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.384+0000 7f612f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f613015e540 0x7f61300ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60446/0 (socket says 172.21.15.156:60446) 2023-11-28T11:08:28.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.384+0000 7f612f7fe700 1 -- 172.21.15.156:0/164632743 learned_addr learned my addr 172.21.15.156:0/164632743 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.384+0000 7f613487d700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61300cedf0 0x7f61301f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.384+0000 7f612f7fe700 1 -- 172.21.15.156:0/164632743 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f613006e500 msgr2=0x7f61301597b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:28.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.384+0000 7f612f7fe700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f613006e500 0x7f61301597b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.384+0000 7f612f7fe700 1 -- 172.21.15.156:0/164632743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61300cedf0 msgr2=0x7f61301f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.384+0000 7f612f7fe700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61300cedf0 0x7f61301f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.384+0000 7f612f7fe700 1 -- 172.21.15.156:0/164632743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612000da70 con 0x7f613015e540 2023-11-28T11:08:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.384+0000 7f613487d700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61300cedf0 0x7f61301f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.384+0000 7f612f7fe700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f613015e540 0x7f61300ce8b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6124009c40 tx=0x7f6124004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7f39ad969700 1 --2- 172.21.15.156:0/1491311414 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f398c068340 0x7f398c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.383+0000 7fb0ef262700 1 -- start start 2023-11-28T11:08:28.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ef262700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0e806d4f0 0x7fb0e81f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ef262700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0e80b0bb0 0x7fb0e815af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ef262700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e8158890 0x7fb0e815b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ef262700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8154fb0 con 0x7fb0e806d4f0 2023-11-28T11:08:28.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ef262700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80b2630 con 0x7fb0e8158890 2023-11-28T11:08:28.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ef262700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8159f70 con 0x7fb0e80b0bb0 2023-11-28T11:08:28.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ed7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e8158890 0x7fb0e815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ed7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e8158890 0x7fb0e815b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:43232/0 (socket says 172.21.15.156:43232) 2023-11-28T11:08:28.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ed7ff700 1 -- 172.21.15.156:0/2003404258 learned_addr learned my addr 172.21.15.156:0/2003404258 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:28.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ecffe700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0e806d4f0 0x7fb0e81f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0e7fff700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0e80b0bb0 0x7fb0e815af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7f39ad969700 1 --2- 172.21.15.156:0/1491311414 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f398c068340 0x7f398c06a7f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3998015040 tx=0x7f399800a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ed7ff700 1 -- 172.21.15.156:0/2003404258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0e80b0bb0 msgr2=0x7fb0e815af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ed7ff700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0e80b0bb0 0x7fb0e815af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ed7ff700 1 -- 172.21.15.156:0/2003404258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0e806d4f0 msgr2=0x7fb0e81f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ed7ff700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0e806d4f0 0x7fb0e81f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ed7ff700 1 -- 172.21.15.156:0/2003404258 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e000c300 con 0x7fb0e8158890 2023-11-28T11:08:28.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fec82840700 1 -- 172.21.15.156:0/3989325461 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c0b0d80 con 0x7fec7c0cc730 2023-11-28T11:08:28.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.385+0000 7fb0ed7ff700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e8158890 0x7fb0e815b460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8067360 tx=0x7fb0e0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.386+0000 7fec82840700 1 -- 172.21.15.156:0/3989325461 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c165550 con 0x7fec7c0cc730 2023-11-28T11:08:28.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.386+0000 7feb46fdc700 1 -- 172.21.15.156:0/4174260695 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb200052f0 con 0x7feb400b0bb0 2023-11-28T11:08:28.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.386+0000 7fec82840700 1 -- 172.21.15.156:0/3989325461 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c0b0f10 con 0x7fec7c0cc730 2023-11-28T11:08:28.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.387+0000 7feb3d7fa700 1 -- 172.21.15.156:0/4174260695 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feb34003d70 con 0x7feb400b0bb0 2023-11-28T11:08:28.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.387+0000 7feb3d7fa700 1 --2- 172.21.15.156:0/4174260695 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7feb240682f0 0x7feb2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.387+0000 7feb44d78700 1 --2- 172.21.15.156:0/4174260695 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7feb240682f0 0x7feb2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.387+0000 7feb3d7fa700 1 -- 172.21.15.156:0/4174260695 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feb34087a40 con 0x7feb400b0bb0 2023-11-28T11:08:28.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.388+0000 7feb44d78700 1 --2- 172.21.15.156:0/4174260695 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7feb240682f0 0x7feb2406a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009670 tx=0x7feb2c006bd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.388+0000 7f399effd700 1 -- 172.21.15.156:0/1491311414 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f39a404e5e0 con 0x7f39940037f0 2023-11-28T11:08:28.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.387+0000 7fec78ff9700 1 -- 172.21.15.156:0/3989325461 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fec70021070 con 0x7fec7c0cc730 2023-11-28T11:08:28.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.388+0000 7fec78ff9700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fec68068340 0x7fec6806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.388+0000 7fec78ff9700 1 -- 172.21.15.156:0/3989325461 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fec70085c40 con 0x7fec7c0cc730 2023-11-28T11:08:28.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.388+0000 7fec7b7fe700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fec68068340 0x7fec6806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.389+0000 7fec7b7fe700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fec68068340 0x7fec6806a7f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0090b0 tx=0x7fec6c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.390+0000 7fb0e57fa700 1 -- 172.21.15.156:0/2003404258 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000c840 con 0x7fb0e8158890 2023-11-28T11:08:28.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.390+0000 7fb0e57fa700 1 -- 172.21.15.156:0/2003404258 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e000c9a0 con 0x7fb0e8158890 2023-11-28T11:08:28.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.390+0000 7fb0e57fa700 1 -- 172.21.15.156:0/2003404258 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0003c60 con 0x7fb0e8158890 2023-11-28T11:08:28.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.390+0000 7fb0ef262700 1 -- 172.21.15.156:0/2003404258 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e80c9370 con 0x7fb0e8158890 2023-11-28T11:08:28.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.390+0000 7fb0ef262700 1 -- 172.21.15.156:0/2003404258 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e815ccf0 con 0x7fb0e8158890 2023-11-28T11:08:28.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.390+0000 7f8604ff9700 1 -- 172.21.15.156:0/393958933 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f860005dfe0 con 0x7f86080c4a20 2023-11-28T11:08:28.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.390+0000 7f8604ff9700 1 -- 172.21.15.156:0/393958933 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8600052290 con 0x7f86080c4a20 2023-11-28T11:08:28.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.392+0000 7fb0e57fa700 1 -- 172.21.15.156:0/2003404258 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb0e0029020 con 0x7fb0e8158890 2023-11-28T11:08:28.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.392+0000 7fb0e57fa700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb0cc068340 0x7fb0cc06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.392+0000 7fb0e57fa700 1 -- 172.21.15.156:0/2003404258 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0e0014070 con 0x7fb0e8158890 2023-11-28T11:08:28.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.394+0000 7feb3d7fa700 1 -- 172.21.15.156:0/4174260695 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feb340540c0 con 0x7feb400b0bb0 2023-11-28T11:08:28.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.395+0000 7fec78ff9700 1 -- 172.21.15.156:0/3989325461 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fec700524c0 con 0x7fec7c0cc730 2023-11-28T11:08:28.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.395+0000 7f612cff9700 1 -- 172.21.15.156:0/164632743 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124004450 con 0x7f613015e540 2023-11-28T11:08:28.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.395+0000 7f612cff9700 1 -- 172.21.15.156:0/164632743 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61240045b0 con 0x7f613015e540 2023-11-28T11:08:28.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.395+0000 7f612cff9700 1 -- 172.21.15.156:0/164632743 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61240038e0 con 0x7f613015e540 2023-11-28T11:08:28.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.395+0000 7f61362e0700 1 -- 172.21.15.156:0/164632743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6124009550 con 0x7f613015e540 2023-11-28T11:08:28.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.395+0000 7f61362e0700 1 -- 172.21.15.156:0/164632743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61300664c0 con 0x7f613015e540 2023-11-28T11:08:28.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.397+0000 7f612cff9700 1 -- 172.21.15.156:0/164632743 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f612400f050 con 0x7f613015e540 2023-11-28T11:08:28.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.397+0000 7f612cff9700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f61180684a0 0x7f611806a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.397+0000 7f612cff9700 1 -- 172.21.15.156:0/164632743 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6124086d00 con 0x7f613015e540 2023-11-28T11:08:28.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.398+0000 7f612ffff700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f61180684a0 0x7f611806a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.398+0000 7f612ffff700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f61180684a0 0x7f611806a950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6120006ca0 tx=0x7f612000d750 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.401+0000 7fb0ecffe700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb0cc068340 0x7fb0cc06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.401+0000 7fb0ecffe700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb0cc068340 0x7fb0cc06a7f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4005910 tx=0x7fb0d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.409+0000 7f61167fc700 1 -- 172.21.15.156:0/164632743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61100052f0 con 0x7f613015e540 2023-11-28T11:08:28.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.403+0000 7fb0ef262700 1 -- 172.21.15.156:0/2003404258 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c80052f0 con 0x7fb0e8158890 2023-11-28T11:08:28.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.408+0000 7fb0e57fa700 1 -- 172.21.15.156:0/2003404258 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb0e0058a50 con 0x7fb0e8158890 2023-11-28T11:08:28.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.418+0000 7f612cff9700 1 -- 172.21.15.156:0/164632743 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6124015080 con 0x7f613015e540 2023-11-28T11:08:28.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.760+0000 7fbac7a56700 1 -- 172.21.15.156:0/1568806790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbac00b5fa0 con 0x7fbaac0043f0 2023-11-28T11:08:28.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.761+0000 7fbab5ffb700 1 -- 172.21.15.156:0/1568806790 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbac00b5fa0 con 0x7fbaac0043f0 2023-11-28T11:08:28.764 INFO:teuthology.orchestra.run.smithi156.stdout:55834574853 2023-11-28T11:08:28.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.764+0000 7fbac7a56700 1 -- 172.21.15.156:0/1568806790 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbaa40683c0 msgr2=0x7fbaa406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.764+0000 7fbac7a56700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbaa40683c0 0x7fbaa406a870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbab0000fb0 tx=0x7fbab000a000 comp rx=0 tx=0).stop 2023-11-28T11:08:28.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.764+0000 7fbac7a56700 1 -- 172.21.15.156:0/1568806790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaac0043f0 msgr2=0x7fbac0202f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.764+0000 7fbac7a56700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaac0043f0 0x7fbac0202f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbac00b5bb0 tx=0x7fbabc00eac0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.765+0000 7fbac7a56700 1 -- 172.21.15.156:0/1568806790 shutdown_connections 2023-11-28T11:08:28.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.765+0000 7fbac7a56700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbac00b6da0 0x7fbac01ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.765+0000 7fbac7a56700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaac0043f0 0x7fbac0202f20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.765+0000 7fbac7a56700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbaa40683c0 0x7fbaa406a870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.765+0000 7fbac7a56700 1 --2- 172.21.15.156:0/1568806790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbac00b6930 0x7fbac015c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.765+0000 7fbac7a56700 1 -- 172.21.15.156:0/1568806790 >> 172.21.15.156:0/1568806790 conn(0x7fbac009c3f0 msgr2=0x7fbac0157070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.765+0000 7fbac7a56700 1 -- 172.21.15.156:0/1568806790 shutdown_connections 2023-11-28T11:08:28.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.765+0000 7fbac7a56700 1 -- 172.21.15.156:0/1568806790 wait complete. 2023-11-28T11:08:28.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.777+0000 7f860ea07700 1 -- 172.21.15.156:0/393958933 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8608071dc0 con 0x7f86080c4a20 2023-11-28T11:08:28.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.777+0000 7f8604ff9700 1 -- 172.21.15.156:0/393958933 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f86000558b0 con 0x7f86080c4a20 2023-11-28T11:08:28.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.778+0000 7f39b03ce700 1 -- 172.21.15.156:0/1491311414 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f39a80b4bc0 con 0x7f39940037f0 2023-11-28T11:08:28.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.778+0000 7feb46fdc700 1 -- 172.21.15.156:0/4174260695 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7feb200055a0 con 0x7feb400b0bb0 2023-11-28T11:08:28.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.778+0000 7f399effd700 1 -- 172.21.15.156:0/1491311414 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f39a401a090 con 0x7f39940037f0 2023-11-28T11:08:28.780 INFO:teuthology.orchestra.run.smithi156.stdout:55834574855 2023-11-28T11:08:28.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.780+0000 7f2ea2053700 1 -- 172.21.15.156:0/2750177424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2e9c0b4c00 con 0x7f2e9c160f00 2023-11-28T11:08:28.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.780+0000 7f2e93fff700 1 -- 172.21.15.156:0/2750177424 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2e94057780 con 0x7f2e9c160f00 2023-11-28T11:08:28.782 INFO:teuthology.orchestra.run.smithi156.stdout:55834574854 2023-11-28T11:08:28.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.782+0000 7feb3d7fa700 1 -- 172.21.15.156:0/4174260695 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feb340576e0 con 0x7feb400b0bb0 2023-11-28T11:08:28.783 INFO:teuthology.orchestra.run.smithi156.stdout:55834574853 2023-11-28T11:08:28.785 INFO:teuthology.orchestra.run.smithi156.stdout:55834574853 2023-11-28T11:08:28.786 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.0 2023-11-28T11:08:28.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7f39b03ce700 1 -- 172.21.15.156:0/1491311414 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f398c068340 msgr2=0x7f398c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7f39b03ce700 1 --2- 172.21.15.156:0/1491311414 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f398c068340 0x7f398c06a7f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3998015040 tx=0x7f399800a000 comp rx=0 tx=0).stop 2023-11-28T11:08:28.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7f39b03ce700 1 -- 172.21.15.156:0/1491311414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 msgr2=0x7f39a8155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7f39b03ce700 1 --2- 172.21.15.156:0/1491311414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 0x7f39a8155310 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f39a4007370 tx=0x7f39a4007680 comp rx=0 tx=0).stop 2023-11-28T11:08:28.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7feb46fdc700 1 -- 172.21.15.156:0/4174260695 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7feb240682f0 msgr2=0x7feb2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7feb46fdc700 1 --2- 172.21.15.156:0/4174260695 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7feb240682f0 0x7feb2406a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009670 tx=0x7feb2c006bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7feb46fdc700 1 -- 172.21.15.156:0/4174260695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb400b0bb0 msgr2=0x7feb401f64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7feb46fdc700 1 --2- 172.21.15.156:0/4174260695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb400b0bb0 0x7feb401f64e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feb34004000 tx=0x7feb34004750 comp rx=0 tx=0).stop 2023-11-28T11:08:28.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7f860ea07700 1 -- 172.21.15.156:0/393958933 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f85f40682f0 msgr2=0x7f85f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7f860ea07700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f85f40682f0 0x7f85f406a7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0085a0 tx=0x7f85fc00f040 comp rx=0 tx=0).stop 2023-11-28T11:08:28.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7f860ea07700 1 -- 172.21.15.156:0/393958933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86080c4a20 msgr2=0x7f86080c4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7f860ea07700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86080c4a20 0x7f86080c4e90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f860000d1b0 tx=0x7f860000d4c0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7feb46fdc700 1 -- 172.21.15.156:0/4174260695 shutdown_connections 2023-11-28T11:08:28.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7feb46fdc700 1 --2- 172.21.15.156:0/4174260695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb40158890 0x7feb400b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7feb46fdc700 1 --2- 172.21.15.156:0/4174260695 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7feb240682f0 0x7feb2406a7a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7feb46fdc700 1 --2- 172.21.15.156:0/4174260695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb4006d4f0 0x7feb4015b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.785+0000 7feb46fdc700 1 --2- 172.21.15.156:0/4174260695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb400b0bb0 0x7feb401f64e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.786+0000 7feb46fdc700 1 -- 172.21.15.156:0/4174260695 >> 172.21.15.156:0/4174260695 conn(0x7feb400cb670 msgr2=0x7feb40155e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.786+0000 7feb46fdc700 1 -- 172.21.15.156:0/4174260695 shutdown_connections 2023-11-28T11:08:28.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.786+0000 7feb46fdc700 1 -- 172.21.15.156:0/4174260695 wait complete. 2023-11-28T11:08:28.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.787+0000 7f2ea2053700 1 -- 172.21.15.156:0/2750177424 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e7c0683c0 msgr2=0x7f2e7c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.787+0000 7f2ea2053700 1 --2- 172.21.15.156:0/2750177424 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e7c0683c0 0x7f2e7c06a870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c202870 tx=0x7f2e8c006c60 comp rx=0 tx=0).stop 2023-11-28T11:08:28.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.787+0000 7f2ea2053700 1 -- 172.21.15.156:0/2750177424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c160f00 msgr2=0x7f2e9c201890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.787+0000 7f2ea2053700 1 --2- 172.21.15.156:0/2750177424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c160f00 0x7f2e9c201890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400b730 tx=0x7f2e94004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.788+0000 7fec82840700 1 -- 172.21.15.156:0/3989325461 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fec7c068240 con 0x7fec7c0cc730 2023-11-28T11:08:28.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.788+0000 7f860ea07700 1 -- 172.21.15.156:0/393958933 shutdown_connections 2023-11-28T11:08:28.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.788+0000 7f860ea07700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86080c2680 0x7f86080c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.788+0000 7f860ea07700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f0009a00 0x7f8608147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.788+0000 7f860ea07700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f85f40682f0 0x7f85f406a7a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.788+0000 7f860ea07700 1 --2- 172.21.15.156:0/393958933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86080c4a20 0x7f86080c4e90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.788+0000 7f860ea07700 1 -- 172.21.15.156:0/393958933 >> 172.21.15.156:0/393958933 conn(0x7f86080cb670 msgr2=0x7f86080cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.788+0000 7f860ea07700 1 -- 172.21.15.156:0/393958933 shutdown_connections 2023-11-28T11:08:28.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.788+0000 7fec78ff9700 1 -- 172.21.15.156:0/3989325461 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fec70015020 con 0x7fec7c0cc730 2023-11-28T11:08:28.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.788+0000 7f860ea07700 1 -- 172.21.15.156:0/393958933 wait complete. 2023-11-28T11:08:28.793 INFO:teuthology.orchestra.run.smithi156.stdout:55834574856 2023-11-28T11:08:28.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.789+0000 7f39b03ce700 1 -- 172.21.15.156:0/1491311414 shutdown_connections 2023-11-28T11:08:28.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.789+0000 7f39b03ce700 1 --2- 172.21.15.156:0/1491311414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a80c73c0 0x7f39a80afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.789+0000 7f39b03ce700 1 --2- 172.21.15.156:0/1491311414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39940037f0 0x7f39a8155310 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.789+0000 7f39b03ce700 1 --2- 172.21.15.156:0/1491311414 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f398c068340 0x7f398c06a7f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.789+0000 7f39b03ce700 1 --2- 172.21.15.156:0/1491311414 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39a80c6f50 0x7f39a80af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.789+0000 7f39b03ce700 1 -- 172.21.15.156:0/1491311414 >> 172.21.15.156:0/1491311414 conn(0x7f39a80cb670 msgr2=0x7f39a80cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.789+0000 7f39b03ce700 1 -- 172.21.15.156:0/1491311414 shutdown_connections 2023-11-28T11:08:28.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.789+0000 7f39b03ce700 1 -- 172.21.15.156:0/1491311414 wait complete. 2023-11-28T11:08:28.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.791+0000 7fec82840700 1 -- 172.21.15.156:0/3989325461 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fec68068340 msgr2=0x7fec6806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.791+0000 7fec82840700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fec68068340 0x7fec6806a7f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0090b0 tx=0x7fec6c008030 comp rx=0 tx=0).stop 2023-11-28T11:08:28.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.791+0000 7fec82840700 1 -- 172.21.15.156:0/3989325461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec7c0cc730 msgr2=0x7fec7c142990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.791+0000 7fec82840700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec7c0cc730 0x7fec7c142990 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec7000b530 tx=0x7fec7000b840 comp rx=0 tx=0).stop 2023-11-28T11:08:28.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.791+0000 7fec82840700 1 -- 172.21.15.156:0/3989325461 shutdown_connections 2023-11-28T11:08:28.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.791+0000 7fec82840700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec7c160f00 0x7fec7c0c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.791+0000 7fec82840700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fec68068340 0x7fec6806a7f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.791+0000 7fec82840700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec7c0cc730 0x7fec7c142990 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.791+0000 7fec82840700 1 --2- 172.21.15.156:0/3989325461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec7c160b30 0x7fec7c147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.791+0000 7fec82840700 1 -- 172.21.15.156:0/3989325461 >> 172.21.15.156:0/3989325461 conn(0x7fec7c0cb670 msgr2=0x7fec7c06bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.791+0000 7fec82840700 1 -- 172.21.15.156:0/3989325461 shutdown_connections 2023-11-28T11:08:28.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.791+0000 7fec82840700 1 -- 172.21.15.156:0/3989325461 wait complete. 2023-11-28T11:08:28.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.796+0000 7f2ea2053700 1 -- 172.21.15.156:0/2750177424 shutdown_connections 2023-11-28T11:08:28.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.796+0000 7f2ea2053700 1 --2- 172.21.15.156:0/2750177424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c160f00 0x7f2e9c201890 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.796+0000 7f2ea2053700 1 --2- 172.21.15.156:0/2750177424 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e7c0683c0 0x7f2e7c06a870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.796+0000 7f2ea2053700 1 --2- 172.21.15.156:0/2750177424 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c0cc730 0x7f2e9c0bcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.796+0000 7f2ea2053700 1 --2- 172.21.15.156:0/2750177424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c160b30 0x7f2e9c0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.796+0000 7f2ea2053700 1 -- 172.21.15.156:0/2750177424 >> 172.21.15.156:0/2750177424 conn(0x7f2e9c0cb670 msgr2=0x7f2e9c0ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.796+0000 7f2ea2053700 1 -- 172.21.15.156:0/2750177424 shutdown_connections 2023-11-28T11:08:28.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.796+0000 7f2ea2053700 1 -- 172.21.15.156:0/2750177424 wait complete. 2023-11-28T11:08:28.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.797+0000 7fb0ef262700 1 -- 172.21.15.156:0/2003404258 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb0c80055a0 con 0x7fb0e8158890 2023-11-28T11:08:28.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.798+0000 7fb0e57fa700 1 -- 172.21.15.156:0/2003404258 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb0e0058640 con 0x7fb0e8158890 2023-11-28T11:08:28.799 INFO:teuthology.orchestra.run.smithi156.stdout:55834574855 2023-11-28T11:08:28.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.802+0000 7fb0ef262700 1 -- 172.21.15.156:0/2003404258 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb0cc068340 msgr2=0x7fb0cc06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.802+0000 7fb0ef262700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb0cc068340 0x7fb0cc06a7f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4005910 tx=0x7fb0d4009040 comp rx=0 tx=0).stop 2023-11-28T11:08:28.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.802+0000 7fb0ef262700 1 -- 172.21.15.156:0/2003404258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e8158890 msgr2=0x7fb0e815b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.802+0000 7fb0ef262700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e8158890 0x7fb0e815b460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8067360 tx=0x7fb0e0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.802+0000 7fb0ef262700 1 -- 172.21.15.156:0/2003404258 shutdown_connections 2023-11-28T11:08:28.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.802+0000 7fb0ef262700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e8158890 0x7fb0e815b460 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.802+0000 7fb0ef262700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0e806d4f0 0x7fb0e81f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.802+0000 7fb0ef262700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb0cc068340 0x7fb0cc06a7f0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4005910 tx=0x7fb0d4009040 comp rx=0 tx=0).stop 2023-11-28T11:08:28.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.802+0000 7fb0ef262700 1 --2- 172.21.15.156:0/2003404258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0e80b0bb0 0x7fb0e815af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.802+0000 7fb0ef262700 1 -- 172.21.15.156:0/2003404258 >> 172.21.15.156:0/2003404258 conn(0x7fb0e80cb670 msgr2=0x7fb0e80cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.804+0000 7fb0ef262700 1 -- 172.21.15.156:0/2003404258 shutdown_connections 2023-11-28T11:08:28.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.804+0000 7fb0ef262700 1 -- 172.21.15.156:0/2003404258 wait complete. 2023-11-28T11:08:28.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.804+0000 7f61167fc700 1 -- 172.21.15.156:0/164632743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f61100055a0 con 0x7f613015e540 2023-11-28T11:08:28.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.810+0000 7f612cff9700 1 -- 172.21.15.156:0/164632743 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6124053600 con 0x7f613015e540 2023-11-28T11:08:28.811 INFO:teuthology.orchestra.run.smithi156.stdout:55834574855 2023-11-28T11:08:28.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.1 2023-11-28T11:08:28.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.813+0000 7f61167fc700 1 -- 172.21.15.156:0/164632743 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f61180684a0 msgr2=0x7f611806a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.813+0000 7f61167fc700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f61180684a0 0x7f611806a950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6120006ca0 tx=0x7f612000d750 comp rx=0 tx=0).stop 2023-11-28T11:08:28.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.813+0000 7f61167fc700 1 -- 172.21.15.156:0/164632743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f613015e540 msgr2=0x7f61300ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.813+0000 7f61167fc700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f613015e540 0x7f61300ce8b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6124009c40 tx=0x7f6124004750 comp rx=0 tx=0).stop 2023-11-28T11:08:28.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.813+0000 7f61167fc700 1 -- 172.21.15.156:0/164632743 shutdown_connections 2023-11-28T11:08:28.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.813+0000 7f61167fc700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61300cedf0 0x7f61301f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.813+0000 7f61167fc700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f61180684a0 0x7f611806a950 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.813+0000 7f61167fc700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f613006e500 0x7f61301597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.813+0000 7f61167fc700 1 --2- 172.21.15.156:0/164632743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f613015e540 0x7f61300ce8b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.813+0000 7f61167fc700 1 -- 172.21.15.156:0/164632743 >> 172.21.15.156:0/164632743 conn(0x7f61300b12b0 msgr2=0x7f61300caf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.814+0000 7f61167fc700 1 -- 172.21.15.156:0/164632743 shutdown_connections 2023-11-28T11:08:28.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:28.814+0000 7f61167fc700 1 -- 172.21.15.156:0/164632743 wait complete. 2023-11-28T11:08:28.817 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.4 2023-11-28T11:08:28.817 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.3 2023-11-28T11:08:28.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2023-11-28T11:08:28.819 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:28.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.5 2023-11-28T11:08:28.826 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-11-28T11:08:28.827 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:28.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-11-28T11:08:28.829 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:29.787 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T11:08:29.814 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T11:08:29.817 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T11:08:29.818 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T11:08:29.821 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T11:08:29.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.965+0000 7f144144e700 1 Processor -- start 2023-11-28T11:08:29.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.965+0000 7f144144e700 1 -- start start 2023-11-28T11:08:29.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.966+0000 7f144144e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 0x7f143c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.966+0000 7f144144e700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f143c06ba90 0x7f143c0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.966+0000 7f144144e700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143c0cc5f0 con 0x7f143c0cbdf0 2023-11-28T11:08:29.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.966+0000 7f144144e700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f143c15d470 con 0x7f143c06ba90 2023-11-28T11:08:29.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.966+0000 7f144144e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c15d750 con 0x7f143c15cef0 2023-11-28T11:08:29.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f143affd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 0x7f143c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f143b7fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f143c06ba90 0x7f143c0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f143affd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 0x7f143c0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36096/0 (socket says 172.21.15.156:36096) 2023-11-28T11:08:29.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f143affd700 1 -- 172.21.15.156:0/308101640 learned_addr learned my addr 172.21.15.156:0/308101640 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:29.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f143b7fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f143c06ba90 0x7f143c0ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:43248/0 (socket says 172.21.15.156:43248) 2023-11-28T11:08:29.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f143a7fc700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f143c0cbdf0 0x7f143c06b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:57474/0 (socket says 172.21.15.156:57474) 2023-11-28T11:08:29.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f143b7fe700 1 -- 172.21.15.156:0/308101640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 msgr2=0x7f143c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:29.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f143b7fe700 1 --2- 172.21.15.156:0/308101640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 0x7f143c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f143b7fe700 1 -- 172.21.15.156:0/308101640 >> v1:172.21.15.186:6789/0 conn(0x7f143c0cbdf0 legacy=0x7f143c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f143b7fe700 1 -- 172.21.15.156:0/308101640 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c15da30 con 0x7f143c06ba90 2023-11-28T11:08:29.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f143affd700 1 --2- 172.21.15.156:0/308101640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 0x7f143c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:29.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f143b7fe700 1 --2- 172.21.15.156:0/308101640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f143c06ba90 0x7f143c0ca650 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1434009d90 tx=0x7f143400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c4ca72727d7368 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f1439ffb700 1 -- 172.21.15.156:0/308101640 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143400e070 con 0x7f143c06ba90 2023-11-28T11:08:29.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f1439ffb700 1 -- 172.21.15.156:0/308101640 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143400bb20 con 0x7f143c06ba90 2023-11-28T11:08:29.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.967+0000 7f1439ffb700 1 -- 172.21.15.156:0/308101640 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434012430 con 0x7f143c06ba90 2023-11-28T11:08:29.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.968+0000 7f144144e700 1 -- 172.21.15.156:0/308101640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f143c06ba90 msgr2=0x7f143c0ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.968+0000 7f144144e700 1 --2- 172.21.15.156:0/308101640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f143c06ba90 0x7f143c0ca650 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1434009d90 tx=0x7f143400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:29.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.968+0000 7f144144e700 1 -- 172.21.15.156:0/308101640 shutdown_connections 2023-11-28T11:08:29.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.968+0000 7f144144e700 1 --2- 172.21.15.156:0/308101640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f143c06ba90 0x7f143c0ca650 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.968+0000 7f144144e700 1 --2- 172.21.15.156:0/308101640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 0x7f143c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.968+0000 7f144144e700 1 -- 172.21.15.156:0/308101640 >> 172.21.15.156:0/308101640 conn(0x7f143c0d0000 msgr2=0x7f143c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.968+0000 7f144144e700 1 -- 172.21.15.156:0/308101640 shutdown_connections 2023-11-28T11:08:29.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.968+0000 7f144144e700 1 -- 172.21.15.156:0/308101640 wait complete. 2023-11-28T11:08:29.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.969+0000 7f42a0dd3700 1 Processor -- start 2023-11-28T11:08:29.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.969+0000 7f42a0dd3700 1 -- start start 2023-11-28T11:08:29.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.970+0000 7f144144e700 1 Processor -- start 2023-11-28T11:08:29.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.970+0000 7f144144e700 1 -- start start 2023-11-28T11:08:29.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.970+0000 7f42a0dd3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f429c0cec70 0x7f429c065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.970+0000 7f42a0dd3700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429c066580 con 0x7f429c06ec90 2023-11-28T11:08:29.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.970+0000 7f42a0dd3700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429c0cded0 con 0x7f429c0cb0e0 2023-11-28T11:08:29.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.970+0000 7f42a0dd3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f429c0ce1b0 con 0x7f429c0cec70 2023-11-28T11:08:29.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f429ad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f429c0cec70 0x7f429c065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f429ad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f429c0cec70 0x7f429c065ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60474/0 (socket says 172.21.15.156:60474) 2023-11-28T11:08:29.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f429ad9d700 1 -- 172.21.15.156:0/1886728788 learned_addr learned my addr 172.21.15.156:0/1886728788 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:29.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f429ad9d700 1 -- 172.21.15.156:0/1886728788 >> v1:172.21.15.186:6789/0 conn(0x7f429c0cb0e0 legacy=0x7f429c06d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f429ad9d700 1 -- 172.21.15.156:0/1886728788 >> v1:172.21.15.156:6789/0 conn(0x7f429c06ec90 legacy=0x7f429c0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f429ad9d700 1 -- 172.21.15.156:0/1886728788 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c0ce490 con 0x7f429c0cec70 2023-11-28T11:08:29.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f429ad9d700 1 --2- 172.21.15.156:0/1886728788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c0cec70 0x7f429c065ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4294009e50 tx=0x7f429400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444ee6581eaeea1a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f144144e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06ba90 0x7f143c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f144144e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f143c0cbdf0 0x7f143c0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f144144e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 0x7f143c15a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f144144e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c0c61f0 con 0x7f143c06ba90 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f144144e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c0c7c10 con 0x7f143c0cbdf0 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.971+0000 7f144144e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c0c7ef0 con 0x7f143c15cef0 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.972+0000 7f143affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06ba90 0x7f143c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.972+0000 7f143affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06ba90 0x7f143c0ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60476/0 (socket says 172.21.15.156:60476) 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.972+0000 7f143affd700 1 -- 172.21.15.156:0/917804887 learned_addr learned my addr 172.21.15.156:0/917804887 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.972+0000 7f143b7fe700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 0x7f143c15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.972+0000 7f143affd700 1 -- 172.21.15.156:0/917804887 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 msgr2=0x7f143c15a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.972+0000 7f143affd700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 0x7f143c15a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.972+0000 7f143affd700 1 -- 172.21.15.156:0/917804887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f143c0cbdf0 msgr2=0x7f143c0cd280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.972+0000 7f143affd700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f143c0cbdf0 0x7f143c0cd280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.972+0000 7f143affd700 1 -- 172.21.15.156:0/917804887 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c009550 con 0x7f143c06ba90 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.972+0000 7f143b7fe700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 0x7f143c15a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.972+0000 7f143affd700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06ba90 0x7f143c0ccc30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f142c00e9b0 tx=0x7f142c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.974+0000 7fd7a659f700 1 Processor -- start 2023-11-28T11:08:29.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.974+0000 7f429959a700 1 -- 172.21.15.156:0/1886728788 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400e070 con 0x7f429c0cec70 2023-11-28T11:08:29.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.974+0000 7f14237fe700 1 -- 172.21.15.156:0/917804887 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c00c850 con 0x7f143c06ba90 2023-11-28T11:08:29.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.974+0000 7f144144e700 1 -- 172.21.15.156:0/917804887 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1434009a70 con 0x7f143c06ba90 2023-11-28T11:08:29.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.974+0000 7f144144e700 1 -- 172.21.15.156:0/917804887 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143c15dff0 con 0x7f143c06ba90 2023-11-28T11:08:29.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.974+0000 7fd7a659f700 1 -- start start 2023-11-28T11:08:29.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a659f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd7a006a270 0x7fd7a00b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a659f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a00ac8b0 0x7fd7a00cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a659f700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a00b4fb0 con 0x7fd7a00b4b10 2023-11-28T11:08:29.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a659f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a0071730 con 0x7fd7a006a270 2023-11-28T11:08:29.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a659f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a00cf830 con 0x7fd7a00ac8b0 2023-11-28T11:08:29.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.974+0000 7f429959a700 1 -- 172.21.15.156:0/1886728788 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429400b9c0 con 0x7f429c0cec70 2023-11-28T11:08:29.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.974+0000 7f429959a700 1 -- 172.21.15.156:0/1886728788 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400bd70 con 0x7f429c0cec70 2023-11-28T11:08:29.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7f42a0dd3700 1 -- 172.21.15.156:0/1886728788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c0cec70 msgr2=0x7f429c065ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7f42a0dd3700 1 --2- 172.21.15.156:0/1886728788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c0cec70 0x7f429c065ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4294009e50 tx=0x7f429400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:29.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a559d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd7a006a270 0x7fd7a00b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a559d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd7a006a270 0x7fd7a00b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60482/0 (socket says 172.21.15.156:60482) 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a559d700 1 -- 172.21.15.156:0/1893456144 learned_addr learned my addr 172.21.15.156:0/1893456144 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7f14237fe700 1 -- 172.21.15.156:0/917804887 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c00c9b0 con 0x7f143c06ba90 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7f14237fe700 1 -- 172.21.15.156:0/917804887 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c00cd20 con 0x7f143c06ba90 2023-11-28T11:08:29.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a5d9e700 1 --2- 172.21.15.156:0/1893456144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a00ac8b0 0x7fd7a00cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7f42a0dd3700 1 -- 172.21.15.156:0/1886728788 shutdown_connections 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7f42a0dd3700 1 --2- 172.21.15.156:0/1886728788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c0cec70 0x7f429c065ef0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7f42a0dd3700 1 -- 172.21.15.156:0/1886728788 >> 172.21.15.156:0/1886728788 conn(0x7f429c0b12b0 msgr2=0x7f429c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7f42a0dd3700 1 -- 172.21.15.156:0/1886728788 shutdown_connections 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7f42a0dd3700 1 -- 172.21.15.156:0/1886728788 wait complete. 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a559d700 1 -- 172.21.15.156:0/1893456144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a00ac8b0 msgr2=0x7fd7a00cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a559d700 1 --2- 172.21.15.156:0/1893456144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a00ac8b0 0x7fd7a00cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a559d700 1 -- 172.21.15.156:0/1893456144 >> v1:172.21.15.186:6789/0 conn(0x7fd7a00b4b10 legacy=0x7fd7a006a640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a559d700 1 -- 172.21.15.156:0/1893456144 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a0071940 con 0x7fd7a006a270 2023-11-28T11:08:29.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.975+0000 7fd7a5d9e700 1 --2- 172.21.15.156:0/1893456144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a00ac8b0 0x7fd7a00cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:29.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.976+0000 7f144144e700 1 -- 172.21.15.156:0/917804887 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141c0052f0 con 0x7f143c06ba90 2023-11-28T11:08:29.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.977+0000 7fd7a559d700 1 --2- 172.21.15.156:0/1893456144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a006a270 0x7fd7a00b3a70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0098f0 tx=0x7fd79c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b287bedc91eac58e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.977+0000 7fd797fff700 1 -- 172.21.15.156:0/1893456144 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c004230 con 0x7fd7a006a270 2023-11-28T11:08:29.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.977+0000 7f14237fe700 1 -- 172.21.15.156:0/917804887 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f142c019030 con 0x7f143c06ba90 2023-11-28T11:08:29.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.977+0000 7fd797fff700 1 -- 172.21.15.156:0/1893456144 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c004390 con 0x7fd7a006a270 2023-11-28T11:08:29.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.977+0000 7fd797fff700 1 -- 172.21.15.156:0/1893456144 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0036a0 con 0x7fd7a006a270 2023-11-28T11:08:29.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.977+0000 7f9cb540a700 1 Processor -- start 2023-11-28T11:08:29.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.977+0000 7f9cb540a700 1 -- start start 2023-11-28T11:08:29.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.977+0000 7f42a0dd3700 1 Processor -- start 2023-11-28T11:08:29.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.977+0000 7f42a0dd3700 1 -- start start 2023-11-28T11:08:29.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.977+0000 7f14237fe700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14280682f0 0x7f142806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.977+0000 7f14237fe700 1 -- 172.21.15.156:0/917804887 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f142c085cf0 con 0x7f143c06ba90 2023-11-28T11:08:29.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f143a7fc700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14280682f0 0x7f142806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f143a7fc700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14280682f0 0x7f142806a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f143c0ba790 tx=0x7f1430007a50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f42a0dd3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c06ec90 0x7f429c1505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f42a0dd3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f429c0cb0e0 0x7f429c0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f42a0dd3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f429c0cec70 0x7f429c0c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f42a0dd3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c1521a0 con 0x7f429c06ec90 2023-11-28T11:08:29.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f42a0dd3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c0c5ed0 con 0x7f429c0cb0e0 2023-11-28T11:08:29.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f42a0dd3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429c0c6890 con 0x7f429c0cec70 2023-11-28T11:08:29.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f429a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c06ec90 0x7f429c1505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f429a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c06ec90 0x7f429c1505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60490/0 (socket says 172.21.15.156:60490) 2023-11-28T11:08:29.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f429a59c700 1 -- 172.21.15.156:0/135797884 learned_addr learned my addr 172.21.15.156:0/135797884 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:29.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f9cb540a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00b0670 0x7f9cb00b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f9cb540a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9cb00c94c0 0x7f9cb00c9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f9cb540a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb00ccc00 con 0x7f9cb00cdc80 2023-11-28T11:08:29.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f9cb540a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb00b1020 con 0x7f9cb00c94c0 2023-11-28T11:08:29.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.978+0000 7f9cb540a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb00ccec0 con 0x7f9cb00b0670 2023-11-28T11:08:29.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7fd7a659f700 1 -- 172.21.15.156:0/1893456144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a006a270 msgr2=0x7fd7a00b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7fd7a659f700 1 --2- 172.21.15.156:0/1893456144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a006a270 0x7fd7a00b3a70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0098f0 tx=0x7fd79c009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:29.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f429ad9d700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f429c0cec70 0x7f429c0c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f9cae7fc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00b0670 0x7f9cb00b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f9cae7fc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00b0670 0x7f9cb00b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36126/0 (socket says 172.21.15.156:36126) 2023-11-28T11:08:29.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f9cae7fc700 1 -- 172.21.15.156:0/1839169030 learned_addr learned my addr 172.21.15.156:0/1839169030 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:29.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f429a59c700 1 -- 172.21.15.156:0/135797884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f429c0cec70 msgr2=0x7f429c0c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f429a59c700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f429c0cec70 0x7f429c0c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f429a59c700 1 -- 172.21.15.156:0/135797884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f429c0cb0e0 msgr2=0x7f429c0c5330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:08:29.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f429a59c700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f429c0cb0e0 0x7f429c0c5330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f429a59c700 1 -- 172.21.15.156:0/135797884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4294009b30 con 0x7f429c06ec90 2023-11-28T11:08:29.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f9cae7fc700 1 -- 172.21.15.156:0/1839169030 >> v2:172.21.15.156:3300/0 conn(0x7f9cb00c94c0 msgr2=0x7f9cb00c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:29.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f9cae7fc700 1 --2- 172.21.15.156:0/1839169030 >> v2:172.21.15.156:3300/0 conn(0x7f9cb00c94c0 0x7f9cb00c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f9cae7fc700 1 -- 172.21.15.156:0/1839169030 >> v1:172.21.15.156:6789/0 conn(0x7f9cb00cdc80 legacy=0x7f9cb0159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f9cae7fc700 1 -- 172.21.15.156:0/1839169030 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb00cd1a0 con 0x7f9cb00b0670 2023-11-28T11:08:29.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f429a59c700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c06ec90 0x7f429c1505f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c450 tx=0x7f428c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.979+0000 7f9cae7fc700 1 --2- 172.21.15.156:0/1839169030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00b0670 0x7f9cb00b0ae0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40098f0 tx=0x7f9ca4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d457e5225ef3a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.981+0000 7f9cadffb700 1 -- 172.21.15.156:0/1839169030 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca400b8a0 con 0x7f9cb00b0670 2023-11-28T11:08:29.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.981+0000 7f9cadffb700 1 -- 172.21.15.156:0/1839169030 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca400ba00 con 0x7f9cb00b0670 2023-11-28T11:08:29.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.981+0000 7f9cadffb700 1 -- 172.21.15.156:0/1839169030 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca400bd90 con 0x7f9cb00b0670 2023-11-28T11:08:29.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.981+0000 7f9cb540a700 1 -- 172.21.15.156:0/1839169030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00b0670 msgr2=0x7f9cb00b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.981+0000 7f9cb540a700 1 --2- 172.21.15.156:0/1839169030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00b0670 0x7f9cb00b0ae0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40098f0 tx=0x7f9ca4009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:29.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.981+0000 7f9cb540a700 1 -- 172.21.15.156:0/1839169030 shutdown_connections 2023-11-28T11:08:29.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.981+0000 7f9cb540a700 1 --2- 172.21.15.156:0/1839169030 >> v2:172.21.15.156:3300/0 conn(0x7f9cb00c94c0 0x7f9cb00c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.981+0000 7f9cb540a700 1 --2- 172.21.15.156:0/1839169030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00b0670 0x7f9cb00b0ae0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.981+0000 7f9cb540a700 1 -- 172.21.15.156:0/1839169030 >> 172.21.15.156:0/1839169030 conn(0x7f9cb00cb670 msgr2=0x7f9cb00cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.981+0000 7f9cb540a700 1 -- 172.21.15.156:0/1839169030 shutdown_connections 2023-11-28T11:08:29.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.981+0000 7f9cb540a700 1 -- 172.21.15.156:0/1839169030 wait complete. 2023-11-28T11:08:29.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.983+0000 7fd7a659f700 1 -- 172.21.15.156:0/1893456144 shutdown_connections 2023-11-28T11:08:29.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.983+0000 7fd7a659f700 1 --2- 172.21.15.156:0/1893456144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a00ac8b0 0x7fd7a00cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.983+0000 7fd7a659f700 1 --2- 172.21.15.156:0/1893456144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a006a270 0x7fd7a00b3a70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.983+0000 7fd7a659f700 1 -- 172.21.15.156:0/1893456144 >> 172.21.15.156:0/1893456144 conn(0x7fd7a006b380 msgr2=0x7fd7a006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.983+0000 7f9cb540a700 1 Processor -- start 2023-11-28T11:08:29.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.983+0000 7f9cb540a700 1 -- start start 2023-11-28T11:08:29.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.983+0000 7f428affd700 1 -- 172.21.15.156:0/135797884 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c0052d0 con 0x7f429c06ec90 2023-11-28T11:08:29.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.983+0000 7f428affd700 1 -- 172.21.15.156:0/135797884 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428c005430 con 0x7f429c06ec90 2023-11-28T11:08:29.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.983+0000 7f14237fe700 1 -- 172.21.15.156:0/917804887 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f142c052470 con 0x7f143c06ba90 2023-11-28T11:08:29.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.983+0000 7f42a0dd3700 1 -- 172.21.15.156:0/135797884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429c0cb6c0 con 0x7f429c06ec90 2023-11-28T11:08:29.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.983+0000 7f428affd700 1 -- 172.21.15.156:0/135797884 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c003880 con 0x7f429c06ec90 2023-11-28T11:08:29.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f42a0dd3700 1 -- 172.21.15.156:0/135797884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429c0ce770 con 0x7f429c06ec90 2023-11-28T11:08:29.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.985+0000 7fd7a659f700 1 -- 172.21.15.156:0/1893456144 shutdown_connections 2023-11-28T11:08:29.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.983+0000 7f9cb540a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cb00b0670 0x7f9cb015ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9cb540a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb00c94c0 0x7f9cb0070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9cb540a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00cdc80 0x7f9cb00c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9cb540a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb0155c90 con 0x7f9cb00c94c0 2023-11-28T11:08:29.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9cb540a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb00b1c60 con 0x7f9cb00b0670 2023-11-28T11:08:29.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9cb540a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb015c100 con 0x7f9cb00cdc80 2023-11-28T11:08:29.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9caf7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00cdc80 0x7f9cb00c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9caeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cb00b0670 0x7f9cb015ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9caeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cb00b0670 0x7f9cb015ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:43282/0 (socket says 172.21.15.156:43282) 2023-11-28T11:08:29.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9caf7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00cdc80 0x7f9cb00c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36134/0 (socket says 172.21.15.156:36134) 2023-11-28T11:08:29.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9caeffd700 1 -- 172.21.15.156:0/466993441 learned_addr learned my addr 172.21.15.156:0/466993441 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:29.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9caeffd700 1 -- 172.21.15.156:0/466993441 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00cdc80 msgr2=0x7f9cb00c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9caeffd700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00cdc80 0x7f9cb00c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9caeffd700 1 -- 172.21.15.156:0/466993441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb00c94c0 msgr2=0x7f9cb0070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9caeffd700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb00c94c0 0x7f9cb0070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9caeffd700 1 -- 172.21.15.156:0/466993441 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca4009550 con 0x7f9cb00b0670 2023-11-28T11:08:30.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.984+0000 7f9caf7fe700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00cdc80 0x7f9cb00c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:30.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.985+0000 7fd7a659f700 1 -- 172.21.15.156:0/1893456144 wait complete. 2023-11-28T11:08:30.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.985+0000 7f428affd700 1 -- 172.21.15.156:0/135797884 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f428c019030 con 0x7f429c06ec90 2023-11-28T11:08:30.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.986+0000 7f9caeffd700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cb00b0670 0x7f9cb015ba70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000c4e0 tx=0x7f9ca000c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.986+0000 7f428affd700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f427c0682f0 0x7f427c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.986+0000 7f428affd700 1 -- 172.21.15.156:0/135797884 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f428c084e60 con 0x7f429c06ec90 2023-11-28T11:08:30.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.986+0000 7f4299d9b700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f427c0682f0 0x7f427c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.986+0000 7f42a0dd3700 1 -- 172.21.15.156:0/135797884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429c06f060 con 0x7f429c06ec90 2023-11-28T11:08:30.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.986+0000 7f4299d9b700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f427c0682f0 0x7f427c06a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4290000f60 tx=0x7f4290007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.987+0000 7f9c977fe700 1 -- 172.21.15.156:0/466993441 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00052d0 con 0x7f9cb00b0670 2023-11-28T11:08:30.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.987+0000 7fd7a659f700 1 Processor -- start 2023-11-28T11:08:30.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.987+0000 7f9c977fe700 1 -- 172.21.15.156:0/466993441 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca0005430 con 0x7f9cb00b0670 2023-11-28T11:08:30.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.987+0000 7f9cb540a700 1 -- 172.21.15.156:0/466993441 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb00cd760 con 0x7f9cb00b0670 2023-11-28T11:08:30.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.987+0000 7f9c977fe700 1 -- 172.21.15.156:0/466993441 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00038c0 con 0x7f9cb00b0670 2023-11-28T11:08:30.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.987+0000 7f9cb540a700 1 -- 172.21.15.156:0/466993441 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb00cd480 con 0x7f9cb00b0670 2023-11-28T11:08:30.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.987+0000 7fd7a659f700 1 -- start start 2023-11-28T11:08:30.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.988+0000 7fd7a659f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a006a270 0x7fd7a015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.988+0000 7fd7a659f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a00ac8b0 0x7fd7a0152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.988+0000 7fd7a659f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a00b4b10 0x7fd7a015de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.988+0000 7fd7a659f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a015ef10 con 0x7fd7a006a270 2023-11-28T11:08:30.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.988+0000 7fd7a659f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a01f6760 con 0x7fd7a00b4b10 2023-11-28T11:08:30.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.988+0000 7fd7a659f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a015ec30 con 0x7fd7a00ac8b0 2023-11-28T11:08:30.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.988+0000 7fd7a559d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a006a270 0x7fd7a015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.988+0000 7fd7a559d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a006a270 0x7fd7a015d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60508/0 (socket says 172.21.15.156:60508) 2023-11-28T11:08:30.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.988+0000 7fd7a559d700 1 -- 172.21.15.156:0/366813241 learned_addr learned my addr 172.21.15.156:0/366813241 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:30.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.989+0000 7fd7a559d700 1 -- 172.21.15.156:0/366813241 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a00ac8b0 msgr2=0x7fd7a0152370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:08:30.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.989+0000 7fd7a559d700 1 --2- 172.21.15.156:0/366813241 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a00ac8b0 0x7fd7a0152370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.989+0000 7fd7a559d700 1 -- 172.21.15.156:0/366813241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a00b4b10 msgr2=0x7fd7a015de20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:08:30.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.989+0000 7fd7a559d700 1 --2- 172.21.15.156:0/366813241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a00b4b10 0x7fd7a015de20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.989+0000 7fd7a559d700 1 -- 172.21.15.156:0/366813241 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd798009a70 con 0x7fd7a006a270 2023-11-28T11:08:30.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.989+0000 7f9c977fe700 1 -- 172.21.15.156:0/466993441 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ca0019030 con 0x7f9cb00b0670 2023-11-28T11:08:30.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.989+0000 7fd7a559d700 1 --2- 172.21.15.156:0/366813241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a006a270 0x7fd7a015d6a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009130 tx=0x7fd79c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.989+0000 7f9cb540a700 1 -- 172.21.15.156:0/466993441 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb0153ad0 con 0x7f9cb00b0670 2023-11-28T11:08:30.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.989+0000 7f9c977fe700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9c9c0682f0 0x7f9c9c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.989+0000 7f9c977fe700 1 -- 172.21.15.156:0/466993441 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ca0084f10 con 0x7f9cb00b0670 2023-11-28T11:08:30.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.989+0000 7f9cae7fc700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9c9c0682f0 0x7f9c9c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.990+0000 7f9cae7fc700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9c9c0682f0 0x7f9c9c06a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0142e80 tx=0x7f9ca4010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.992+0000 7fd795ffb700 1 -- 172.21.15.156:0/366813241 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c011630 con 0x7fd7a006a270 2023-11-28T11:08:30.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.992+0000 7f428affd700 1 -- 172.21.15.156:0/135797884 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f428c04b5e0 con 0x7f429c06ec90 2023-11-28T11:08:30.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.992+0000 7fd795ffb700 1 -- 172.21.15.156:0/366813241 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c011790 con 0x7fd7a006a270 2023-11-28T11:08:30.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.992+0000 7fd795ffb700 1 -- 172.21.15.156:0/366813241 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c011b00 con 0x7fd7a006a270 2023-11-28T11:08:30.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.992+0000 7fd7a659f700 1 -- 172.21.15.156:0/366813241 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79c009550 con 0x7fd7a006a270 2023-11-28T11:08:30.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.993+0000 7fd7a659f700 1 -- 172.21.15.156:0/366813241 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a00cfa40 con 0x7fd7a006a270 2023-11-28T11:08:30.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.994+0000 7fd7a659f700 1 -- 172.21.15.156:0/366813241 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a0071c20 con 0x7fd7a006a270 2023-11-28T11:08:30.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.994+0000 7f9c977fe700 1 -- 172.21.15.156:0/466993441 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9ca0051770 con 0x7f9cb00b0670 2023-11-28T11:08:30.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.994+0000 7fd795ffb700 1 -- 172.21.15.156:0/366813241 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd79c017020 con 0x7fd7a006a270 2023-11-28T11:08:30.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.995+0000 7fd795ffb700 1 --2- 172.21.15.156:0/366813241 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7840682f0 0x7fd78406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.995+0000 7fd795ffb700 1 -- 172.21.15.156:0/366813241 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd79c0872f0 con 0x7fd7a006a270 2023-11-28T11:08:30.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.995+0000 7fd7a4d9c700 1 --2- 172.21.15.156:0/366813241 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7840682f0 0x7fd78406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:29.995+0000 7fd7a4d9c700 1 --2- 172.21.15.156:0/366813241 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7840682f0 0x7fd78406a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0159dc0 tx=0x7fd790006c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.003+0000 7fd795ffb700 1 -- 172.21.15.156:0/366813241 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd79c053b50 con 0x7fd7a006a270 2023-11-28T11:08:30.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.011+0000 7f68909c3700 1 Processor -- start 2023-11-28T11:08:30.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.011+0000 7f68909c3700 1 -- start start 2023-11-28T11:08:30.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.012+0000 7f68909c3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688c06e9f0 0x7f688c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.012+0000 7f68909c3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f688c06d0a0 0x7f688c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.012+0000 7f68909c3700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688c06ae60 con 0x7f688c06b830 2023-11-28T11:08:30.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.012+0000 7f68909c3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f688c06a270 con 0x7f688c06d0a0 2023-11-28T11:08:30.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.012+0000 7f68909c3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c06a530 con 0x7f688c06e9f0 2023-11-28T11:08:30.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688ad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f688c06d0a0 0x7f688c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688ad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f688c06d0a0 0x7f688c0ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60520/0 (socket says 172.21.15.156:60520) 2023-11-28T11:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688ad9d700 1 -- 172.21.15.156:0/3942548909 learned_addr learned my addr 172.21.15.156:0/3942548909 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688a59c700 1 --2- 172.21.15.156:0/3942548909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688c06e9f0 0x7f688c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f6889d9b700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f688c06b830 0x7f688c06c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:57524/0 (socket says 172.21.15.156:57524) 2023-11-28T11:08:30.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688959a700 1 -- 172.21.15.156:0/3942548909 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205280940 0 0) 0x7f688c06ae60 con 0x7f688c06b830 2023-11-28T11:08:30.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688ad9d700 1 -- 172.21.15.156:0/3942548909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688c06e9f0 msgr2=0x7f688c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:30.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688ad9d700 1 --2- 172.21.15.156:0/3942548909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688c06e9f0 0x7f688c06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688ad9d700 1 -- 172.21.15.156:0/3942548909 >> v1:172.21.15.186:6789/0 conn(0x7f688c06b830 legacy=0x7f688c06c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688ad9d700 1 -- 172.21.15.156:0/3942548909 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c06a810 con 0x7f688c06d0a0 2023-11-28T11:08:30.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688a59c700 1 --2- 172.21.15.156:0/3942548909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688c06e9f0 0x7f688c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:30.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688ad9d700 1 --2- 172.21.15.156:0/3942548909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f688c06d0a0 0x7f688c0ca240 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6884009e50 tx=0x7f688400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b431f2a3a5b25ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688959a700 1 -- 172.21.15.156:0/3942548909 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688400e070 con 0x7f688c06d0a0 2023-11-28T11:08:30.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688959a700 1 -- 172.21.15.156:0/3942548909 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688400b910 con 0x7f688c06d0a0 2023-11-28T11:08:30.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.013+0000 7f688959a700 1 -- 172.21.15.156:0/3942548909 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688400bcc0 con 0x7f688c06d0a0 2023-11-28T11:08:30.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.014+0000 7f68909c3700 1 -- 172.21.15.156:0/3942548909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f688c06d0a0 msgr2=0x7f688c0ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.014+0000 7f68909c3700 1 --2- 172.21.15.156:0/3942548909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f688c06d0a0 0x7f688c0ca240 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6884009e50 tx=0x7f688400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:30.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.014+0000 7f68909c3700 1 -- 172.21.15.156:0/3942548909 shutdown_connections 2023-11-28T11:08:30.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.014+0000 7f68909c3700 1 --2- 172.21.15.156:0/3942548909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f688c06d0a0 0x7f688c0ca240 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.014+0000 7f68909c3700 1 --2- 172.21.15.156:0/3942548909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688c06e9f0 0x7f688c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.014+0000 7f68909c3700 1 -- 172.21.15.156:0/3942548909 >> 172.21.15.156:0/3942548909 conn(0x7f688c0b12b0 msgr2=0x7f688c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:30.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.014+0000 7f68909c3700 1 -- 172.21.15.156:0/3942548909 shutdown_connections 2023-11-28T11:08:30.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.014+0000 7f68909c3700 1 -- 172.21.15.156:0/3942548909 wait complete. 2023-11-28T11:08:30.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.016+0000 7f68909c3700 1 Processor -- start 2023-11-28T11:08:30.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.016+0000 7f68909c3700 1 -- start start 2023-11-28T11:08:30.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.016+0000 7f68909c3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f688c06b830 0x7f688c153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.016+0000 7f68909c3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688c06d0a0 0x7f688c157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.016+0000 7f68909c3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f688c06e9f0 0x7f688c158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.016+0000 7f68909c3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c0c7540 con 0x7f688c06b830 2023-11-28T11:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.016+0000 7f68909c3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c158570 con 0x7f688c06e9f0 2023-11-28T11:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.016+0000 7f68909c3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c0c6930 con 0x7f688c06d0a0 2023-11-28T11:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f688a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f688c06b830 0x7f688c153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f688a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f688c06b830 0x7f688c153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60522/0 (socket says 172.21.15.156:60522) 2023-11-28T11:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f688a59c700 1 -- 172.21.15.156:0/2932140857 learned_addr learned my addr 172.21.15.156:0/2932140857 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f6889d9b700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688c06d0a0 0x7f688c157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f688ad9d700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f688c06e9f0 0x7f688c158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f688a59c700 1 -- 172.21.15.156:0/2932140857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688c06d0a0 msgr2=0x7f688c157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f688a59c700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688c06d0a0 0x7f688c157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f688a59c700 1 -- 172.21.15.156:0/2932140857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f688c06e9f0 msgr2=0x7f688c158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f688a59c700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f688c06e9f0 0x7f688c158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f688a59c700 1 -- 172.21.15.156:0/2932140857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c0094a0 con 0x7f688c06b830 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f6889d9b700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688c06d0a0 0x7f688c157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f688a59c700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f688c06b830 0x7f688c153f30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f687c00b3f0 tx=0x7f687c00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f687affd700 1 -- 172.21.15.156:0/2932140857 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c019070 con 0x7f688c06b830 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f687affd700 1 -- 172.21.15.156:0/2932140857 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c01d410 con 0x7f688c06b830 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.017+0000 7f687affd700 1 -- 172.21.15.156:0/2932140857 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c01d780 con 0x7f688c06b830 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.018+0000 7f68909c3700 1 -- 172.21.15.156:0/2932140857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6884009b30 con 0x7f688c06b830 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.018+0000 7f68909c3700 1 -- 172.21.15.156:0/2932140857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688c06edc0 con 0x7f688c06b830 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.019+0000 7f68909c3700 1 -- 172.21.15.156:0/2932140857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f688c06aaf0 con 0x7f688c06b830 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.019+0000 7f687affd700 1 -- 172.21.15.156:0/2932140857 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f687c024030 con 0x7f688c06b830 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.020+0000 7f687affd700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f686c0682f0 0x7f686c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.020+0000 7f687affd700 1 -- 172.21.15.156:0/2932140857 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f687c08d8a0 con 0x7f688c06b830 2023-11-28T11:08:30.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.020+0000 7f6889d9b700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f686c0682f0 0x7f686c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.020+0000 7f6889d9b700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f686c0682f0 0x7f686c06a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f688c0ae840 tx=0x7f6880007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.024+0000 7f687affd700 1 -- 172.21.15.156:0/2932140857 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f687c05a100 con 0x7f688c06b830 2023-11-28T11:08:30.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.215+0000 7f68909c3700 1 -- 172.21.15.156:0/2932140857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f688c06ef50 con 0x7f688c06b830 2023-11-28T11:08:30.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.216+0000 7f687affd700 1 -- 172.21.15.156:0/2932140857 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f687c022090 con 0x7f688c06b830 2023-11-28T11:08:30.217 INFO:teuthology.orchestra.run.smithi156.stdout:55834574857 2023-11-28T11:08:30.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.219+0000 7f68909c3700 1 -- 172.21.15.156:0/2932140857 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f686c0682f0 msgr2=0x7f686c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.219+0000 7f68909c3700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f686c0682f0 0x7f686c06a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f688c0ae840 tx=0x7f6880007ce0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.219+0000 7f68909c3700 1 -- 172.21.15.156:0/2932140857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f688c06b830 msgr2=0x7f688c153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.219+0000 7f68909c3700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f688c06b830 0x7f688c153f30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f687c00b3f0 tx=0x7f687c00b7b0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.219+0000 7f68909c3700 1 -- 172.21.15.156:0/2932140857 shutdown_connections 2023-11-28T11:08:30.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.219+0000 7f68909c3700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f688c06e9f0 0x7f688c158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.219+0000 7f68909c3700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f688c06b830 0x7f688c153f30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.219+0000 7f68909c3700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f686c0682f0 0x7f686c06a7a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.219+0000 7f68909c3700 1 --2- 172.21.15.156:0/2932140857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688c06d0a0 0x7f688c157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.219+0000 7f68909c3700 1 -- 172.21.15.156:0/2932140857 >> 172.21.15.156:0/2932140857 conn(0x7f688c0b12b0 msgr2=0x7f688c065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:30.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.219+0000 7f68909c3700 1 -- 172.21.15.156:0/2932140857 shutdown_connections 2023-11-28T11:08:30.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.219+0000 7f68909c3700 1 -- 172.21.15.156:0/2932140857 wait complete. 2023-11-28T11:08:30.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.222+0000 7f144144e700 1 -- 172.21.15.156:0/917804887 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f141c0055a0 con 0x7f143c06ba90 2023-11-28T11:08:30.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.223+0000 7f14237fe700 1 -- 172.21.15.156:0/917804887 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f142c017090 con 0x7f143c06ba90 2023-11-28T11:08:30.224 INFO:teuthology.orchestra.run.smithi156.stdout:55834574855 2023-11-28T11:08:30.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.225+0000 7f144144e700 1 -- 172.21.15.156:0/917804887 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14280682f0 msgr2=0x7f142806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.225+0000 7f144144e700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14280682f0 0x7f142806a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f143c0ba790 tx=0x7f1430007a50 comp rx=0 tx=0).stop 2023-11-28T11:08:30.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.225+0000 7f144144e700 1 -- 172.21.15.156:0/917804887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06ba90 msgr2=0x7f143c0ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.225+0000 7f144144e700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06ba90 0x7f143c0ccc30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f142c00e9b0 tx=0x7f142c00ed70 comp rx=0 tx=0).stop 2023-11-28T11:08:30.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.226+0000 7f144144e700 1 -- 172.21.15.156:0/917804887 shutdown_connections 2023-11-28T11:08:30.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.226+0000 7f144144e700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14280682f0 0x7f142806a7a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.226+0000 7f144144e700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f143c15cef0 0x7f143c15a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.226+0000 7f144144e700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06ba90 0x7f143c0ccc30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.226+0000 7f144144e700 1 --2- 172.21.15.156:0/917804887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f143c0cbdf0 0x7f143c0cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.226+0000 7f144144e700 1 -- 172.21.15.156:0/917804887 >> 172.21.15.156:0/917804887 conn(0x7f143c0d0000 msgr2=0x7f143c0b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:30.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.226+0000 7f144144e700 1 -- 172.21.15.156:0/917804887 shutdown_connections 2023-11-28T11:08:30.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.226+0000 7f144144e700 1 -- 172.21.15.156:0/917804887 wait complete. 2023-11-28T11:08:30.240 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2023-11-28T11:08:30.240 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:30.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-11-28T11:08:30.241 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:30.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.241+0000 7f9cb540a700 1 -- 172.21.15.156:0/466993441 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f9cb00745c0 con 0x7f9cb00b0670 2023-11-28T11:08:30.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.242+0000 7f9c977fe700 1 -- 172.21.15.156:0/466993441 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ca0017090 con 0x7f9cb00b0670 2023-11-28T11:08:30.243 INFO:teuthology.orchestra.run.smithi156.stdout:55834574856 2023-11-28T11:08:30.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.244+0000 7f9cb540a700 1 -- 172.21.15.156:0/466993441 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9c9c0682f0 msgr2=0x7f9c9c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.244+0000 7f9cb540a700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9c9c0682f0 0x7f9c9c06a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0142e80 tx=0x7f9ca4010000 comp rx=0 tx=0).stop 2023-11-28T11:08:30.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.244+0000 7f9cb540a700 1 -- 172.21.15.156:0/466993441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cb00b0670 msgr2=0x7f9cb015ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.244+0000 7f9cb540a700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cb00b0670 0x7f9cb015ba70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000c4e0 tx=0x7f9ca000c7f0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.244+0000 7f9cb540a700 1 -- 172.21.15.156:0/466993441 shutdown_connections 2023-11-28T11:08:30.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.244+0000 7f9cb540a700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9c9c0682f0 0x7f9c9c06a7a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.244+0000 7f9cb540a700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9cb00cdc80 0x7f9cb00c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.244+0000 7f9cb540a700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cb00b0670 0x7f9cb015ba70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.244+0000 7f9cb540a700 1 --2- 172.21.15.156:0/466993441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb00c94c0 0x7f9cb0070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.244+0000 7f9cb540a700 1 -- 172.21.15.156:0/466993441 >> 172.21.15.156:0/466993441 conn(0x7f9cb00cb670 msgr2=0x7f9cb006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:30.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.244+0000 7f9cb540a700 1 -- 172.21.15.156:0/466993441 shutdown_connections 2023-11-28T11:08:30.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.244+0000 7f9cb540a700 1 -- 172.21.15.156:0/466993441 wait complete. 2023-11-28T11:08:30.257 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-11-28T11:08:30.257 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:30.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.328+0000 7f42a0dd3700 1 -- 172.21.15.156:0/135797884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f429c06e7e0 con 0x7f429c06ec90 2023-11-28T11:08:30.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.328+0000 7f428affd700 1 -- 172.21.15.156:0/135797884 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f428c017020 con 0x7f429c06ec90 2023-11-28T11:08:30.330 INFO:teuthology.orchestra.run.smithi156.stdout:55834574855 2023-11-28T11:08:30.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.330+0000 7f42a0dd3700 1 -- 172.21.15.156:0/135797884 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f427c0682f0 msgr2=0x7f427c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.330+0000 7f42a0dd3700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f427c0682f0 0x7f427c06a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4290000f60 tx=0x7f4290007a40 comp rx=0 tx=0).stop 2023-11-28T11:08:30.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.330+0000 7f42a0dd3700 1 -- 172.21.15.156:0/135797884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c06ec90 msgr2=0x7f429c1505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.330+0000 7f42a0dd3700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c06ec90 0x7f429c1505f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c450 tx=0x7f428c00c810 comp rx=0 tx=0).stop 2023-11-28T11:08:30.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.330+0000 7f42a0dd3700 1 -- 172.21.15.156:0/135797884 shutdown_connections 2023-11-28T11:08:30.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.330+0000 7f42a0dd3700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f427c0682f0 0x7f427c06a7a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.330+0000 7f42a0dd3700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f429c0cec70 0x7f429c0c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.330+0000 7f42a0dd3700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c06ec90 0x7f429c1505f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.330+0000 7f42a0dd3700 1 --2- 172.21.15.156:0/135797884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f429c0cb0e0 0x7f429c0c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.330+0000 7f42a0dd3700 1 -- 172.21.15.156:0/135797884 >> 172.21.15.156:0/135797884 conn(0x7f429c0b12b0 msgr2=0x7f429c0713f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:30.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.330+0000 7f42a0dd3700 1 -- 172.21.15.156:0/135797884 shutdown_connections 2023-11-28T11:08:30.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.330+0000 7f42a0dd3700 1 -- 172.21.15.156:0/135797884 wait complete. 2023-11-28T11:08:30.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.332+0000 7fd7a659f700 1 -- 172.21.15.156:0/366813241 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd7a015c7b0 con 0x7fd7a006a270 2023-11-28T11:08:30.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.333+0000 7fd795ffb700 1 -- 172.21.15.156:0/366813241 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd79c057170 con 0x7fd7a006a270 2023-11-28T11:08:30.334 INFO:teuthology.orchestra.run.smithi156.stdout:55834574855 2023-11-28T11:08:30.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.335+0000 7fd7a659f700 1 -- 172.21.15.156:0/366813241 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7840682f0 msgr2=0x7fd78406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.335+0000 7fd7a659f700 1 --2- 172.21.15.156:0/366813241 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7840682f0 0x7fd78406a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0159dc0 tx=0x7fd790006c60 comp rx=0 tx=0).stop 2023-11-28T11:08:30.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.335+0000 7fd7a659f700 1 -- 172.21.15.156:0/366813241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a006a270 msgr2=0x7fd7a015d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.335+0000 7fd7a659f700 1 --2- 172.21.15.156:0/366813241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a006a270 0x7fd7a015d6a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009130 tx=0x7fd79c004750 comp rx=0 tx=0).stop 2023-11-28T11:08:30.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.335+0000 7fd7a659f700 1 -- 172.21.15.156:0/366813241 shutdown_connections 2023-11-28T11:08:30.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.335+0000 7fd7a659f700 1 --2- 172.21.15.156:0/366813241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a00b4b10 0x7fd7a015de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.335+0000 7fd7a659f700 1 --2- 172.21.15.156:0/366813241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a006a270 0x7fd7a015d6a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.335+0000 7fd7a659f700 1 --2- 172.21.15.156:0/366813241 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7840682f0 0x7fd78406a7a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.335+0000 7fd7a659f700 1 --2- 172.21.15.156:0/366813241 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a00ac8b0 0x7fd7a0152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.335+0000 7fd7a659f700 1 -- 172.21.15.156:0/366813241 >> 172.21.15.156:0/366813241 conn(0x7fd7a006b380 msgr2=0x7fd7a00af720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:30.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.335+0000 7fd7a659f700 1 -- 172.21.15.156:0/366813241 shutdown_connections 2023-11-28T11:08:30.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.335+0000 7fd7a659f700 1 -- 172.21.15.156:0/366813241 wait complete. 2023-11-28T11:08:30.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-11-28T11:08:30.345 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:30.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-11-28T11:08:30.349 DEBUG:teuthology.parallel:result is None 2023-11-28T11:08:30.349 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T11:08:30.349 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:08:30.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.509+0000 7ff09bc58700 1 Processor -- start 2023-11-28T11:08:30.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.509+0000 7ff09bc58700 1 -- start start 2023-11-28T11:08:30.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff09bc58700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff0940b20b0 0x7ff0940b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff09bc58700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0940b0230 con 0x7ff0940d1330 2023-11-28T11:08:30.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff09bc58700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0940b4570 con 0x7ff0940d01b0 2023-11-28T11:08:30.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff09bc58700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09415cec0 con 0x7ff0940b20b0 2023-11-28T11:08:30.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff0991f3700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff0940d1330 0x7ff0940b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47178/0 (socket says 172.21.15.156:47178) 2023-11-28T11:08:30.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff0991f3700 1 -- 172.21.15.156:0/1364008706 learned_addr learned my addr 172.21.15.156:0/1364008706 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:30.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff09a1f5700 1 --2- 172.21.15.156:0/1364008706 >> v2:172.21.15.186:3300/0 conn(0x7ff0940b20b0 0x7ff0940b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301062789 0 0) 0x7ff0940b0230 con 0x7ff0940d1330 2023-11-28T11:08:30.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0800035b0 con 0x7ff0940d1330 2023-11-28T11:08:30.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2253376300 0 0) 0x7ff0800035b0 con 0x7ff0940d1330 2023-11-28T11:08:30.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 >> v2:172.21.15.186:3300/0 conn(0x7ff0940b20b0 msgr2=0x7ff0940b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:30.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff0989f2700 1 --2- 172.21.15.156:0/1364008706 >> v2:172.21.15.186:3300/0 conn(0x7ff0940b20b0 0x7ff0940b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 >> v1:172.21.15.186:6789/0 conn(0x7ff0940d01b0 legacy=0x7ff0940d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0940b4850 con 0x7ff0940d1330 2023-11-28T11:08:30.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7ff084001670 con 0x7ff0940d1330 2023-11-28T11:08:30.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 >> v1:172.21.15.156:6789/0 conn(0x7ff0940d1330 legacy=0x7ff0940b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.510+0000 7ff0989f2700 1 --2- 172.21.15.156:0/1364008706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff080003a00 0x7ff080003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff0989f2700 1 --2- 172.21.15.156:0/1364008706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff080004850 0x7ff080004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff0989f2700 1 --2- 172.21.15.156:0/1364008706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff080005200 0x7ff08000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09415cec0 con 0x7ff080005200 2023-11-28T11:08:30.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08000c080 con 0x7ff080003a00 2023-11-28T11:08:30.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08000c240 con 0x7ff080004850 2023-11-28T11:08:30.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff09a1f5700 1 --2- 172.21.15.156:0/1364008706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff080005200 0x7ff08000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff0999f4700 1 --2- 172.21.15.156:0/1364008706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff080003a00 0x7ff080003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff0991f3700 1 --2- 172.21.15.156:0/1364008706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff080004850 0x7ff080004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff09a1f5700 1 -- 172.21.15.156:0/1364008706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff080004850 msgr2=0x7ff080004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:30.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff09a1f5700 1 --2- 172.21.15.156:0/1364008706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff080004850 0x7ff080004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff09a1f5700 1 -- 172.21.15.156:0/1364008706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff080003a00 msgr2=0x7ff080003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:30.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff09a1f5700 1 --2- 172.21.15.156:0/1364008706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff080003a00 0x7ff080003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff09a1f5700 1 -- 172.21.15.156:0/1364008706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0800035b0 con 0x7ff080005200 2023-11-28T11:08:30.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff09a1f5700 1 -- 172.21.15.156:0/1364008706 reap_dead start 2023-11-28T11:08:30.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff09a1f5700 1 --2- 172.21.15.156:0/1364008706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff080005200 0x7ff08000ba10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009f90 tx=0x7ff080005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4ad410e42869b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00ec30 con 0x7ff080005200 2023-11-28T11:08:30.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c00f040 con 0x7ff080005200 2023-11-28T11:08:30.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.511+0000 7ff0989f2700 1 -- 172.21.15.156:0/1364008706 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c014480 con 0x7ff080005200 2023-11-28T11:08:30.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.512+0000 7ff09bc58700 1 -- 172.21.15.156:0/1364008706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff080005200 msgr2=0x7ff08000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.512+0000 7ff09bc58700 1 --2- 172.21.15.156:0/1364008706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff080005200 0x7ff08000ba10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009f90 tx=0x7ff080005880 comp rx=0 tx=0).stop 2023-11-28T11:08:30.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.512+0000 7ff09bc58700 1 -- 172.21.15.156:0/1364008706 shutdown_connections 2023-11-28T11:08:30.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.512+0000 7ff09bc58700 1 --2- 172.21.15.156:0/1364008706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff080005200 0x7ff08000ba10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.512+0000 7ff09bc58700 1 -- 172.21.15.156:0/1364008706 >> 172.21.15.156:0/1364008706 conn(0x7ff09409c3f0 msgr2=0x7ff0940d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:30.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.512+0000 7ff09bc58700 1 -- 172.21.15.156:0/1364008706 shutdown_connections 2023-11-28T11:08:30.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.512+0000 7ff09bc58700 1 -- 172.21.15.156:0/1364008706 wait complete. 2023-11-28T11:08:30.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.513+0000 7ff09bc58700 1 Processor -- start 2023-11-28T11:08:30.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.513+0000 7ff09bc58700 1 -- start start 2023-11-28T11:08:30.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.514+0000 7ff09bc58700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff080005200 0x7ff09415fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.514+0000 7ff09bc58700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09409cd60 0x7ff0940c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.514+0000 7ff09bc58700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0940c8000 0x7ff0940c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.514+0000 7ff09bc58700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff094151080 con 0x7ff09409cd60 2023-11-28T11:08:30.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.514+0000 7ff09bc58700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09415d3f0 con 0x7ff080005200 2023-11-28T11:08:30.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.514+0000 7ff09bc58700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09415df70 con 0x7ff0940c8000 2023-11-28T11:08:30.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.514+0000 7ff0991f3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09409cd60 0x7ff0940c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.514+0000 7ff0991f3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09409cd60 0x7ff0940c7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60544/0 (socket says 172.21.15.156:60544) 2023-11-28T11:08:30.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.514+0000 7ff0991f3700 1 -- 172.21.15.156:0/944636228 learned_addr learned my addr 172.21.15.156:0/944636228 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:30.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff09a1f5700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0940c8000 0x7ff0940c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff0999f4700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff080005200 0x7ff09415fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff0991f3700 1 -- 172.21.15.156:0/944636228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0940c8000 msgr2=0x7ff0940c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff0991f3700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0940c8000 0x7ff0940c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff0991f3700 1 -- 172.21.15.156:0/944636228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff080005200 msgr2=0x7ff09415fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff0991f3700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff080005200 0x7ff09415fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff0991f3700 1 -- 172.21.15.156:0/944636228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0940afec0 con 0x7ff09409cd60 2023-11-28T11:08:30.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff0991f3700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09409cd60 0x7ff0940c7ac0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff0840072c0 tx=0x7ff0840075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff08a7fc700 1 -- 172.21.15.156:0/944636228 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08400f6c0 con 0x7ff09409cd60 2023-11-28T11:08:30.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff08a7fc700 1 -- 172.21.15.156:0/944636228 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08400f820 con 0x7ff09409cd60 2023-11-28T11:08:30.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff09bc58700 1 -- 172.21.15.156:0/944636228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0940afbe0 con 0x7ff09409cd60 2023-11-28T11:08:30.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff08a7fc700 1 -- 172.21.15.156:0/944636228 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08400fbd0 con 0x7ff09409cd60 2023-11-28T11:08:30.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.515+0000 7ff09bc58700 1 -- 172.21.15.156:0/944636228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0940b2600 con 0x7ff09409cd60 2023-11-28T11:08:30.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.516+0000 7ff09bc58700 1 -- 172.21.15.156:0/944636228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff094068690 con 0x7ff09409cd60 2023-11-28T11:08:30.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.516+0000 7ff08a7fc700 1 -- 172.21.15.156:0/944636228 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff08400fd30 con 0x7ff09409cd60 2023-11-28T11:08:30.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.517+0000 7ff08a7fc700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0780682f0 0x7ff07806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.517+0000 7ff0999f4700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0780682f0 0x7ff07806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.517+0000 7ff08a7fc700 1 -- 172.21.15.156:0/944636228 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff084085d70 con 0x7ff09409cd60 2023-11-28T11:08:30.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.517+0000 7ff0999f4700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0780682f0 0x7ff07806a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff090000c00 tx=0x7ff090009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.521+0000 7ff08a7fc700 1 -- 172.21.15.156:0/944636228 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff0840525d0 con 0x7ff09409cd60 2023-11-28T11:08:30.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.704+0000 7ff09bc58700 1 -- 172.21.15.156:0/944636228 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff09415e0b0 con 0x7ff0780682f0 2023-11-28T11:08:30.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.712+0000 7ff08a7fc700 1 -- 172.21.15.156:0/944636228 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286621 (secure 0 0 0) 0x7ff09415e0b0 con 0x7ff0780682f0 2023-11-28T11:08:30.716 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:30.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.715+0000 7ff09bc58700 1 -- 172.21.15.156:0/944636228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0780682f0 msgr2=0x7ff07806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.715+0000 7ff09bc58700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0780682f0 0x7ff07806a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff090000c00 tx=0x7ff090009040 comp rx=0 tx=0).stop 2023-11-28T11:08:30.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.715+0000 7ff09bc58700 1 -- 172.21.15.156:0/944636228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09409cd60 msgr2=0x7ff0940c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.715+0000 7ff09bc58700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09409cd60 0x7ff0940c7ac0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff0840072c0 tx=0x7ff0840075d0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.715+0000 7ff09bc58700 1 -- 172.21.15.156:0/944636228 shutdown_connections 2023-11-28T11:08:30.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.715+0000 7ff09bc58700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0780682f0 0x7ff07806a7a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.715+0000 7ff09bc58700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0940c8000 0x7ff0940c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.715+0000 7ff09bc58700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff080005200 0x7ff09415fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.715+0000 7ff09bc58700 1 --2- 172.21.15.156:0/944636228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09409cd60 0x7ff0940c7ac0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.715+0000 7ff09bc58700 1 -- 172.21.15.156:0/944636228 >> 172.21.15.156:0/944636228 conn(0x7ff09409c3f0 msgr2=0x7ff09406c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:30.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.715+0000 7ff09bc58700 1 -- 172.21.15.156:0/944636228 shutdown_connections 2023-11-28T11:08:30.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.715+0000 7ff09bc58700 1 -- 172.21.15.156:0/944636228 wait complete. 2023-11-28T11:08:30.720 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:08:30.728 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-11-28T11:08:30.528645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66216,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":754908504,"statfs":{"total":773094113280,"available":773026308096,"internally_reserved":0,"allocated":3186688,"data_stored":1902188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003562"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072799+0000","last_change":"2023-11-28T11:08:17.367751+0000","last_active":"2023-11-28T11:08:19.072799+0000","last_peered":"2023-11-28T11:08:19.072799+0000","last_clean":"2023-11-28T11:08:19.072799+0000","last_became_active":"2023-11-28T11:08:17.367481+0000","last_became_peered":"2023-11-28T11:08:17.367481+0000","last_unstale":"2023-11-28T11:08:19.072799+0000","last_undegraded":"2023-11-28T11:08:19.072799+0000","last_fullsized":"2023-11-28T11:08:19.072799+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:09.760690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073854+0000","last_change":"2023-11-28T11:08:17.369749+0000","last_active":"2023-11-28T11:08:19.073854+0000","last_peered":"2023-11-28T11:08:19.073854+0000","last_clean":"2023-11-28T11:08:19.073854+0000","last_became_active":"2023-11-28T11:08:17.369479+0000","last_became_peered":"2023-11-28T11:08:17.369479+0000","last_unstale":"2023-11-28T11:08:19.073854+0000","last_undegraded":"2023-11-28T11:08:19.073854+0000","last_fullsized":"2023-11-28T11:08:19.073854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369159+0000","last_change":"2023-11-28T11:08:17.370726+0000","last_active":"2023-11-28T11:08:18.369159+0000","last_peered":"2023-11-28T11:08:18.369159+0000","last_clean":"2023-11-28T11:08:18.369159+0000","last_became_active":"2023-11-28T11:08:17.367062+0000","last_became_peered":"2023-11-28T11:08:17.367062+0000","last_unstale":"2023-11-28T11:08:18.369159+0000","last_undegraded":"2023-11-28T11:08:18.369159+0000","last_fullsized":"2023-11-28T11:08:18.369159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:27.219209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073872+0000","last_change":"2023-11-28T11:08:17.383989+0000","last_active":"2023-11-28T11:08:19.073872+0000","last_peered":"2023-11-28T11:08:19.073872+0000","last_clean":"2023-11-28T11:08:19.073872+0000","last_became_active":"2023-11-28T11:08:17.381618+0000","last_became_peered":"2023-11-28T11:08:17.381618+0000","last_unstale":"2023-11-28T11:08:19.073872+0000","last_undegraded":"2023-11-28T11:08:19.073872+0000","last_fullsized":"2023-11-28T11:08:19.073872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:09:00.918897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673454+0000","last_change":"2023-11-28T11:08:17.377110+0000","last_active":"2023-11-28T11:08:18.673454+0000","last_peered":"2023-11-28T11:08:18.673454+0000","last_clean":"2023-11-28T11:08:18.673454+0000","last_became_active":"2023-11-28T11:08:17.373583+0000","last_became_peered":"2023-11-28T11:08:17.373583+0000","last_unstale":"2023-11-28T11:08:18.673454+0000","last_undegraded":"2023-11-28T11:08:18.673454+0000","last_fullsized":"2023-11-28T11:08:18.673454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:24:59.277184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368780+0000","last_change":"2023-11-28T11:08:17.370700+0000","last_active":"2023-11-28T11:08:18.368780+0000","last_peered":"2023-11-28T11:08:18.368780+0000","last_clean":"2023-11-28T11:08:18.368780+0000","last_became_active":"2023-11-28T11:08:17.367222+0000","last_became_peered":"2023-11-28T11:08:17.367222+0000","last_unstale":"2023-11-28T11:08:18.368780+0000","last_undegraded":"2023-11-28T11:08:18.368780+0000","last_fullsized":"2023-11-28T11:08:18.368780+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:22.140101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073497+0000","last_change":"2023-11-28T11:08:17.374848+0000","last_active":"2023-11-28T11:08:19.073497+0000","last_peered":"2023-11-28T11:08:19.073497+0000","last_clean":"2023-11-28T11:08:19.073497+0000","last_became_active":"2023-11-28T11:08:17.374536+0000","last_became_peered":"2023-11-28T11:08:17.374536+0000","last_unstale":"2023-11-28T11:08:19.073497+0000","last_undegraded":"2023-11-28T11:08:19.073497+0000","last_fullsized":"2023-11-28T11:08:19.073497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074252+0000","last_change":"2023-11-28T11:08:17.366207+0000","last_active":"2023-11-28T11:08:19.074252+0000","last_peered":"2023-11-28T11:08:19.074252+0000","last_clean":"2023-11-28T11:08:19.074252+0000","last_became_active":"2023-11-28T11:08:17.365918+0000","last_became_peered":"2023-11-28T11:08:17.365918+0000","last_unstale":"2023-11-28T11:08:19.074252+0000","last_undegraded":"2023-11-28T11:08:19.074252+0000","last_fullsized":"2023-11-28T11:08:19.074252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673623+0000","last_change":"2023-11-28T11:08:17.376435+0000","last_active":"2023-11-28T11:08:18.673623+0000","last_peered":"2023-11-28T11:08:18.673623+0000","last_clean":"2023-11-28T11:08:18.673623+0000","last_became_active":"2023-11-28T11:08:17.373543+0000","last_became_peered":"2023-11-28T11:08:17.373543+0000","last_unstale":"2023-11-28T11:08:18.673623+0000","last_undegraded":"2023-11-28T11:08:18.673623+0000","last_fullsized":"2023-11-28T11:08:18.673623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074030+0000","last_change":"2023-11-28T11:08:17.367284+0000","last_active":"2023-11-28T11:08:19.074030+0000","last_peered":"2023-11-28T11:08:19.074030+0000","last_clean":"2023-11-28T11:08:19.074030+0000","last_became_active":"2023-11-28T11:08:17.366992+0000","last_became_peered":"2023-11-28T11:08:17.366992+0000","last_unstale":"2023-11-28T11:08:19.074030+0000","last_undegraded":"2023-11-28T11:08:19.074030+0000","last_fullsized":"2023-11-28T11:08:19.074030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369258+0000","last_change":"2023-11-28T11:08:17.376061+0000","last_active":"2023-11-28T11:08:18.369258+0000","last_peered":"2023-11-28T11:08:18.369258+0000","last_clean":"2023-11-28T11:08:18.369258+0000","last_became_active":"2023-11-28T11:08:17.375789+0000","last_became_peered":"2023-11-28T11:08:17.375789+0000","last_unstale":"2023-11-28T11:08:18.369258+0000","last_undegraded":"2023-11-28T11:08:18.369258+0000","last_fullsized":"2023-11-28T11:08:18.369258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:10.040305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072686+0000","last_change":"2023-11-28T11:08:17.372872+0000","last_active":"2023-11-28T11:08:19.072686+0000","last_peered":"2023-11-28T11:08:19.072686+0000","last_clean":"2023-11-28T11:08:19.072686+0000","last_became_active":"2023-11-28T11:08:17.372624+0000","last_became_peered":"2023-11-28T11:08:17.372624+0000","last_unstale":"2023-11-28T11:08:19.072686+0000","last_undegraded":"2023-11-28T11:08:19.072686+0000","last_fullsized":"2023-11-28T11:08:19.072686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:34.711368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072664+0000","last_change":"2023-11-28T11:08:17.374949+0000","last_active":"2023-11-28T11:08:19.072664+0000","last_peered":"2023-11-28T11:08:19.072664+0000","last_clean":"2023-11-28T11:08:19.072664+0000","last_became_active":"2023-11-28T11:08:17.374690+0000","last_became_peered":"2023-11-28T11:08:17.374690+0000","last_unstale":"2023-11-28T11:08:19.072664+0000","last_undegraded":"2023-11-28T11:08:19.072664+0000","last_fullsized":"2023-11-28T11:08:19.072664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:52.301973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073186+0000","last_change":"2023-11-28T11:08:17.361341+0000","last_active":"2023-11-28T11:08:19.073186+0000","last_peered":"2023-11-28T11:08:19.073186+0000","last_clean":"2023-11-28T11:08:19.073186+0000","last_became_active":"2023-11-28T11:08:17.361053+0000","last_became_peered":"2023-11-28T11:08:17.361053+0000","last_unstale":"2023-11-28T11:08:19.073186+0000","last_undegraded":"2023-11-28T11:08:19.073186+0000","last_fullsized":"2023-11-28T11:08:19.073186+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073689+0000","last_change":"2023-11-28T11:08:17.377341+0000","last_active":"2023-11-28T11:08:19.073689+0000","last_peered":"2023-11-28T11:08:19.073689+0000","last_clean":"2023-11-28T11:08:19.073689+0000","last_became_active":"2023-11-28T11:08:17.376840+0000","last_became_peered":"2023-11-28T11:08:17.376840+0000","last_unstale":"2023-11-28T11:08:19.073689+0000","last_undegraded":"2023-11-28T11:08:19.073689+0000","last_fullsized":"2023-11-28T11:08:19.073689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368722+0000","last_change":"2023-11-28T11:08:17.370526+0000","last_active":"2023-11-28T11:08:18.368722+0000","last_peered":"2023-11-28T11:08:18.368722+0000","last_clean":"2023-11-28T11:08:18.368722+0000","last_became_active":"2023-11-28T11:08:17.367599+0000","last_became_peered":"2023-11-28T11:08:17.367599+0000","last_unstale":"2023-11-28T11:08:18.368722+0000","last_undegraded":"2023-11-28T11:08:18.368722+0000","last_fullsized":"2023-11-28T11:08:18.368722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:21.226857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673664+0000","last_change":"2023-11-28T11:08:17.377283+0000","last_active":"2023-11-28T11:08:18.673664+0000","last_peered":"2023-11-28T11:08:18.673664+0000","last_clean":"2023-11-28T11:08:18.673664+0000","last_became_active":"2023-11-28T11:08:17.373226+0000","last_became_peered":"2023-11-28T11:08:17.373226+0000","last_unstale":"2023-11-28T11:08:18.673664+0000","last_undegraded":"2023-11-28T11:08:18.673664+0000","last_fullsized":"2023-11-28T11:08:18.673664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369217+0000","last_change":"2023-11-28T11:08:17.373771+0000","last_active":"2023-11-28T11:08:18.369217+0000","last_peered":"2023-11-28T11:08:18.369217+0000","last_clean":"2023-11-28T11:08:18.369217+0000","last_became_active":"2023-11-28T11:08:17.373510+0000","last_became_peered":"2023-11-28T11:08:17.373510+0000","last_unstale":"2023-11-28T11:08:18.369217+0000","last_undegraded":"2023-11-28T11:08:18.369217+0000","last_fullsized":"2023-11-28T11:08:18.369217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:14.696335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072793+0000","last_change":"2023-11-28T11:08:17.381345+0000","last_active":"2023-11-28T11:08:19.072793+0000","last_peered":"2023-11-28T11:08:19.072793+0000","last_clean":"2023-11-28T11:08:19.072793+0000","last_became_active":"2023-11-28T11:08:17.381044+0000","last_became_peered":"2023-11-28T11:08:17.381044+0000","last_unstale":"2023-11-28T11:08:19.072793+0000","last_undegraded":"2023-11-28T11:08:19.072793+0000","last_fullsized":"2023-11-28T11:08:19.072793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:09.877040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369092+0000","last_change":"2023-11-28T11:08:17.387391+0000","last_active":"2023-11-28T11:08:18.369092+0000","last_peered":"2023-11-28T11:08:18.369092+0000","last_clean":"2023-11-28T11:08:18.369092+0000","last_became_active":"2023-11-28T11:08:17.387127+0000","last_became_peered":"2023-11-28T11:08:17.387127+0000","last_unstale":"2023-11-28T11:08:18.369092+0000","last_undegraded":"2023-11-28T11:08:18.369092+0000","last_fullsized":"2023-11-28T11:08:18.369092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074429+0000","last_change":"2023-11-28T11:08:17.380537+0000","last_active":"2023-11-28T11:08:19.074429+0000","last_peered":"2023-11-28T11:08:19.074429+0000","last_clean":"2023-11-28T11:08:19.074429+0000","last_became_active":"2023-11-28T11:08:17.380267+0000","last_became_peered":"2023-11-28T11:08:17.380267+0000","last_unstale":"2023-11-28T11:08:19.074429+0000","last_undegraded":"2023-11-28T11:08:19.074429+0000","last_fullsized":"2023-11-28T11:08:19.074429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673493+0000","last_change":"2023-11-28T11:08:17.371837+0000","last_active":"2023-11-28T11:08:18.673493+0000","last_peered":"2023-11-28T11:08:18.673493+0000","last_clean":"2023-11-28T11:08:18.673493+0000","last_became_active":"2023-11-28T11:08:17.371543+0000","last_became_peered":"2023-11-28T11:08:17.371543+0000","last_unstale":"2023-11-28T11:08:18.673493+0000","last_undegraded":"2023-11-28T11:08:18.673493+0000","last_fullsized":"2023-11-28T11:08:18.673493+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365642+0000","last_change":"2023-11-28T11:08:17.370684+0000","last_active":"2023-11-28T11:08:18.365642+0000","last_peered":"2023-11-28T11:08:18.365642+0000","last_clean":"2023-11-28T11:08:18.365642+0000","last_became_active":"2023-11-28T11:08:17.367787+0000","last_became_peered":"2023-11-28T11:08:17.367787+0000","last_unstale":"2023-11-28T11:08:18.365642+0000","last_undegraded":"2023-11-28T11:08:18.365642+0000","last_fullsized":"2023-11-28T11:08:18.365642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:59.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073418+0000","last_change":"2023-11-28T11:08:17.373809+0000","last_active":"2023-11-28T11:08:19.073418+0000","last_peered":"2023-11-28T11:08:19.073418+0000","last_clean":"2023-11-28T11:08:19.073418+0000","last_became_active":"2023-11-28T11:08:17.373529+0000","last_became_peered":"2023-11-28T11:08:17.373529+0000","last_unstale":"2023-11-28T11:08:19.073418+0000","last_undegraded":"2023-11-28T11:08:19.073418+0000","last_fullsized":"2023-11-28T11:08:19.073418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368409+0000","last_change":"2023-11-28T11:08:17.382413+0000","last_active":"2023-11-28T11:08:18.368409+0000","last_peered":"2023-11-28T11:08:18.368409+0000","last_clean":"2023-11-28T11:08:18.368409+0000","last_became_active":"2023-11-28T11:08:17.382132+0000","last_became_peered":"2023-11-28T11:08:17.382132+0000","last_unstale":"2023-11-28T11:08:18.368409+0000","last_undegraded":"2023-11-28T11:08:18.368409+0000","last_fullsized":"2023-11-28T11:08:18.368409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368518+0000","last_change":"2023-11-28T11:08:17.368106+0000","last_active":"2023-11-28T11:08:18.368518+0000","last_peered":"2023-11-28T11:08:18.368518+0000","last_clean":"2023-11-28T11:08:18.368518+0000","last_became_active":"2023-11-28T11:08:17.367562+0000","last_became_peered":"2023-11-28T11:08:17.367562+0000","last_unstale":"2023-11-28T11:08:18.368518+0000","last_undegraded":"2023-11-28T11:08:18.368518+0000","last_fullsized":"2023-11-28T11:08:18.368518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073955+0000","last_change":"2023-11-28T11:08:17.375521+0000","last_active":"2023-11-28T11:08:19.073955+0000","last_peered":"2023-11-28T11:08:19.073955+0000","last_clean":"2023-11-28T11:08:19.073955+0000","last_became_active":"2023-11-28T11:08:17.375256+0000","last_became_peered":"2023-11-28T11:08:17.375256+0000","last_unstale":"2023-11-28T11:08:19.073955+0000","last_undegraded":"2023-11-28T11:08:19.073955+0000","last_fullsized":"2023-11-28T11:08:19.073955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.382174+0000","last_change":"2023-11-28T11:08:16.975491+0000","last_active":"2023-11-28T11:08:17.382174+0000","last_peered":"2023-11-28T11:08:17.382174+0000","last_clean":"2023-11-28T11:08:17.382174+0000","last_became_active":"2023-11-28T11:08:16.975076+0000","last_became_peered":"2023-11-28T11:08:16.975076+0000","last_unstale":"2023-11-28T11:08:17.382174+0000","last_undegraded":"2023-11-28T11:08:17.382174+0000","last_fullsized":"2023-11-28T11:08:17.382174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365845+0000","last_change":"2023-11-28T11:08:17.371715+0000","last_active":"2023-11-28T11:08:18.365845+0000","last_peered":"2023-11-28T11:08:18.365845+0000","last_clean":"2023-11-28T11:08:18.365845+0000","last_became_active":"2023-11-28T11:08:17.368939+0000","last_became_peered":"2023-11-28T11:08:17.368939+0000","last_unstale":"2023-11-28T11:08:18.365845+0000","last_undegraded":"2023-11-28T11:08:18.365845+0000","last_fullsized":"2023-11-28T11:08:18.365845+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362665+0000","last_change":"2023-11-28T11:08:16.975573+0000","last_active":"2023-11-28T11:08:17.362665+0000","last_peered":"2023-11-28T11:08:17.362665+0000","last_clean":"2023-11-28T11:08:17.362665+0000","last_became_active":"2023-11-28T11:08:16.975216+0000","last_became_peered":"2023-11-28T11:08:16.975216+0000","last_unstale":"2023-11-28T11:08:17.362665+0000","last_undegraded":"2023-11-28T11:08:17.362665+0000","last_fullsized":"2023-11-28T11:08:17.362665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673721+0000","last_change":"2023-11-28T11:08:17.378243+0000","last_active":"2023-11-28T11:08:18.673721+0000","last_peered":"2023-11-28T11:08:18.673721+0000","last_clean":"2023-11-28T11:08:18.673721+0000","last_became_active":"2023-11-28T11:08:17.377866+0000","last_became_peered":"2023-11-28T11:08:17.377866+0000","last_unstale":"2023-11-28T11:08:18.673721+0000","last_undegraded":"2023-11-28T11:08:18.673721+0000","last_fullsized":"2023-11-28T11:08:18.673721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.356027+0000","last_change":"2023-11-28T11:08:16.972779+0000","last_active":"2023-11-28T11:08:17.356027+0000","last_peered":"2023-11-28T11:08:17.356027+0000","last_clean":"2023-11-28T11:08:17.356027+0000","last_became_active":"2023-11-28T11:08:16.972510+0000","last_became_peered":"2023-11-28T11:08:16.972510+0000","last_unstale":"2023-11-28T11:08:17.356027+0000","last_undegraded":"2023-11-28T11:08:17.356027+0000","last_fullsized":"2023-11-28T11:08:17.356027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074026+0000","last_change":"2023-11-28T11:08:17.380638+0000","last_active":"2023-11-28T11:08:19.074026+0000","last_peered":"2023-11-28T11:08:19.074026+0000","last_clean":"2023-11-28T11:08:19.074026+0000","last_became_active":"2023-11-28T11:08:17.380333+0000","last_became_peered":"2023-11-28T11:08:17.380333+0000","last_unstale":"2023-11-28T11:08:19.074026+0000","last_undegraded":"2023-11-28T11:08:19.074026+0000","last_fullsized":"2023-11-28T11:08:19.074026+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:34.711368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.361717+0000","last_change":"2023-11-28T11:08:16.976366+0000","last_active":"2023-11-28T11:08:17.361717+0000","last_peered":"2023-11-28T11:08:17.361717+0000","last_clean":"2023-11-28T11:08:17.361717+0000","last_became_active":"2023-11-28T11:08:16.976072+0000","last_became_peered":"2023-11-28T11:08:16.976072+0000","last_unstale":"2023-11-28T11:08:17.361717+0000","last_undegraded":"2023-11-28T11:08:17.361717+0000","last_fullsized":"2023-11-28T11:08:17.361717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365642+0000","last_change":"2023-11-28T11:08:17.364878+0000","last_active":"2023-11-28T11:08:18.365642+0000","last_peered":"2023-11-28T11:08:18.365642+0000","last_clean":"2023-11-28T11:08:18.365642+0000","last_became_active":"2023-11-28T11:08:17.364628+0000","last_became_peered":"2023-11-28T11:08:17.364628+0000","last_unstale":"2023-11-28T11:08:18.365642+0000","last_undegraded":"2023-11-28T11:08:18.365642+0000","last_fullsized":"2023-11-28T11:08:18.365642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:12.679494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.358873+0000","last_change":"2023-11-28T11:08:16.975388+0000","last_active":"2023-11-28T11:08:17.358873+0000","last_peered":"2023-11-28T11:08:17.358873+0000","last_clean":"2023-11-28T11:08:17.358873+0000","last_became_active":"2023-11-28T11:08:16.974977+0000","last_became_peered":"2023-11-28T11:08:16.974977+0000","last_unstale":"2023-11-28T11:08:17.358873+0000","last_undegraded":"2023-11-28T11:08:17.358873+0000","last_fullsized":"2023-11-28T11:08:17.358873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365734+0000","last_change":"2023-11-28T11:08:17.376435+0000","last_active":"2023-11-28T11:08:18.365734+0000","last_peered":"2023-11-28T11:08:18.365734+0000","last_clean":"2023-11-28T11:08:18.365734+0000","last_became_active":"2023-11-28T11:08:17.376170+0000","last_became_peered":"2023-11-28T11:08:17.376170+0000","last_unstale":"2023-11-28T11:08:18.365734+0000","last_undegraded":"2023-11-28T11:08:18.365734+0000","last_fullsized":"2023-11-28T11:08:18.365734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:24:59.277184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.381284+0000","last_change":"2023-11-28T11:08:16.975413+0000","last_active":"2023-11-28T11:08:17.381284+0000","last_peered":"2023-11-28T11:08:17.381284+0000","last_clean":"2023-11-28T11:08:17.381284+0000","last_became_active":"2023-11-28T11:08:16.975058+0000","last_became_peered":"2023-11-28T11:08:16.975058+0000","last_unstale":"2023-11-28T11:08:17.381284+0000","last_undegraded":"2023-11-28T11:08:17.381284+0000","last_fullsized":"2023-11-28T11:08:17.381284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368111+0000","last_change":"2023-11-28T11:08:17.381210+0000","last_active":"2023-11-28T11:08:18.368111+0000","last_peered":"2023-11-28T11:08:18.368111+0000","last_clean":"2023-11-28T11:08:18.368111+0000","last_became_active":"2023-11-28T11:08:17.380941+0000","last_became_peered":"2023-11-28T11:08:17.380941+0000","last_unstale":"2023-11-28T11:08:18.368111+0000","last_undegraded":"2023-11-28T11:08:18.368111+0000","last_fullsized":"2023-11-28T11:08:18.368111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:35.096193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.387746+0000","last_change":"2023-11-28T11:08:16.973937+0000","last_active":"2023-11-28T11:08:17.387746+0000","last_peered":"2023-11-28T11:08:17.387746+0000","last_clean":"2023-11-28T11:08:17.387746+0000","last_became_active":"2023-11-28T11:08:16.973594+0000","last_became_peered":"2023-11-28T11:08:16.973594+0000","last_unstale":"2023-11-28T11:08:17.387746+0000","last_undegraded":"2023-11-28T11:08:17.387746+0000","last_fullsized":"2023-11-28T11:08:17.387746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:13.313856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369025+0000","last_change":"2023-11-28T11:08:17.380894+0000","last_active":"2023-11-28T11:08:18.369025+0000","last_peered":"2023-11-28T11:08:18.369025+0000","last_clean":"2023-11-28T11:08:18.369025+0000","last_became_active":"2023-11-28T11:08:17.380600+0000","last_became_peered":"2023-11-28T11:08:17.380600+0000","last_unstale":"2023-11-28T11:08:18.369025+0000","last_undegraded":"2023-11-28T11:08:18.369025+0000","last_fullsized":"2023-11-28T11:08:18.369025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377908+0000","last_change":"2023-11-28T11:08:16.977393+0000","last_active":"2023-11-28T11:08:17.377908+0000","last_peered":"2023-11-28T11:08:17.377908+0000","last_clean":"2023-11-28T11:08:17.377908+0000","last_became_active":"2023-11-28T11:08:16.976585+0000","last_became_peered":"2023-11-28T11:08:16.976585+0000","last_unstale":"2023-11-28T11:08:17.377908+0000","last_undegraded":"2023-11-28T11:08:17.377908+0000","last_fullsized":"2023-11-28T11:08:17.377908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:36.981686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365857+0000","last_change":"2023-11-28T11:08:17.376235+0000","last_active":"2023-11-28T11:08:18.365857+0000","last_peered":"2023-11-28T11:08:18.365857+0000","last_clean":"2023-11-28T11:08:18.365857+0000","last_became_active":"2023-11-28T11:08:17.375675+0000","last_became_peered":"2023-11-28T11:08:17.375675+0000","last_unstale":"2023-11-28T11:08:18.365857+0000","last_undegraded":"2023-11-28T11:08:18.365857+0000","last_fullsized":"2023-11-28T11:08:18.365857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.359416+0000","last_change":"2023-11-28T11:08:16.974456+0000","last_active":"2023-11-28T11:08:17.359416+0000","last_peered":"2023-11-28T11:08:17.359416+0000","last_clean":"2023-11-28T11:08:17.359416+0000","last_became_active":"2023-11-28T11:08:16.974158+0000","last_became_peered":"2023-11-28T11:08:16.974158+0000","last_unstale":"2023-11-28T11:08:17.359416+0000","last_undegraded":"2023-11-28T11:08:17.359416+0000","last_fullsized":"2023-11-28T11:08:17.359416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074033+0000","last_change":"2023-11-28T11:08:17.375410+0000","last_active":"2023-11-28T11:08:19.074033+0000","last_peered":"2023-11-28T11:08:19.074033+0000","last_clean":"2023-11-28T11:08:19.074033+0000","last_became_active":"2023-11-28T11:08:17.374828+0000","last_became_peered":"2023-11-28T11:08:17.374828+0000","last_unstale":"2023-11-28T11:08:19.074033+0000","last_undegraded":"2023-11-28T11:08:19.074033+0000","last_fullsized":"2023-11-28T11:08:19.074033+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365862+0000","last_change":"2023-11-28T11:08:16.978277+0000","last_active":"2023-11-28T11:08:17.365862+0000","last_peered":"2023-11-28T11:08:17.365862+0000","last_clean":"2023-11-28T11:08:17.365862+0000","last_became_active":"2023-11-28T11:08:16.977994+0000","last_became_peered":"2023-11-28T11:08:16.977994+0000","last_unstale":"2023-11-28T11:08:17.365862+0000","last_undegraded":"2023-11-28T11:08:17.365862+0000","last_fullsized":"2023-11-28T11:08:17.365862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:55.975785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673702+0000","last_change":"2023-11-28T11:08:17.369372+0000","last_active":"2023-11-28T11:08:18.673702+0000","last_peered":"2023-11-28T11:08:18.673702+0000","last_clean":"2023-11-28T11:08:18.673702+0000","last_became_active":"2023-11-28T11:08:17.369115+0000","last_became_peered":"2023-11-28T11:08:17.369115+0000","last_unstale":"2023-11-28T11:08:18.673702+0000","last_undegraded":"2023-11-28T11:08:18.673702+0000","last_fullsized":"2023-11-28T11:08:18.673702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.386102+0000","last_change":"2023-11-28T11:08:16.974290+0000","last_active":"2023-11-28T11:08:17.386102+0000","last_peered":"2023-11-28T11:08:17.386102+0000","last_clean":"2023-11-28T11:08:17.386102+0000","last_became_active":"2023-11-28T11:08:16.973952+0000","last_became_peered":"2023-11-28T11:08:16.973952+0000","last_unstale":"2023-11-28T11:08:17.386102+0000","last_undegraded":"2023-11-28T11:08:17.386102+0000","last_fullsized":"2023-11-28T11:08:17.386102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365978+0000","last_change":"2023-11-28T11:08:17.376526+0000","last_active":"2023-11-28T11:08:18.365978+0000","last_peered":"2023-11-28T11:08:18.365978+0000","last_clean":"2023-11-28T11:08:18.365978+0000","last_became_active":"2023-11-28T11:08:17.376285+0000","last_became_peered":"2023-11-28T11:08:17.376285+0000","last_unstale":"2023-11-28T11:08:18.365978+0000","last_undegraded":"2023-11-28T11:08:18.365978+0000","last_fullsized":"2023-11-28T11:08:18.365978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377922+0000","last_change":"2023-11-28T11:08:16.976879+0000","last_active":"2023-11-28T11:08:17.377922+0000","last_peered":"2023-11-28T11:08:17.377922+0000","last_clean":"2023-11-28T11:08:17.377922+0000","last_became_active":"2023-11-28T11:08:16.976530+0000","last_became_peered":"2023-11-28T11:08:16.976530+0000","last_unstale":"2023-11-28T11:08:17.377922+0000","last_undegraded":"2023-11-28T11:08:17.377922+0000","last_fullsized":"2023-11-28T11:08:17.377922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:59.917012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073041+0000","last_change":"2023-11-28T11:08:17.375192+0000","last_active":"2023-11-28T11:08:19.073041+0000","last_peered":"2023-11-28T11:08:19.073041+0000","last_clean":"2023-11-28T11:08:19.073041+0000","last_became_active":"2023-11-28T11:08:17.374903+0000","last_became_peered":"2023-11-28T11:08:17.374903+0000","last_unstale":"2023-11-28T11:08:19.073041+0000","last_undegraded":"2023-11-28T11:08:19.073041+0000","last_fullsized":"2023-11-28T11:08:19.073041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377956+0000","last_change":"2023-11-28T11:08:16.980934+0000","last_active":"2023-11-28T11:08:17.377956+0000","last_peered":"2023-11-28T11:08:17.377956+0000","last_clean":"2023-11-28T11:08:17.377956+0000","last_became_active":"2023-11-28T11:08:16.980585+0000","last_became_peered":"2023-11-28T11:08:16.980585+0000","last_unstale":"2023-11-28T11:08:17.377956+0000","last_undegraded":"2023-11-28T11:08:17.377956+0000","last_fullsized":"2023-11-28T11:08:17.377956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:33.709483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368728+0000","last_change":"2023-11-28T11:08:17.374922+0000","last_active":"2023-11-28T11:08:18.368728+0000","last_peered":"2023-11-28T11:08:18.368728+0000","last_clean":"2023-11-28T11:08:18.368728+0000","last_became_active":"2023-11-28T11:08:17.374631+0000","last_became_peered":"2023-11-28T11:08:17.374631+0000","last_unstale":"2023-11-28T11:08:18.368728+0000","last_undegraded":"2023-11-28T11:08:18.368728+0000","last_fullsized":"2023-11-28T11:08:18.368728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:28:26.671538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.375561+0000","last_change":"2023-11-28T11:08:16.978233+0000","last_active":"2023-11-28T11:08:17.375561+0000","last_peered":"2023-11-28T11:08:17.375561+0000","last_clean":"2023-11-28T11:08:17.375561+0000","last_became_active":"2023-11-28T11:08:16.977953+0000","last_became_peered":"2023-11-28T11:08:16.977953+0000","last_unstale":"2023-11-28T11:08:17.375561+0000","last_undegraded":"2023-11-28T11:08:17.375561+0000","last_fullsized":"2023-11-28T11:08:17.375561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073837+0000","last_change":"2023-11-28T11:08:17.381282+0000","last_active":"2023-11-28T11:08:19.073837+0000","last_peered":"2023-11-28T11:08:19.073837+0000","last_clean":"2023-11-28T11:08:19.073837+0000","last_became_active":"2023-11-28T11:08:17.381013+0000","last_became_peered":"2023-11-28T11:08:17.381013+0000","last_unstale":"2023-11-28T11:08:19.073837+0000","last_undegraded":"2023-11-28T11:08:19.073837+0000","last_fullsized":"2023-11-28T11:08:19.073837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:09.760690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357896+0000","last_change":"2023-11-28T11:08:16.976378+0000","last_active":"2023-11-28T11:08:17.357896+0000","last_peered":"2023-11-28T11:08:17.357896+0000","last_clean":"2023-11-28T11:08:17.357896+0000","last_became_active":"2023-11-28T11:08:16.974426+0000","last_became_peered":"2023-11-28T11:08:16.974426+0000","last_unstale":"2023-11-28T11:08:17.357896+0000","last_undegraded":"2023-11-28T11:08:17.357896+0000","last_fullsized":"2023-11-28T11:08:17.357896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673645+0000","last_change":"2023-11-28T11:08:17.366381+0000","last_active":"2023-11-28T11:08:18.673645+0000","last_peered":"2023-11-28T11:08:18.673645+0000","last_clean":"2023-11-28T11:08:18.673645+0000","last_became_active":"2023-11-28T11:08:17.366129+0000","last_became_peered":"2023-11-28T11:08:17.366129+0000","last_unstale":"2023-11-28T11:08:18.673645+0000","last_undegraded":"2023-11-28T11:08:18.673645+0000","last_fullsized":"2023-11-28T11:08:18.673645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.380305+0000","last_change":"2023-11-28T11:08:16.973851+0000","last_active":"2023-11-28T11:08:17.380305+0000","last_peered":"2023-11-28T11:08:17.380305+0000","last_clean":"2023-11-28T11:08:17.380305+0000","last_became_active":"2023-11-28T11:08:16.973588+0000","last_became_peered":"2023-11-28T11:08:16.973588+0000","last_unstale":"2023-11-28T11:08:17.380305+0000","last_undegraded":"2023-11-28T11:08:17.380305+0000","last_fullsized":"2023-11-28T11:08:17.380305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369164+0000","last_change":"2023-11-28T11:08:17.372448+0000","last_active":"2023-11-28T11:08:18.369164+0000","last_peered":"2023-11-28T11:08:18.369164+0000","last_clean":"2023-11-28T11:08:18.369164+0000","last_became_active":"2023-11-28T11:08:17.372132+0000","last_became_peered":"2023-11-28T11:08:17.372132+0000","last_unstale":"2023-11-28T11:08:18.369164+0000","last_undegraded":"2023-11-28T11:08:18.369164+0000","last_fullsized":"2023-11-28T11:08:18.369164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:12.679494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.371186+0000","last_change":"2023-11-28T11:08:16.975018+0000","last_active":"2023-11-28T11:08:17.371186+0000","last_peered":"2023-11-28T11:08:17.371186+0000","last_clean":"2023-11-28T11:08:17.371186+0000","last_became_active":"2023-11-28T11:08:16.974709+0000","last_became_peered":"2023-11-28T11:08:16.974709+0000","last_unstale":"2023-11-28T11:08:17.371186+0000","last_undegraded":"2023-11-28T11:08:17.371186+0000","last_fullsized":"2023-11-28T11:08:17.371186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369338+0000","last_change":"2023-11-28T11:08:17.372568+0000","last_active":"2023-11-28T11:08:18.369338+0000","last_peered":"2023-11-28T11:08:18.369338+0000","last_clean":"2023-11-28T11:08:18.369338+0000","last_became_active":"2023-11-28T11:08:17.372279+0000","last_became_peered":"2023-11-28T11:08:17.372279+0000","last_unstale":"2023-11-28T11:08:18.369338+0000","last_undegraded":"2023-11-28T11:08:18.369338+0000","last_fullsized":"2023-11-28T11:08:18.369338+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:02.566272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364207+0000","last_change":"2023-11-28T11:08:16.976973+0000","last_active":"2023-11-28T11:08:17.364207+0000","last_peered":"2023-11-28T11:08:17.364207+0000","last_clean":"2023-11-28T11:08:17.364207+0000","last_became_active":"2023-11-28T11:08:16.976661+0000","last_became_peered":"2023-11-28T11:08:16.976661+0000","last_unstale":"2023-11-28T11:08:17.364207+0000","last_undegraded":"2023-11-28T11:08:17.364207+0000","last_fullsized":"2023-11-28T11:08:17.364207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:43.575239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369260+0000","last_change":"2023-11-28T11:08:17.381198+0000","last_active":"2023-11-28T11:08:18.369260+0000","last_peered":"2023-11-28T11:08:18.369260+0000","last_clean":"2023-11-28T11:08:18.369260+0000","last_became_active":"2023-11-28T11:08:17.380937+0000","last_became_peered":"2023-11-28T11:08:17.380937+0000","last_unstale":"2023-11-28T11:08:18.369260+0000","last_undegraded":"2023-11-28T11:08:18.369260+0000","last_fullsized":"2023-11-28T11:08:18.369260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.355350+0000","last_change":"2023-11-28T11:08:16.966279+0000","last_active":"2023-11-28T11:08:17.355350+0000","last_peered":"2023-11-28T11:08:17.355350+0000","last_clean":"2023-11-28T11:08:17.355350+0000","last_became_active":"2023-11-28T11:08:16.965878+0000","last_became_peered":"2023-11-28T11:08:16.965878+0000","last_unstale":"2023-11-28T11:08:17.355350+0000","last_undegraded":"2023-11-28T11:08:17.355350+0000","last_fullsized":"2023-11-28T11:08:17.355350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368520+0000","last_change":"2023-11-28T11:08:17.373270+0000","last_active":"2023-11-28T11:08:18.368520+0000","last_peered":"2023-11-28T11:08:18.368520+0000","last_clean":"2023-11-28T11:08:18.368520+0000","last_became_active":"2023-11-28T11:08:17.373017+0000","last_became_peered":"2023-11-28T11:08:17.373017+0000","last_unstale":"2023-11-28T11:08:18.368520+0000","last_undegraded":"2023-11-28T11:08:18.368520+0000","last_fullsized":"2023-11-28T11:08:18.368520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365774+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:08:17.365774+0000","last_peered":"2023-11-28T11:08:17.365774+0000","last_clean":"2023-11-28T11:08:17.365774+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:08:17.365774+0000","last_undegraded":"2023-11-28T11:08:17.365774+0000","last_fullsized":"2023-11-28T11:08:17.365774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.358583+0000","last_change":"2023-11-28T11:08:16.975874+0000","last_active":"2023-11-28T11:08:17.358583+0000","last_peered":"2023-11-28T11:08:17.358583+0000","last_clean":"2023-11-28T11:08:17.358583+0000","last_became_active":"2023-11-28T11:08:16.975225+0000","last_became_peered":"2023-11-28T11:08:16.975225+0000","last_unstale":"2023-11-28T11:08:17.358583+0000","last_undegraded":"2023-11-28T11:08:17.358583+0000","last_fullsized":"2023-11-28T11:08:17.358583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368149+0000","last_change":"2023-11-28T11:08:17.371631+0000","last_active":"2023-11-28T11:08:18.368149+0000","last_peered":"2023-11-28T11:08:18.368149+0000","last_clean":"2023-11-28T11:08:18.368149+0000","last_became_active":"2023-11-28T11:08:17.371342+0000","last_became_peered":"2023-11-28T11:08:17.371342+0000","last_unstale":"2023-11-28T11:08:18.368149+0000","last_undegraded":"2023-11-28T11:08:18.368149+0000","last_fullsized":"2023-11-28T11:08:18.368149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357509+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:08:17.357509+0000","last_peered":"2023-11-28T11:08:17.357509+0000","last_clean":"2023-11-28T11:08:17.357509+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:08:17.357509+0000","last_undegraded":"2023-11-28T11:08:17.357509+0000","last_fullsized":"2023-11-28T11:08:17.357509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362711+0000","last_change":"2023-11-28T11:08:16.974904+0000","last_active":"2023-11-28T11:08:17.362711+0000","last_peered":"2023-11-28T11:08:17.362711+0000","last_clean":"2023-11-28T11:08:17.362711+0000","last_became_active":"2023-11-28T11:08:16.974634+0000","last_became_peered":"2023-11-28T11:08:16.974634+0000","last_unstale":"2023-11-28T11:08:17.362711+0000","last_undegraded":"2023-11-28T11:08:17.362711+0000","last_fullsized":"2023-11-28T11:08:17.362711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368410+0000","last_change":"2023-11-28T11:08:17.371181+0000","last_active":"2023-11-28T11:08:18.368410+0000","last_peered":"2023-11-28T11:08:18.368410+0000","last_clean":"2023-11-28T11:08:18.368410+0000","last_became_active":"2023-11-28T11:08:17.370931+0000","last_became_peered":"2023-11-28T11:08:17.370931+0000","last_unstale":"2023-11-28T11:08:18.368410+0000","last_undegraded":"2023-11-28T11:08:18.368410+0000","last_fullsized":"2023-11-28T11:08:18.368410+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.359510+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:08:17.359510+0000","last_peered":"2023-11-28T11:08:17.359510+0000","last_clean":"2023-11-28T11:08:17.359510+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:08:17.359510+0000","last_undegraded":"2023-11-28T11:08:17.359510+0000","last_fullsized":"2023-11-28T11:08:17.359510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362780+0000","last_change":"2023-11-28T11:08:16.972359+0000","last_active":"2023-11-28T11:08:17.362780+0000","last_peered":"2023-11-28T11:08:17.362780+0000","last_clean":"2023-11-28T11:08:17.362780+0000","last_became_active":"2023-11-28T11:08:16.972074+0000","last_became_peered":"2023-11-28T11:08:16.972074+0000","last_unstale":"2023-11-28T11:08:17.362780+0000","last_undegraded":"2023-11-28T11:08:17.362780+0000","last_fullsized":"2023-11-28T11:08:17.362780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368494+0000","last_change":"2023-11-28T11:08:17.369522+0000","last_active":"2023-11-28T11:08:18.368494+0000","last_peered":"2023-11-28T11:08:18.368494+0000","last_clean":"2023-11-28T11:08:18.368494+0000","last_became_active":"2023-11-28T11:08:17.368628+0000","last_became_peered":"2023-11-28T11:08:17.368628+0000","last_unstale":"2023-11-28T11:08:18.368494+0000","last_undegraded":"2023-11-28T11:08:18.368494+0000","last_fullsized":"2023-11-28T11:08:18.368494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357776+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:08:17.357776+0000","last_peered":"2023-11-28T11:08:17.357776+0000","last_clean":"2023-11-28T11:08:17.357776+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:08:17.357776+0000","last_undegraded":"2023-11-28T11:08:17.357776+0000","last_fullsized":"2023-11-28T11:08:17.357776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364014+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:08:17.364014+0000","last_peered":"2023-11-28T11:08:17.364014+0000","last_clean":"2023-11-28T11:08:17.364014+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:08:17.364014+0000","last_undegraded":"2023-11-28T11:08:17.364014+0000","last_fullsized":"2023-11-28T11:08:17.364014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.355796+0000","last_change":"2023-11-28T11:08:16.968819+0000","last_active":"2023-11-28T11:08:17.355796+0000","last_peered":"2023-11-28T11:08:17.355796+0000","last_clean":"2023-11-28T11:08:17.355796+0000","last_became_active":"2023-11-28T11:08:16.968536+0000","last_became_peered":"2023-11-28T11:08:16.968536+0000","last_unstale":"2023-11-28T11:08:17.355796+0000","last_undegraded":"2023-11-28T11:08:17.355796+0000","last_fullsized":"2023-11-28T11:08:17.355796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073714+0000","last_change":"2023-11-28T11:08:17.381098+0000","last_active":"2023-11-28T11:08:19.073714+0000","last_peered":"2023-11-28T11:08:19.073714+0000","last_clean":"2023-11-28T11:08:19.073714+0000","last_became_active":"2023-11-28T11:08:17.380823+0000","last_became_peered":"2023-11-28T11:08:17.380823+0000","last_unstale":"2023-11-28T11:08:19.073714+0000","last_undegraded":"2023-11-28T11:08:19.073714+0000","last_fullsized":"2023-11-28T11:08:19.073714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:56.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364256+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:08:17.364256+0000","last_peered":"2023-11-28T11:08:17.364256+0000","last_clean":"2023-11-28T11:08:17.364256+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:08:17.364256+0000","last_undegraded":"2023-11-28T11:08:17.364256+0000","last_fullsized":"2023-11-28T11:08:17.364256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.360038+0000","last_change":"2023-11-28T11:08:16.978195+0000","last_active":"2023-11-28T11:08:17.360038+0000","last_peered":"2023-11-28T11:08:17.360038+0000","last_clean":"2023-11-28T11:08:17.360038+0000","last_became_active":"2023-11-28T11:08:16.977906+0000","last_became_peered":"2023-11-28T11:08:16.977906+0000","last_unstale":"2023-11-28T11:08:17.360038+0000","last_undegraded":"2023-11-28T11:08:17.360038+0000","last_fullsized":"2023-11-28T11:08:17.360038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673777+0000","last_change":"2023-11-28T11:08:17.369849+0000","last_active":"2023-11-28T11:08:18.673777+0000","last_peered":"2023-11-28T11:08:18.673777+0000","last_clean":"2023-11-28T11:08:18.673777+0000","last_became_active":"2023-11-28T11:08:17.369614+0000","last_became_peered":"2023-11-28T11:08:17.369614+0000","last_unstale":"2023-11-28T11:08:18.673777+0000","last_undegraded":"2023-11-28T11:08:18.673777+0000","last_fullsized":"2023-11-28T11:08:18.673777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.368123+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:08:17.368123+0000","last_peered":"2023-11-28T11:08:17.368123+0000","last_clean":"2023-11-28T11:08:17.368123+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:08:17.368123+0000","last_undegraded":"2023-11-28T11:08:17.368123+0000","last_fullsized":"2023-11-28T11:08:17.368123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365891+0000","last_change":"2023-11-28T11:08:16.975131+0000","last_active":"2023-11-28T11:08:17.365891+0000","last_peered":"2023-11-28T11:08:17.365891+0000","last_clean":"2023-11-28T11:08:17.365891+0000","last_became_active":"2023-11-28T11:08:16.974846+0000","last_became_peered":"2023-11-28T11:08:16.974846+0000","last_unstale":"2023-11-28T11:08:17.365891+0000","last_undegraded":"2023-11-28T11:08:17.365891+0000","last_fullsized":"2023-11-28T11:08:17.365891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673513+0000","last_change":"2023-11-28T11:08:17.366819+0000","last_active":"2023-11-28T11:08:18.673513+0000","last_peered":"2023-11-28T11:08:18.673513+0000","last_clean":"2023-11-28T11:08:18.673513+0000","last_became_active":"2023-11-28T11:08:17.366576+0000","last_became_peered":"2023-11-28T11:08:17.366576+0000","last_unstale":"2023-11-28T11:08:18.673513+0000","last_undegraded":"2023-11-28T11:08:18.673513+0000","last_fullsized":"2023-11-28T11:08:18.673513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.379003+0000","last_change":"2023-11-28T11:08:16.973768+0000","last_active":"2023-11-28T11:08:17.379003+0000","last_peered":"2023-11-28T11:08:17.379003+0000","last_clean":"2023-11-28T11:08:17.379003+0000","last_became_active":"2023-11-28T11:08:16.973505+0000","last_became_peered":"2023-11-28T11:08:16.973505+0000","last_unstale":"2023-11-28T11:08:17.379003+0000","last_undegraded":"2023-11-28T11:08:17.379003+0000","last_fullsized":"2023-11-28T11:08:17.379003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.367956+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:08:17.367956+0000","last_peered":"2023-11-28T11:08:17.367956+0000","last_clean":"2023-11-28T11:08:17.367956+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:08:17.367956+0000","last_undegraded":"2023-11-28T11:08:17.367956+0000","last_fullsized":"2023-11-28T11:08:17.367956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365501+0000","last_change":"2023-11-28T11:08:17.375741+0000","last_active":"2023-11-28T11:08:18.365501+0000","last_peered":"2023-11-28T11:08:18.365501+0000","last_clean":"2023-11-28T11:08:18.365501+0000","last_became_active":"2023-11-28T11:08:17.375349+0000","last_became_peered":"2023-11-28T11:08:17.375349+0000","last_unstale":"2023-11-28T11:08:18.365501+0000","last_undegraded":"2023-11-28T11:08:18.365501+0000","last_fullsized":"2023-11-28T11:08:18.365501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.372744+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:08:17.372744+0000","last_peered":"2023-11-28T11:08:17.372744+0000","last_clean":"2023-11-28T11:08:17.372744+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:08:17.372744+0000","last_undegraded":"2023-11-28T11:08:17.372744+0000","last_fullsized":"2023-11-28T11:08:17.372744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673785+0000","last_change":"2023-11-28T11:08:16.974692+0000","last_active":"2023-11-28T11:08:18.673785+0000","last_peered":"2023-11-28T11:08:18.673785+0000","last_clean":"2023-11-28T11:08:18.673785+0000","last_became_active":"2023-11-28T11:08:16.974255+0000","last_became_peered":"2023-11-28T11:08:16.974255+0000","last_unstale":"2023-11-28T11:08:18.673785+0000","last_undegraded":"2023-11-28T11:08:18.673785+0000","last_fullsized":"2023-11-28T11:08:18.673785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368842+0000","last_change":"2023-11-28T11:08:17.373312+0000","last_active":"2023-11-28T11:08:18.368842+0000","last_peered":"2023-11-28T11:08:18.368842+0000","last_clean":"2023-11-28T11:08:18.368842+0000","last_became_active":"2023-11-28T11:08:17.373033+0000","last_became_peered":"2023-11-28T11:08:17.373033+0000","last_unstale":"2023-11-28T11:08:18.368842+0000","last_undegraded":"2023-11-28T11:08:18.368842+0000","last_fullsized":"2023-11-28T11:08:18.368842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:25.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377530+0000","last_change":"2023-11-28T11:08:16.976409+0000","last_active":"2023-11-28T11:08:17.377530+0000","last_peered":"2023-11-28T11:08:17.377530+0000","last_clean":"2023-11-28T11:08:17.377530+0000","last_became_active":"2023-11-28T11:08:16.976127+0000","last_became_peered":"2023-11-28T11:08:16.976127+0000","last_unstale":"2023-11-28T11:08:17.377530+0000","last_undegraded":"2023-11-28T11:08:17.377530+0000","last_fullsized":"2023-11-28T11:08:17.377530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:11.421587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072815+0000","last_change":"2023-11-28T11:08:17.374219+0000","last_active":"2023-11-28T11:08:19.072815+0000","last_peered":"2023-11-28T11:08:19.072815+0000","last_clean":"2023-11-28T11:08:19.072815+0000","last_became_active":"2023-11-28T11:08:17.374010+0000","last_became_peered":"2023-11-28T11:08:17.374010+0000","last_unstale":"2023-11-28T11:08:19.072815+0000","last_undegraded":"2023-11-28T11:08:19.072815+0000","last_fullsized":"2023-11-28T11:08:19.072815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:50.027648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.375928+0000","last_change":"2023-11-28T11:08:16.978678+0000","last_active":"2023-11-28T11:08:17.375928+0000","last_peered":"2023-11-28T11:08:17.375928+0000","last_clean":"2023-11-28T11:08:17.375928+0000","last_became_active":"2023-11-28T11:08:16.978317+0000","last_became_peered":"2023-11-28T11:08:16.978317+0000","last_unstale":"2023-11-28T11:08:17.375928+0000","last_undegraded":"2023-11-28T11:08:17.375928+0000","last_fullsized":"2023-11-28T11:08:17.375928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368851+0000","last_change":"2023-11-28T11:08:17.373647+0000","last_active":"2023-11-28T11:08:18.368851+0000","last_peered":"2023-11-28T11:08:18.368851+0000","last_clean":"2023-11-28T11:08:18.368851+0000","last_became_active":"2023-11-28T11:08:17.373288+0000","last_became_peered":"2023-11-28T11:08:17.373288+0000","last_unstale":"2023-11-28T11:08:18.368851+0000","last_undegraded":"2023-11-28T11:08:18.368851+0000","last_fullsized":"2023-11-28T11:08:18.368851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:59.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.376913+0000","last_change":"2023-11-28T11:08:16.976048+0000","last_active":"2023-11-28T11:08:17.376913+0000","last_peered":"2023-11-28T11:08:17.376913+0000","last_clean":"2023-11-28T11:08:17.376913+0000","last_became_active":"2023-11-28T11:08:16.975600+0000","last_became_peered":"2023-11-28T11:08:16.975600+0000","last_unstale":"2023-11-28T11:08:17.376913+0000","last_undegraded":"2023-11-28T11:08:17.376913+0000","last_fullsized":"2023-11-28T11:08:17.376913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368940+0000","last_change":"2023-11-28T11:08:17.372086+0000","last_active":"2023-11-28T11:08:18.368940+0000","last_peered":"2023-11-28T11:08:18.368940+0000","last_clean":"2023-11-28T11:08:18.368940+0000","last_became_active":"2023-11-28T11:08:17.371833+0000","last_became_peered":"2023-11-28T11:08:17.371833+0000","last_unstale":"2023-11-28T11:08:18.368940+0000","last_undegraded":"2023-11-28T11:08:18.368940+0000","last_fullsized":"2023-11-28T11:08:18.368940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:46.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.380725+0000","last_change":"2023-11-28T11:08:16.964126+0000","last_active":"2023-11-28T11:08:17.380725+0000","last_peered":"2023-11-28T11:08:17.380725+0000","last_clean":"2023-11-28T11:08:17.380725+0000","last_became_active":"2023-11-28T11:08:16.963838+0000","last_became_peered":"2023-11-28T11:08:16.963838+0000","last_unstale":"2023-11-28T11:08:17.380725+0000","last_undegraded":"2023-11-28T11:08:17.380725+0000","last_fullsized":"2023-11-28T11:08:17.380725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365611+0000","last_change":"2023-11-28T11:08:17.361919+0000","last_active":"2023-11-28T11:08:18.365611+0000","last_peered":"2023-11-28T11:08:18.365611+0000","last_clean":"2023-11-28T11:08:18.365611+0000","last_became_active":"2023-11-28T11:08:17.361654+0000","last_became_peered":"2023-11-28T11:08:17.361654+0000","last_unstale":"2023-11-28T11:08:18.365611+0000","last_undegraded":"2023-11-28T11:08:18.365611+0000","last_fullsized":"2023-11-28T11:08:18.365611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:46.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.372346+0000","last_change":"2023-11-28T11:08:16.974232+0000","last_active":"2023-11-28T11:08:17.372346+0000","last_peered":"2023-11-28T11:08:17.372346+0000","last_clean":"2023-11-28T11:08:17.372346+0000","last_became_active":"2023-11-28T11:08:16.972983+0000","last_became_peered":"2023-11-28T11:08:16.972983+0000","last_unstale":"2023-11-28T11:08:17.372346+0000","last_undegraded":"2023-11-28T11:08:17.372346+0000","last_fullsized":"2023-11-28T11:08:17.372346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368650+0000","last_change":"2023-11-28T11:08:17.386923+0000","last_active":"2023-11-28T11:08:18.368650+0000","last_peered":"2023-11-28T11:08:18.368650+0000","last_clean":"2023-11-28T11:08:18.368650+0000","last_became_active":"2023-11-28T11:08:17.386690+0000","last_became_peered":"2023-11-28T11:08:17.386690+0000","last_unstale":"2023-11-28T11:08:18.368650+0000","last_undegraded":"2023-11-28T11:08:18.368650+0000","last_fullsized":"2023-11-28T11:08:18.368650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:35.096193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074332+0000","last_change":"2023-11-28T11:08:17.385964+0000","last_active":"2023-11-28T11:08:19.074332+0000","last_peered":"2023-11-28T11:08:19.074332+0000","last_clean":"2023-11-28T11:08:19.074332+0000","last_became_active":"2023-11-28T11:08:17.385673+0000","last_became_peered":"2023-11-28T11:08:17.385673+0000","last_unstale":"2023-11-28T11:08:19.074332+0000","last_undegraded":"2023-11-28T11:08:19.074332+0000","last_fullsized":"2023-11-28T11:08:19.074332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:37.983570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365540+0000","last_change":"2023-11-28T11:08:17.372667+0000","last_active":"2023-11-28T11:08:18.365540+0000","last_peered":"2023-11-28T11:08:18.365540+0000","last_clean":"2023-11-28T11:08:18.365540+0000","last_became_active":"2023-11-28T11:08:17.372363+0000","last_became_peered":"2023-11-28T11:08:17.372363+0000","last_unstale":"2023-11-28T11:08:18.365540+0000","last_undegraded":"2023-11-28T11:08:18.365540+0000","last_fullsized":"2023-11-28T11:08:18.365540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:25.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368769+0000","last_change":"2023-11-28T11:08:17.386764+0000","last_active":"2023-11-28T11:08:18.368769+0000","last_peered":"2023-11-28T11:08:18.368769+0000","last_clean":"2023-11-28T11:08:18.368769+0000","last_became_active":"2023-11-28T11:08:17.386525+0000","last_became_peered":"2023-11-28T11:08:17.386525+0000","last_unstale":"2023-11-28T11:08:18.368769+0000","last_undegraded":"2023-11-28T11:08:18.368769+0000","last_fullsized":"2023-11-28T11:08:18.368769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.366116+0000","last_change":"2023-11-28T11:08:17.378211+0000","last_active":"2023-11-28T11:08:18.366116+0000","last_peered":"2023-11-28T11:08:18.366116+0000","last_clean":"2023-11-28T11:08:18.366116+0000","last_became_active":"2023-11-28T11:08:17.376746+0000","last_became_peered":"2023-11-28T11:08:17.376746+0000","last_unstale":"2023-11-28T11:08:18.366116+0000","last_undegraded":"2023-11-28T11:08:18.366116+0000","last_fullsized":"2023-11-28T11:08:18.366116+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368303+0000","last_change":"2023-11-28T11:08:17.375172+0000","last_active":"2023-11-28T11:08:18.368303+0000","last_peered":"2023-11-28T11:08:18.368303+0000","last_clean":"2023-11-28T11:08:18.368303+0000","last_became_active":"2023-11-28T11:08:17.374911+0000","last_became_peered":"2023-11-28T11:08:17.374911+0000","last_unstale":"2023-11-28T11:08:18.368303+0000","last_undegraded":"2023-11-28T11:08:18.368303+0000","last_fullsized":"2023-11-28T11:08:18.368303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9108,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362732,"statfs":{"total":96636764160,"available":96627437568,"internally_reserved":0,"allocated":864256,"data_stored":681731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7988,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363852,"statfs":{"total":96636764160,"available":96628584448,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1239999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.0250000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.0620000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7988,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363852,"statfs":{"total":96636764160,"available":96628584448,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.224}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":680533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7924,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363916,"statfs":{"total":96636764160,"available":96628649984,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8056,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363784,"statfs":{"total":96636764160,"available":96628514816,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":91203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7996,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363844,"statfs":{"total":96636764160,"available":96628576256,"internally_reserved":0,"allocated":249856,"data_stored":90235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:08:30.733 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:08:30.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.889+0000 7f210154a700 1 Processor -- start 2023-11-28T11:08:30.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.889+0000 7f210154a700 1 -- start start 2023-11-28T11:08:30.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.890+0000 7f210154a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f20fc06dee0 0x7f20fc06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.890+0000 7f210154a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20fc160540 0x7f20fc1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.890+0000 7f210154a700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20fc15d060 con 0x7f20fc071040 2023-11-28T11:08:30.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.890+0000 7f210154a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20fc15d340 con 0x7f20fc06dee0 2023-11-28T11:08:30.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.890+0000 7f210154a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc15d620 con 0x7f20fc160540 2023-11-28T11:08:30.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.890+0000 7f20fa7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f20fc06dee0 0x7f20fc06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.890+0000 7f20fa7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f20fc06dee0 0x7f20fc06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60552/0 (socket says 172.21.15.156:60552) 2023-11-28T11:08:30.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.890+0000 7f20fa7fc700 1 -- 172.21.15.156:0/2847723452 learned_addr learned my addr 172.21.15.156:0/2847723452 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:30.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.891+0000 7f20fb7fe700 1 --2- 172.21.15.156:0/2847723452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20fc160540 0x7f20fc1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.891+0000 7f20fa7fc700 1 -- 172.21.15.156:0/2847723452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20fc160540 msgr2=0x7f20fc1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:30.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.891+0000 7f20fa7fc700 1 --2- 172.21.15.156:0/2847723452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20fc160540 0x7f20fc1609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.891+0000 7f20fa7fc700 1 -- 172.21.15.156:0/2847723452 >> v1:172.21.15.186:6789/0 conn(0x7f20fc071040 legacy=0x7f20fc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.891+0000 7f20fa7fc700 1 -- 172.21.15.156:0/2847723452 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc15d900 con 0x7f20fc06dee0 2023-11-28T11:08:30.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.891+0000 7f20fa7fc700 1 --2- 172.21.15.156:0/2847723452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20fc06dee0 0x7f20fc06e330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0098f0 tx=0x7f20ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e999215ce26d395 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.891+0000 7f20f9ffb700 1 -- 172.21.15.156:0/2847723452 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00b8a0 con 0x7f20fc06dee0 2023-11-28T11:08:30.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.891+0000 7f20f9ffb700 1 -- 172.21.15.156:0/2847723452 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec00ba00 con 0x7f20fc06dee0 2023-11-28T11:08:30.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.891+0000 7f20f9ffb700 1 -- 172.21.15.156:0/2847723452 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00bd90 con 0x7f20fc06dee0 2023-11-28T11:08:30.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.891+0000 7f210154a700 1 -- 172.21.15.156:0/2847723452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20fc06dee0 msgr2=0x7f20fc06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.891+0000 7f210154a700 1 --2- 172.21.15.156:0/2847723452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20fc06dee0 0x7f20fc06e330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0098f0 tx=0x7f20ec009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:30.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.892+0000 7f210154a700 1 -- 172.21.15.156:0/2847723452 shutdown_connections 2023-11-28T11:08:30.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.892+0000 7f210154a700 1 --2- 172.21.15.156:0/2847723452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20fc160540 0x7f20fc1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.892+0000 7f210154a700 1 --2- 172.21.15.156:0/2847723452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20fc06dee0 0x7f20fc06e330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.892+0000 7f210154a700 1 -- 172.21.15.156:0/2847723452 >> 172.21.15.156:0/2847723452 conn(0x7f20fc0d0000 msgr2=0x7f20fc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:30.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.892+0000 7f210154a700 1 -- 172.21.15.156:0/2847723452 shutdown_connections 2023-11-28T11:08:30.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.892+0000 7f210154a700 1 -- 172.21.15.156:0/2847723452 wait complete. 2023-11-28T11:08:30.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.893+0000 7f210154a700 1 Processor -- start 2023-11-28T11:08:30.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.893+0000 7f210154a700 1 -- start start 2023-11-28T11:08:30.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f210154a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20fc06dee0 0x7f20fc152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f210154a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20fc071040 0x7f20fc0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f210154a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20fc160540 0x7f20fc0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f210154a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0c6020 con 0x7f20fc06dee0 2023-11-28T11:08:30.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f210154a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc1f6cb0 con 0x7f20fc160540 2023-11-28T11:08:30.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f210154a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc1f63c0 con 0x7f20fc071040 2023-11-28T11:08:30.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20faffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20fc06dee0 0x7f20fc152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20faffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20fc06dee0 0x7f20fc152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60556/0 (socket says 172.21.15.156:60556) 2023-11-28T11:08:30.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20faffd700 1 -- 172.21.15.156:0/962912243 learned_addr learned my addr 172.21.15.156:0/962912243 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:30.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20faffd700 1 -- 172.21.15.156:0/962912243 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20fc071040 msgr2=0x7f20fc0b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20faffd700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20fc071040 0x7f20fc0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20faffd700 1 -- 172.21.15.156:0/962912243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20fc160540 msgr2=0x7f20fc0b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20fb7fe700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20fc160540 0x7f20fc0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20faffd700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20fc160540 0x7f20fc0b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20faffd700 1 -- 172.21.15.156:0/962912243 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec009550 con 0x7f20fc06dee0 2023-11-28T11:08:30.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20faffd700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20fc06dee0 0x7f20fc152be0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f20f000c450 tx=0x7f20f000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20e37fe700 1 -- 172.21.15.156:0/962912243 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f00052d0 con 0x7f20fc06dee0 2023-11-28T11:08:30.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20e37fe700 1 -- 172.21.15.156:0/962912243 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f0005430 con 0x7f20fc06dee0 2023-11-28T11:08:30.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f20e37fe700 1 -- 172.21.15.156:0/962912243 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f00038c0 con 0x7f20fc06dee0 2023-11-28T11:08:30.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f210154a700 1 -- 172.21.15.156:0/962912243 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc15ccd0 con 0x7f20fc06dee0 2023-11-28T11:08:30.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.894+0000 7f210154a700 1 -- 172.21.15.156:0/962912243 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc06e870 con 0x7f20fc06dee0 2023-11-28T11:08:30.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.895+0000 7f210154a700 1 -- 172.21.15.156:0/962912243 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20fc06a270 con 0x7f20fc06dee0 2023-11-28T11:08:30.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.896+0000 7f20e37fe700 1 -- 172.21.15.156:0/962912243 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f20f0019030 con 0x7f20fc06dee0 2023-11-28T11:08:30.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.896+0000 7f20e37fe700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20e8068320 0x7f20e806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.896+0000 7f20e37fe700 1 -- 172.21.15.156:0/962912243 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f20f0084f40 con 0x7f20fc06dee0 2023-11-28T11:08:30.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.896+0000 7f20fa7fc700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20e8068320 0x7f20e806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.897+0000 7f20fa7fc700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20e8068320 0x7f20e806a7d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0727f0 tx=0x7f20ec010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:30.900+0000 7f20e37fe700 1 -- 172.21.15.156:0/962912243 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f20f00517a0 con 0x7f20fc06dee0 2023-11-28T11:08:31.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.078+0000 7f210154a700 1 -- 172.21.15.156:0/962912243 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f20fc153120 con 0x7f20e8068320 2023-11-28T11:08:31.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.086+0000 7f20e37fe700 1 -- 172.21.15.156:0/962912243 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286621 (secure 0 0 0) 0x7f20fc153120 con 0x7f20e8068320 2023-11-28T11:08:31.090 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.089+0000 7f210154a700 1 -- 172.21.15.156:0/962912243 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20e8068320 msgr2=0x7f20e806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.089+0000 7f210154a700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20e8068320 0x7f20e806a7d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0727f0 tx=0x7f20ec010000 comp rx=0 tx=0).stop 2023-11-28T11:08:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.089+0000 7f210154a700 1 -- 172.21.15.156:0/962912243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20fc06dee0 msgr2=0x7f20fc152be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.089+0000 7f210154a700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20fc06dee0 0x7f20fc152be0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f20f000c450 tx=0x7f20f000c810 comp rx=0 tx=0).stop 2023-11-28T11:08:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.089+0000 7f210154a700 1 -- 172.21.15.156:0/962912243 shutdown_connections 2023-11-28T11:08:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.089+0000 7f210154a700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20fc160540 0x7f20fc0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.089+0000 7f210154a700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20fc06dee0 0x7f20fc152be0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.089+0000 7f210154a700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20e8068320 0x7f20e806a7d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.089+0000 7f210154a700 1 --2- 172.21.15.156:0/962912243 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20fc071040 0x7f20fc0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.089+0000 7f210154a700 1 -- 172.21.15.156:0/962912243 >> 172.21.15.156:0/962912243 conn(0x7f20fc0d0000 msgr2=0x7f20fc071410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.089+0000 7f210154a700 1 -- 172.21.15.156:0/962912243 shutdown_connections 2023-11-28T11:08:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.089+0000 7f210154a700 1 -- 172.21.15.156:0/962912243 wait complete. 2023-11-28T11:08:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:08:31.103 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-11-28T11:08:30.528645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66216,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":754908504,"statfs":{"total":773094113280,"available":773026308096,"internally_reserved":0,"allocated":3186688,"data_stored":1902188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003562"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072799+0000","last_change":"2023-11-28T11:08:17.367751+0000","last_active":"2023-11-28T11:08:19.072799+0000","last_peered":"2023-11-28T11:08:19.072799+0000","last_clean":"2023-11-28T11:08:19.072799+0000","last_became_active":"2023-11-28T11:08:17.367481+0000","last_became_peered":"2023-11-28T11:08:17.367481+0000","last_unstale":"2023-11-28T11:08:19.072799+0000","last_undegraded":"2023-11-28T11:08:19.072799+0000","last_fullsized":"2023-11-28T11:08:19.072799+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:09.760690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073854+0000","last_change":"2023-11-28T11:08:17.369749+0000","last_active":"2023-11-28T11:08:19.073854+0000","last_peered":"2023-11-28T11:08:19.073854+0000","last_clean":"2023-11-28T11:08:19.073854+0000","last_became_active":"2023-11-28T11:08:17.369479+0000","last_became_peered":"2023-11-28T11:08:17.369479+0000","last_unstale":"2023-11-28T11:08:19.073854+0000","last_undegraded":"2023-11-28T11:08:19.073854+0000","last_fullsized":"2023-11-28T11:08:19.073854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369159+0000","last_change":"2023-11-28T11:08:17.370726+0000","last_active":"2023-11-28T11:08:18.369159+0000","last_peered":"2023-11-28T11:08:18.369159+0000","last_clean":"2023-11-28T11:08:18.369159+0000","last_became_active":"2023-11-28T11:08:17.367062+0000","last_became_peered":"2023-11-28T11:08:17.367062+0000","last_unstale":"2023-11-28T11:08:18.369159+0000","last_undegraded":"2023-11-28T11:08:18.369159+0000","last_fullsized":"2023-11-28T11:08:18.369159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:27.219209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073872+0000","last_change":"2023-11-28T11:08:17.383989+0000","last_active":"2023-11-28T11:08:19.073872+0000","last_peered":"2023-11-28T11:08:19.073872+0000","last_clean":"2023-11-28T11:08:19.073872+0000","last_became_active":"2023-11-28T11:08:17.381618+0000","last_became_peered":"2023-11-28T11:08:17.381618+0000","last_unstale":"2023-11-28T11:08:19.073872+0000","last_undegraded":"2023-11-28T11:08:19.073872+0000","last_fullsized":"2023-11-28T11:08:19.073872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:09:00.918897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673454+0000","last_change":"2023-11-28T11:08:17.377110+0000","last_active":"2023-11-28T11:08:18.673454+0000","last_peered":"2023-11-28T11:08:18.673454+0000","last_clean":"2023-11-28T11:08:18.673454+0000","last_became_active":"2023-11-28T11:08:17.373583+0000","last_became_peered":"2023-11-28T11:08:17.373583+0000","last_unstale":"2023-11-28T11:08:18.673454+0000","last_undegraded":"2023-11-28T11:08:18.673454+0000","last_fullsized":"2023-11-28T11:08:18.673454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:24:59.277184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368780+0000","last_change":"2023-11-28T11:08:17.370700+0000","last_active":"2023-11-28T11:08:18.368780+0000","last_peered":"2023-11-28T11:08:18.368780+0000","last_clean":"2023-11-28T11:08:18.368780+0000","last_became_active":"2023-11-28T11:08:17.367222+0000","last_became_peered":"2023-11-28T11:08:17.367222+0000","last_unstale":"2023-11-28T11:08:18.368780+0000","last_undegraded":"2023-11-28T11:08:18.368780+0000","last_fullsized":"2023-11-28T11:08:18.368780+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:22.140101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073497+0000","last_change":"2023-11-28T11:08:17.374848+0000","last_active":"2023-11-28T11:08:19.073497+0000","last_peered":"2023-11-28T11:08:19.073497+0000","last_clean":"2023-11-28T11:08:19.073497+0000","last_became_active":"2023-11-28T11:08:17.374536+0000","last_became_peered":"2023-11-28T11:08:17.374536+0000","last_unstale":"2023-11-28T11:08:19.073497+0000","last_undegraded":"2023-11-28T11:08:19.073497+0000","last_fullsized":"2023-11-28T11:08:19.073497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074252+0000","last_change":"2023-11-28T11:08:17.366207+0000","last_active":"2023-11-28T11:08:19.074252+0000","last_peered":"2023-11-28T11:08:19.074252+0000","last_clean":"2023-11-28T11:08:19.074252+0000","last_became_active":"2023-11-28T11:08:17.365918+0000","last_became_peered":"2023-11-28T11:08:17.365918+0000","last_unstale":"2023-11-28T11:08:19.074252+0000","last_undegraded":"2023-11-28T11:08:19.074252+0000","last_fullsized":"2023-11-28T11:08:19.074252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673623+0000","last_change":"2023-11-28T11:08:17.376435+0000","last_active":"2023-11-28T11:08:18.673623+0000","last_peered":"2023-11-28T11:08:18.673623+0000","last_clean":"2023-11-28T11:08:18.673623+0000","last_became_active":"2023-11-28T11:08:17.373543+0000","last_became_peered":"2023-11-28T11:08:17.373543+0000","last_unstale":"2023-11-28T11:08:18.673623+0000","last_undegraded":"2023-11-28T11:08:18.673623+0000","last_fullsized":"2023-11-28T11:08:18.673623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074030+0000","last_change":"2023-11-28T11:08:17.367284+0000","last_active":"2023-11-28T11:08:19.074030+0000","last_peered":"2023-11-28T11:08:19.074030+0000","last_clean":"2023-11-28T11:08:19.074030+0000","last_became_active":"2023-11-28T11:08:17.366992+0000","last_became_peered":"2023-11-28T11:08:17.366992+0000","last_unstale":"2023-11-28T11:08:19.074030+0000","last_undegraded":"2023-11-28T11:08:19.074030+0000","last_fullsized":"2023-11-28T11:08:19.074030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369258+0000","last_change":"2023-11-28T11:08:17.376061+0000","last_active":"2023-11-28T11:08:18.369258+0000","last_peered":"2023-11-28T11:08:18.369258+0000","last_clean":"2023-11-28T11:08:18.369258+0000","last_became_active":"2023-11-28T11:08:17.375789+0000","last_became_peered":"2023-11-28T11:08:17.375789+0000","last_unstale":"2023-11-28T11:08:18.369258+0000","last_undegraded":"2023-11-28T11:08:18.369258+0000","last_fullsized":"2023-11-28T11:08:18.369258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:10.040305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072686+0000","last_change":"2023-11-28T11:08:17.372872+0000","last_active":"2023-11-28T11:08:19.072686+0000","last_peered":"2023-11-28T11:08:19.072686+0000","last_clean":"2023-11-28T11:08:19.072686+0000","last_became_active":"2023-11-28T11:08:17.372624+0000","last_became_peered":"2023-11-28T11:08:17.372624+0000","last_unstale":"2023-11-28T11:08:19.072686+0000","last_undegraded":"2023-11-28T11:08:19.072686+0000","last_fullsized":"2023-11-28T11:08:19.072686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:34.711368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072664+0000","last_change":"2023-11-28T11:08:17.374949+0000","last_active":"2023-11-28T11:08:19.072664+0000","last_peered":"2023-11-28T11:08:19.072664+0000","last_clean":"2023-11-28T11:08:19.072664+0000","last_became_active":"2023-11-28T11:08:17.374690+0000","last_became_peered":"2023-11-28T11:08:17.374690+0000","last_unstale":"2023-11-28T11:08:19.072664+0000","last_undegraded":"2023-11-28T11:08:19.072664+0000","last_fullsized":"2023-11-28T11:08:19.072664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:52.301973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073186+0000","last_change":"2023-11-28T11:08:17.361341+0000","last_active":"2023-11-28T11:08:19.073186+0000","last_peered":"2023-11-28T11:08:19.073186+0000","last_clean":"2023-11-28T11:08:19.073186+0000","last_became_active":"2023-11-28T11:08:17.361053+0000","last_became_peered":"2023-11-28T11:08:17.361053+0000","last_unstale":"2023-11-28T11:08:19.073186+0000","last_undegraded":"2023-11-28T11:08:19.073186+0000","last_fullsized":"2023-11-28T11:08:19.073186+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073689+0000","last_change":"2023-11-28T11:08:17.377341+0000","last_active":"2023-11-28T11:08:19.073689+0000","last_peered":"2023-11-28T11:08:19.073689+0000","last_clean":"2023-11-28T11:08:19.073689+0000","last_became_active":"2023-11-28T11:08:17.376840+0000","last_became_peered":"2023-11-28T11:08:17.376840+0000","last_unstale":"2023-11-28T11:08:19.073689+0000","last_undegraded":"2023-11-28T11:08:19.073689+0000","last_fullsized":"2023-11-28T11:08:19.073689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368722+0000","last_change":"2023-11-28T11:08:17.370526+0000","last_active":"2023-11-28T11:08:18.368722+0000","last_peered":"2023-11-28T11:08:18.368722+0000","last_clean":"2023-11-28T11:08:18.368722+0000","last_became_active":"2023-11-28T11:08:17.367599+0000","last_became_peered":"2023-11-28T11:08:17.367599+0000","last_unstale":"2023-11-28T11:08:18.368722+0000","last_undegraded":"2023-11-28T11:08:18.368722+0000","last_fullsized":"2023-11-28T11:08:18.368722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:21.226857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673664+0000","last_change":"2023-11-28T11:08:17.377283+0000","last_active":"2023-11-28T11:08:18.673664+0000","last_peered":"2023-11-28T11:08:18.673664+0000","last_clean":"2023-11-28T11:08:18.673664+0000","last_became_active":"2023-11-28T11:08:17.373226+0000","last_became_peered":"2023-11-28T11:08:17.373226+0000","last_unstale":"2023-11-28T11:08:18.673664+0000","last_undegraded":"2023-11-28T11:08:18.673664+0000","last_fullsized":"2023-11-28T11:08:18.673664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369217+0000","last_change":"2023-11-28T11:08:17.373771+0000","last_active":"2023-11-28T11:08:18.369217+0000","last_peered":"2023-11-28T11:08:18.369217+0000","last_clean":"2023-11-28T11:08:18.369217+0000","last_became_active":"2023-11-28T11:08:17.373510+0000","last_became_peered":"2023-11-28T11:08:17.373510+0000","last_unstale":"2023-11-28T11:08:18.369217+0000","last_undegraded":"2023-11-28T11:08:18.369217+0000","last_fullsized":"2023-11-28T11:08:18.369217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:14.696335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072793+0000","last_change":"2023-11-28T11:08:17.381345+0000","last_active":"2023-11-28T11:08:19.072793+0000","last_peered":"2023-11-28T11:08:19.072793+0000","last_clean":"2023-11-28T11:08:19.072793+0000","last_became_active":"2023-11-28T11:08:17.381044+0000","last_became_peered":"2023-11-28T11:08:17.381044+0000","last_unstale":"2023-11-28T11:08:19.072793+0000","last_undegraded":"2023-11-28T11:08:19.072793+0000","last_fullsized":"2023-11-28T11:08:19.072793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:09.877040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369092+0000","last_change":"2023-11-28T11:08:17.387391+0000","last_active":"2023-11-28T11:08:18.369092+0000","last_peered":"2023-11-28T11:08:18.369092+0000","last_clean":"2023-11-28T11:08:18.369092+0000","last_became_active":"2023-11-28T11:08:17.387127+0000","last_became_peered":"2023-11-28T11:08:17.387127+0000","last_unstale":"2023-11-28T11:08:18.369092+0000","last_undegraded":"2023-11-28T11:08:18.369092+0000","last_fullsized":"2023-11-28T11:08:18.369092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074429+0000","last_change":"2023-11-28T11:08:17.380537+0000","last_active":"2023-11-28T11:08:19.074429+0000","last_peered":"2023-11-28T11:08:19.074429+0000","last_clean":"2023-11-28T11:08:19.074429+0000","last_became_active":"2023-11-28T11:08:17.380267+0000","last_became_peered":"2023-11-28T11:08:17.380267+0000","last_unstale":"2023-11-28T11:08:19.074429+0000","last_undegraded":"2023-11-28T11:08:19.074429+0000","last_fullsized":"2023-11-28T11:08:19.074429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673493+0000","last_change":"2023-11-28T11:08:17.371837+0000","last_active":"2023-11-28T11:08:18.673493+0000","last_peered":"2023-11-28T11:08:18.673493+0000","last_clean":"2023-11-28T11:08:18.673493+0000","last_became_active":"2023-11-28T11:08:17.371543+0000","last_became_peered":"2023-11-28T11:08:17.371543+0000","last_unstale":"2023-11-28T11:08:18.673493+0000","last_undegraded":"2023-11-28T11:08:18.673493+0000","last_fullsized":"2023-11-28T11:08:18.673493+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365642+0000","last_change":"2023-11-28T11:08:17.370684+0000","last_active":"2023-11-28T11:08:18.365642+0000","last_peered":"2023-11-28T11:08:18.365642+0000","last_clean":"2023-11-28T11:08:18.365642+0000","last_became_active":"2023-11-28T11:08:17.367787+0000","last_became_peered":"2023-11-28T11:08:17.367787+0000","last_unstale":"2023-11-28T11:08:18.365642+0000","last_undegraded":"2023-11-28T11:08:18.365642+0000","last_fullsized":"2023-11-28T11:08:18.365642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:59.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073418+0000","last_change":"2023-11-28T11:08:17.373809+0000","last_active":"2023-11-28T11:08:19.073418+0000","last_peered":"2023-11-28T11:08:19.073418+0000","last_clean":"2023-11-28T11:08:19.073418+0000","last_became_active":"2023-11-28T11:08:17.373529+0000","last_became_peered":"2023-11-28T11:08:17.373529+0000","last_unstale":"2023-11-28T11:08:19.073418+0000","last_undegraded":"2023-11-28T11:08:19.073418+0000","last_fullsized":"2023-11-28T11:08:19.073418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368409+0000","last_change":"2023-11-28T11:08:17.382413+0000","last_active":"2023-11-28T11:08:18.368409+0000","last_peered":"2023-11-28T11:08:18.368409+0000","last_clean":"2023-11-28T11:08:18.368409+0000","last_became_active":"2023-11-28T11:08:17.382132+0000","last_became_peered":"2023-11-28T11:08:17.382132+0000","last_unstale":"2023-11-28T11:08:18.368409+0000","last_undegraded":"2023-11-28T11:08:18.368409+0000","last_fullsized":"2023-11-28T11:08:18.368409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368518+0000","last_change":"2023-11-28T11:08:17.368106+0000","last_active":"2023-11-28T11:08:18.368518+0000","last_peered":"2023-11-28T11:08:18.368518+0000","last_clean":"2023-11-28T11:08:18.368518+0000","last_became_active":"2023-11-28T11:08:17.367562+0000","last_became_peered":"2023-11-28T11:08:17.367562+0000","last_unstale":"2023-11-28T11:08:18.368518+0000","last_undegraded":"2023-11-28T11:08:18.368518+0000","last_fullsized":"2023-11-28T11:08:18.368518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073955+0000","last_change":"2023-11-28T11:08:17.375521+0000","last_active":"2023-11-28T11:08:19.073955+0000","last_peered":"2023-11-28T11:08:19.073955+0000","last_clean":"2023-11-28T11:08:19.073955+0000","last_became_active":"2023-11-28T11:08:17.375256+0000","last_became_peered":"2023-11-28T11:08:17.375256+0000","last_unstale":"2023-11-28T11:08:19.073955+0000","last_undegraded":"2023-11-28T11:08:19.073955+0000","last_fullsized":"2023-11-28T11:08:19.073955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.382174+0000","last_change":"2023-11-28T11:08:16.975491+0000","last_active":"2023-11-28T11:08:17.382174+0000","last_peered":"2023-11-28T11:08:17.382174+0000","last_clean":"2023-11-28T11:08:17.382174+0000","last_became_active":"2023-11-28T11:08:16.975076+0000","last_became_peered":"2023-11-28T11:08:16.975076+0000","last_unstale":"2023-11-28T11:08:17.382174+0000","last_undegraded":"2023-11-28T11:08:17.382174+0000","last_fullsized":"2023-11-28T11:08:17.382174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365845+0000","last_change":"2023-11-28T11:08:17.371715+0000","last_active":"2023-11-28T11:08:18.365845+0000","last_peered":"2023-11-28T11:08:18.365845+0000","last_clean":"2023-11-28T11:08:18.365845+0000","last_became_active":"2023-11-28T11:08:17.368939+0000","last_became_peered":"2023-11-28T11:08:17.368939+0000","last_unstale":"2023-11-28T11:08:18.365845+0000","last_undegraded":"2023-11-28T11:08:18.365845+0000","last_fullsized":"2023-11-28T11:08:18.365845+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362665+0000","last_change":"2023-11-28T11:08:16.975573+0000","last_active":"2023-11-28T11:08:17.362665+0000","last_peered":"2023-11-28T11:08:17.362665+0000","last_clean":"2023-11-28T11:08:17.362665+0000","last_became_active":"2023-11-28T11:08:16.975216+0000","last_became_peered":"2023-11-28T11:08:16.975216+0000","last_unstale":"2023-11-28T11:08:17.362665+0000","last_undegraded":"2023-11-28T11:08:17.362665+0000","last_fullsized":"2023-11-28T11:08:17.362665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673721+0000","last_change":"2023-11-28T11:08:17.378243+0000","last_active":"2023-11-28T11:08:18.673721+0000","last_peered":"2023-11-28T11:08:18.673721+0000","last_clean":"2023-11-28T11:08:18.673721+0000","last_became_active":"2023-11-28T11:08:17.377866+0000","last_became_peered":"2023-11-28T11:08:17.377866+0000","last_unstale":"2023-11-28T11:08:18.673721+0000","last_undegraded":"2023-11-28T11:08:18.673721+0000","last_fullsized":"2023-11-28T11:08:18.673721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.356027+0000","last_change":"2023-11-28T11:08:16.972779+0000","last_active":"2023-11-28T11:08:17.356027+0000","last_peered":"2023-11-28T11:08:17.356027+0000","last_clean":"2023-11-28T11:08:17.356027+0000","last_became_active":"2023-11-28T11:08:16.972510+0000","last_became_peered":"2023-11-28T11:08:16.972510+0000","last_unstale":"2023-11-28T11:08:17.356027+0000","last_undegraded":"2023-11-28T11:08:17.356027+0000","last_fullsized":"2023-11-28T11:08:17.356027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074026+0000","last_change":"2023-11-28T11:08:17.380638+0000","last_active":"2023-11-28T11:08:19.074026+0000","last_peered":"2023-11-28T11:08:19.074026+0000","last_clean":"2023-11-28T11:08:19.074026+0000","last_became_active":"2023-11-28T11:08:17.380333+0000","last_became_peered":"2023-11-28T11:08:17.380333+0000","last_unstale":"2023-11-28T11:08:19.074026+0000","last_undegraded":"2023-11-28T11:08:19.074026+0000","last_fullsized":"2023-11-28T11:08:19.074026+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:34.711368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.361717+0000","last_change":"2023-11-28T11:08:16.976366+0000","last_active":"2023-11-28T11:08:17.361717+0000","last_peered":"2023-11-28T11:08:17.361717+0000","last_clean":"2023-11-28T11:08:17.361717+0000","last_became_active":"2023-11-28T11:08:16.976072+0000","last_became_peered":"2023-11-28T11:08:16.976072+0000","last_unstale":"2023-11-28T11:08:17.361717+0000","last_undegraded":"2023-11-28T11:08:17.361717+0000","last_fullsized":"2023-11-28T11:08:17.361717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365642+0000","last_change":"2023-11-28T11:08:17.364878+0000","last_active":"2023-11-28T11:08:18.365642+0000","last_peered":"2023-11-28T11:08:18.365642+0000","last_clean":"2023-11-28T11:08:18.365642+0000","last_became_active":"2023-11-28T11:08:17.364628+0000","last_became_peered":"2023-11-28T11:08:17.364628+0000","last_unstale":"2023-11-28T11:08:18.365642+0000","last_undegraded":"2023-11-28T11:08:18.365642+0000","last_fullsized":"2023-11-28T11:08:18.365642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:12.679494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.358873+0000","last_change":"2023-11-28T11:08:16.975388+0000","last_active":"2023-11-28T11:08:17.358873+0000","last_peered":"2023-11-28T11:08:17.358873+0000","last_clean":"2023-11-28T11:08:17.358873+0000","last_became_active":"2023-11-28T11:08:16.974977+0000","last_became_peered":"2023-11-28T11:08:16.974977+0000","last_unstale":"2023-11-28T11:08:17.358873+0000","last_undegraded":"2023-11-28T11:08:17.358873+0000","last_fullsized":"2023-11-28T11:08:17.358873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.875156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365734+0000","last_change":"2023-11-28T11:08:17.376435+0000","last_active":"2023-11-28T11:08:18.365734+0000","last_peered":"2023-11-28T11:08:18.365734+0000","last_clean":"2023-11-28T11:08:18.365734+0000","last_became_active":"2023-11-28T11:08:17.376170+0000","last_became_peered":"2023-11-28T11:08:17.376170+0000","last_unstale":"2023-11-28T11:08:18.365734+0000","last_undegraded":"2023-11-28T11:08:18.365734+0000","last_fullsized":"2023-11-28T11:08:18.365734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:24:59.277184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.381284+0000","last_change":"2023-11-28T11:08:16.975413+0000","last_active":"2023-11-28T11:08:17.381284+0000","last_peered":"2023-11-28T11:08:17.381284+0000","last_clean":"2023-11-28T11:08:17.381284+0000","last_became_active":"2023-11-28T11:08:16.975058+0000","last_became_peered":"2023-11-28T11:08:16.975058+0000","last_unstale":"2023-11-28T11:08:17.381284+0000","last_undegraded":"2023-11-28T11:08:17.381284+0000","last_fullsized":"2023-11-28T11:08:17.381284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368111+0000","last_change":"2023-11-28T11:08:17.381210+0000","last_active":"2023-11-28T11:08:18.368111+0000","last_peered":"2023-11-28T11:08:18.368111+0000","last_clean":"2023-11-28T11:08:18.368111+0000","last_became_active":"2023-11-28T11:08:17.380941+0000","last_became_peered":"2023-11-28T11:08:17.380941+0000","last_unstale":"2023-11-28T11:08:18.368111+0000","last_undegraded":"2023-11-28T11:08:18.368111+0000","last_fullsized":"2023-11-28T11:08:18.368111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:35.096193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.387746+0000","last_change":"2023-11-28T11:08:16.973937+0000","last_active":"2023-11-28T11:08:17.387746+0000","last_peered":"2023-11-28T11:08:17.387746+0000","last_clean":"2023-11-28T11:08:17.387746+0000","last_became_active":"2023-11-28T11:08:16.973594+0000","last_became_peered":"2023-11-28T11:08:16.973594+0000","last_unstale":"2023-11-28T11:08:17.387746+0000","last_undegraded":"2023-11-28T11:08:17.387746+0000","last_fullsized":"2023-11-28T11:08:17.387746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:13.313856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369025+0000","last_change":"2023-11-28T11:08:17.380894+0000","last_active":"2023-11-28T11:08:18.369025+0000","last_peered":"2023-11-28T11:08:18.369025+0000","last_clean":"2023-11-28T11:08:18.369025+0000","last_became_active":"2023-11-28T11:08:17.380600+0000","last_became_peered":"2023-11-28T11:08:17.380600+0000","last_unstale":"2023-11-28T11:08:18.369025+0000","last_undegraded":"2023-11-28T11:08:18.369025+0000","last_fullsized":"2023-11-28T11:08:18.369025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377908+0000","last_change":"2023-11-28T11:08:16.977393+0000","last_active":"2023-11-28T11:08:17.377908+0000","last_peered":"2023-11-28T11:08:17.377908+0000","last_clean":"2023-11-28T11:08:17.377908+0000","last_became_active":"2023-11-28T11:08:16.976585+0000","last_became_peered":"2023-11-28T11:08:16.976585+0000","last_unstale":"2023-11-28T11:08:17.377908+0000","last_undegraded":"2023-11-28T11:08:17.377908+0000","last_fullsized":"2023-11-28T11:08:17.377908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:36.981686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365857+0000","last_change":"2023-11-28T11:08:17.376235+0000","last_active":"2023-11-28T11:08:18.365857+0000","last_peered":"2023-11-28T11:08:18.365857+0000","last_clean":"2023-11-28T11:08:18.365857+0000","last_became_active":"2023-11-28T11:08:17.375675+0000","last_became_peered":"2023-11-28T11:08:17.375675+0000","last_unstale":"2023-11-28T11:08:18.365857+0000","last_undegraded":"2023-11-28T11:08:18.365857+0000","last_fullsized":"2023-11-28T11:08:18.365857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.359416+0000","last_change":"2023-11-28T11:08:16.974456+0000","last_active":"2023-11-28T11:08:17.359416+0000","last_peered":"2023-11-28T11:08:17.359416+0000","last_clean":"2023-11-28T11:08:17.359416+0000","last_became_active":"2023-11-28T11:08:16.974158+0000","last_became_peered":"2023-11-28T11:08:16.974158+0000","last_unstale":"2023-11-28T11:08:17.359416+0000","last_undegraded":"2023-11-28T11:08:17.359416+0000","last_fullsized":"2023-11-28T11:08:17.359416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074033+0000","last_change":"2023-11-28T11:08:17.375410+0000","last_active":"2023-11-28T11:08:19.074033+0000","last_peered":"2023-11-28T11:08:19.074033+0000","last_clean":"2023-11-28T11:08:19.074033+0000","last_became_active":"2023-11-28T11:08:17.374828+0000","last_became_peered":"2023-11-28T11:08:17.374828+0000","last_unstale":"2023-11-28T11:08:19.074033+0000","last_undegraded":"2023-11-28T11:08:19.074033+0000","last_fullsized":"2023-11-28T11:08:19.074033+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365862+0000","last_change":"2023-11-28T11:08:16.978277+0000","last_active":"2023-11-28T11:08:17.365862+0000","last_peered":"2023-11-28T11:08:17.365862+0000","last_clean":"2023-11-28T11:08:17.365862+0000","last_became_active":"2023-11-28T11:08:16.977994+0000","last_became_peered":"2023-11-28T11:08:16.977994+0000","last_unstale":"2023-11-28T11:08:17.365862+0000","last_undegraded":"2023-11-28T11:08:17.365862+0000","last_fullsized":"2023-11-28T11:08:17.365862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:55.975785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673702+0000","last_change":"2023-11-28T11:08:17.369372+0000","last_active":"2023-11-28T11:08:18.673702+0000","last_peered":"2023-11-28T11:08:18.673702+0000","last_clean":"2023-11-28T11:08:18.673702+0000","last_became_active":"2023-11-28T11:08:17.369115+0000","last_became_peered":"2023-11-28T11:08:17.369115+0000","last_unstale":"2023-11-28T11:08:18.673702+0000","last_undegraded":"2023-11-28T11:08:18.673702+0000","last_fullsized":"2023-11-28T11:08:18.673702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.386102+0000","last_change":"2023-11-28T11:08:16.974290+0000","last_active":"2023-11-28T11:08:17.386102+0000","last_peered":"2023-11-28T11:08:17.386102+0000","last_clean":"2023-11-28T11:08:17.386102+0000","last_became_active":"2023-11-28T11:08:16.973952+0000","last_became_peered":"2023-11-28T11:08:16.973952+0000","last_unstale":"2023-11-28T11:08:17.386102+0000","last_undegraded":"2023-11-28T11:08:17.386102+0000","last_fullsized":"2023-11-28T11:08:17.386102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365978+0000","last_change":"2023-11-28T11:08:17.376526+0000","last_active":"2023-11-28T11:08:18.365978+0000","last_peered":"2023-11-28T11:08:18.365978+0000","last_clean":"2023-11-28T11:08:18.365978+0000","last_became_active":"2023-11-28T11:08:17.376285+0000","last_became_peered":"2023-11-28T11:08:17.376285+0000","last_unstale":"2023-11-28T11:08:18.365978+0000","last_undegraded":"2023-11-28T11:08:18.365978+0000","last_fullsized":"2023-11-28T11:08:18.365978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377922+0000","last_change":"2023-11-28T11:08:16.976879+0000","last_active":"2023-11-28T11:08:17.377922+0000","last_peered":"2023-11-28T11:08:17.377922+0000","last_clean":"2023-11-28T11:08:17.377922+0000","last_became_active":"2023-11-28T11:08:16.976530+0000","last_became_peered":"2023-11-28T11:08:16.976530+0000","last_unstale":"2023-11-28T11:08:17.377922+0000","last_undegraded":"2023-11-28T11:08:17.377922+0000","last_fullsized":"2023-11-28T11:08:17.377922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:59.917012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073041+0000","last_change":"2023-11-28T11:08:17.375192+0000","last_active":"2023-11-28T11:08:19.073041+0000","last_peered":"2023-11-28T11:08:19.073041+0000","last_clean":"2023-11-28T11:08:19.073041+0000","last_became_active":"2023-11-28T11:08:17.374903+0000","last_became_peered":"2023-11-28T11:08:17.374903+0000","last_unstale":"2023-11-28T11:08:19.073041+0000","last_undegraded":"2023-11-28T11:08:19.073041+0000","last_fullsized":"2023-11-28T11:08:19.073041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377956+0000","last_change":"2023-11-28T11:08:16.980934+0000","last_active":"2023-11-28T11:08:17.377956+0000","last_peered":"2023-11-28T11:08:17.377956+0000","last_clean":"2023-11-28T11:08:17.377956+0000","last_became_active":"2023-11-28T11:08:16.980585+0000","last_became_peered":"2023-11-28T11:08:16.980585+0000","last_unstale":"2023-11-28T11:08:17.377956+0000","last_undegraded":"2023-11-28T11:08:17.377956+0000","last_fullsized":"2023-11-28T11:08:17.377956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:33.709483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368728+0000","last_change":"2023-11-28T11:08:17.374922+0000","last_active":"2023-11-28T11:08:18.368728+0000","last_peered":"2023-11-28T11:08:18.368728+0000","last_clean":"2023-11-28T11:08:18.368728+0000","last_became_active":"2023-11-28T11:08:17.374631+0000","last_became_peered":"2023-11-28T11:08:17.374631+0000","last_unstale":"2023-11-28T11:08:18.368728+0000","last_undegraded":"2023-11-28T11:08:18.368728+0000","last_fullsized":"2023-11-28T11:08:18.368728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:28:26.671538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.375561+0000","last_change":"2023-11-28T11:08:16.978233+0000","last_active":"2023-11-28T11:08:17.375561+0000","last_peered":"2023-11-28T11:08:17.375561+0000","last_clean":"2023-11-28T11:08:17.375561+0000","last_became_active":"2023-11-28T11:08:16.977953+0000","last_became_peered":"2023-11-28T11:08:16.977953+0000","last_unstale":"2023-11-28T11:08:17.375561+0000","last_undegraded":"2023-11-28T11:08:17.375561+0000","last_fullsized":"2023-11-28T11:08:17.375561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073837+0000","last_change":"2023-11-28T11:08:17.381282+0000","last_active":"2023-11-28T11:08:19.073837+0000","last_peered":"2023-11-28T11:08:19.073837+0000","last_clean":"2023-11-28T11:08:19.073837+0000","last_became_active":"2023-11-28T11:08:17.381013+0000","last_became_peered":"2023-11-28T11:08:17.381013+0000","last_unstale":"2023-11-28T11:08:19.073837+0000","last_undegraded":"2023-11-28T11:08:19.073837+0000","last_fullsized":"2023-11-28T11:08:19.073837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:09.760690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357896+0000","last_change":"2023-11-28T11:08:16.976378+0000","last_active":"2023-11-28T11:08:17.357896+0000","last_peered":"2023-11-28T11:08:17.357896+0000","last_clean":"2023-11-28T11:08:17.357896+0000","last_became_active":"2023-11-28T11:08:16.974426+0000","last_became_peered":"2023-11-28T11:08:16.974426+0000","last_unstale":"2023-11-28T11:08:17.357896+0000","last_undegraded":"2023-11-28T11:08:17.357896+0000","last_fullsized":"2023-11-28T11:08:17.357896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673645+0000","last_change":"2023-11-28T11:08:17.366381+0000","last_active":"2023-11-28T11:08:18.673645+0000","last_peered":"2023-11-28T11:08:18.673645+0000","last_clean":"2023-11-28T11:08:18.673645+0000","last_became_active":"2023-11-28T11:08:17.366129+0000","last_became_peered":"2023-11-28T11:08:17.366129+0000","last_unstale":"2023-11-28T11:08:18.673645+0000","last_undegraded":"2023-11-28T11:08:18.673645+0000","last_fullsized":"2023-11-28T11:08:18.673645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.380305+0000","last_change":"2023-11-28T11:08:16.973851+0000","last_active":"2023-11-28T11:08:17.380305+0000","last_peered":"2023-11-28T11:08:17.380305+0000","last_clean":"2023-11-28T11:08:17.380305+0000","last_became_active":"2023-11-28T11:08:16.973588+0000","last_became_peered":"2023-11-28T11:08:16.973588+0000","last_unstale":"2023-11-28T11:08:17.380305+0000","last_undegraded":"2023-11-28T11:08:17.380305+0000","last_fullsized":"2023-11-28T11:08:17.380305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:40.930972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369164+0000","last_change":"2023-11-28T11:08:17.372448+0000","last_active":"2023-11-28T11:08:18.369164+0000","last_peered":"2023-11-28T11:08:18.369164+0000","last_clean":"2023-11-28T11:08:18.369164+0000","last_became_active":"2023-11-28T11:08:17.372132+0000","last_became_peered":"2023-11-28T11:08:17.372132+0000","last_unstale":"2023-11-28T11:08:18.369164+0000","last_undegraded":"2023-11-28T11:08:18.369164+0000","last_fullsized":"2023-11-28T11:08:18.369164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:12.679494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.371186+0000","last_change":"2023-11-28T11:08:16.975018+0000","last_active":"2023-11-28T11:08:17.371186+0000","last_peered":"2023-11-28T11:08:17.371186+0000","last_clean":"2023-11-28T11:08:17.371186+0000","last_became_active":"2023-11-28T11:08:16.974709+0000","last_became_peered":"2023-11-28T11:08:16.974709+0000","last_unstale":"2023-11-28T11:08:17.371186+0000","last_undegraded":"2023-11-28T11:08:17.371186+0000","last_fullsized":"2023-11-28T11:08:17.371186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369338+0000","last_change":"2023-11-28T11:08:17.372568+0000","last_active":"2023-11-28T11:08:18.369338+0000","last_peered":"2023-11-28T11:08:18.369338+0000","last_clean":"2023-11-28T11:08:18.369338+0000","last_became_active":"2023-11-28T11:08:17.372279+0000","last_became_peered":"2023-11-28T11:08:17.372279+0000","last_unstale":"2023-11-28T11:08:18.369338+0000","last_undegraded":"2023-11-28T11:08:18.369338+0000","last_fullsized":"2023-11-28T11:08:18.369338+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:02.566272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364207+0000","last_change":"2023-11-28T11:08:16.976973+0000","last_active":"2023-11-28T11:08:17.364207+0000","last_peered":"2023-11-28T11:08:17.364207+0000","last_clean":"2023-11-28T11:08:17.364207+0000","last_became_active":"2023-11-28T11:08:16.976661+0000","last_became_peered":"2023-11-28T11:08:16.976661+0000","last_unstale":"2023-11-28T11:08:17.364207+0000","last_undegraded":"2023-11-28T11:08:17.364207+0000","last_fullsized":"2023-11-28T11:08:17.364207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:43.575239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.369260+0000","last_change":"2023-11-28T11:08:17.381198+0000","last_active":"2023-11-28T11:08:18.369260+0000","last_peered":"2023-11-28T11:08:18.369260+0000","last_clean":"2023-11-28T11:08:18.369260+0000","last_became_active":"2023-11-28T11:08:17.380937+0000","last_became_peered":"2023-11-28T11:08:17.380937+0000","last_unstale":"2023-11-28T11:08:18.369260+0000","last_undegraded":"2023-11-28T11:08:18.369260+0000","last_fullsized":"2023-11-28T11:08:18.369260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:30.383268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.355350+0000","last_change":"2023-11-28T11:08:16.966279+0000","last_active":"2023-11-28T11:08:17.355350+0000","last_peered":"2023-11-28T11:08:17.355350+0000","last_clean":"2023-11-28T11:08:17.355350+0000","last_became_active":"2023-11-28T11:08:16.965878+0000","last_became_peered":"2023-11-28T11:08:16.965878+0000","last_unstale":"2023-11-28T11:08:17.355350+0000","last_undegraded":"2023-11-28T11:08:17.355350+0000","last_fullsized":"2023-11-28T11:08:17.355350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368520+0000","last_change":"2023-11-28T11:08:17.373270+0000","last_active":"2023-11-28T11:08:18.368520+0000","last_peered":"2023-11-28T11:08:18.368520+0000","last_clean":"2023-11-28T11:08:18.368520+0000","last_became_active":"2023-11-28T11:08:17.373017+0000","last_became_peered":"2023-11-28T11:08:17.373017+0000","last_unstale":"2023-11-28T11:08:18.368520+0000","last_undegraded":"2023-11-28T11:08:18.368520+0000","last_fullsized":"2023-11-28T11:08:18.368520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:05:22.973814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365774+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:08:17.365774+0000","last_peered":"2023-11-28T11:08:17.365774+0000","last_clean":"2023-11-28T11:08:17.365774+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:08:17.365774+0000","last_undegraded":"2023-11-28T11:08:17.365774+0000","last_fullsized":"2023-11-28T11:08:17.365774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.358583+0000","last_change":"2023-11-28T11:08:16.975874+0000","last_active":"2023-11-28T11:08:17.358583+0000","last_peered":"2023-11-28T11:08:17.358583+0000","last_clean":"2023-11-28T11:08:17.358583+0000","last_became_active":"2023-11-28T11:08:16.975225+0000","last_became_peered":"2023-11-28T11:08:16.975225+0000","last_unstale":"2023-11-28T11:08:17.358583+0000","last_undegraded":"2023-11-28T11:08:17.358583+0000","last_fullsized":"2023-11-28T11:08:17.358583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368149+0000","last_change":"2023-11-28T11:08:17.371631+0000","last_active":"2023-11-28T11:08:18.368149+0000","last_peered":"2023-11-28T11:08:18.368149+0000","last_clean":"2023-11-28T11:08:18.368149+0000","last_became_active":"2023-11-28T11:08:17.371342+0000","last_became_peered":"2023-11-28T11:08:17.371342+0000","last_unstale":"2023-11-28T11:08:18.368149+0000","last_undegraded":"2023-11-28T11:08:18.368149+0000","last_fullsized":"2023-11-28T11:08:18.368149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:06.664416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357509+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:08:17.357509+0000","last_peered":"2023-11-28T11:08:17.357509+0000","last_clean":"2023-11-28T11:08:17.357509+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:08:17.357509+0000","last_undegraded":"2023-11-28T11:08:17.357509+0000","last_fullsized":"2023-11-28T11:08:17.357509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362711+0000","last_change":"2023-11-28T11:08:16.974904+0000","last_active":"2023-11-28T11:08:17.362711+0000","last_peered":"2023-11-28T11:08:17.362711+0000","last_clean":"2023-11-28T11:08:17.362711+0000","last_became_active":"2023-11-28T11:08:16.974634+0000","last_became_peered":"2023-11-28T11:08:16.974634+0000","last_unstale":"2023-11-28T11:08:17.362711+0000","last_undegraded":"2023-11-28T11:08:17.362711+0000","last_fullsized":"2023-11-28T11:08:17.362711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368410+0000","last_change":"2023-11-28T11:08:17.371181+0000","last_active":"2023-11-28T11:08:18.368410+0000","last_peered":"2023-11-28T11:08:18.368410+0000","last_clean":"2023-11-28T11:08:18.368410+0000","last_became_active":"2023-11-28T11:08:17.370931+0000","last_became_peered":"2023-11-28T11:08:17.370931+0000","last_unstale":"2023-11-28T11:08:18.368410+0000","last_undegraded":"2023-11-28T11:08:18.368410+0000","last_fullsized":"2023-11-28T11:08:18.368410+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.359510+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:08:17.359510+0000","last_peered":"2023-11-28T11:08:17.359510+0000","last_clean":"2023-11-28T11:08:17.359510+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:08:17.359510+0000","last_undegraded":"2023-11-28T11:08:17.359510+0000","last_fullsized":"2023-11-28T11:08:17.359510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.362780+0000","last_change":"2023-11-28T11:08:16.972359+0000","last_active":"2023-11-28T11:08:17.362780+0000","last_peered":"2023-11-28T11:08:17.362780+0000","last_clean":"2023-11-28T11:08:17.362780+0000","last_became_active":"2023-11-28T11:08:16.972074+0000","last_became_peered":"2023-11-28T11:08:16.972074+0000","last_unstale":"2023-11-28T11:08:17.362780+0000","last_undegraded":"2023-11-28T11:08:17.362780+0000","last_fullsized":"2023-11-28T11:08:17.362780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368494+0000","last_change":"2023-11-28T11:08:17.369522+0000","last_active":"2023-11-28T11:08:18.368494+0000","last_peered":"2023-11-28T11:08:18.368494+0000","last_clean":"2023-11-28T11:08:18.368494+0000","last_became_active":"2023-11-28T11:08:17.368628+0000","last_became_peered":"2023-11-28T11:08:17.368628+0000","last_unstale":"2023-11-28T11:08:18.368494+0000","last_undegraded":"2023-11-28T11:08:18.368494+0000","last_fullsized":"2023-11-28T11:08:18.368494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:56.558625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.357776+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:08:17.357776+0000","last_peered":"2023-11-28T11:08:17.357776+0000","last_clean":"2023-11-28T11:08:17.357776+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:08:17.357776+0000","last_undegraded":"2023-11-28T11:08:17.357776+0000","last_fullsized":"2023-11-28T11:08:17.357776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364014+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:08:17.364014+0000","last_peered":"2023-11-28T11:08:17.364014+0000","last_clean":"2023-11-28T11:08:17.364014+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:08:17.364014+0000","last_undegraded":"2023-11-28T11:08:17.364014+0000","last_fullsized":"2023-11-28T11:08:17.364014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.355796+0000","last_change":"2023-11-28T11:08:16.968819+0000","last_active":"2023-11-28T11:08:17.355796+0000","last_peered":"2023-11-28T11:08:17.355796+0000","last_clean":"2023-11-28T11:08:17.355796+0000","last_became_active":"2023-11-28T11:08:16.968536+0000","last_became_peered":"2023-11-28T11:08:16.968536+0000","last_unstale":"2023-11-28T11:08:17.355796+0000","last_undegraded":"2023-11-28T11:08:17.355796+0000","last_fullsized":"2023-11-28T11:08:17.355796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:33.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.073714+0000","last_change":"2023-11-28T11:08:17.381098+0000","last_active":"2023-11-28T11:08:19.073714+0000","last_peered":"2023-11-28T11:08:19.073714+0000","last_clean":"2023-11-28T11:08:19.073714+0000","last_became_active":"2023-11-28T11:08:17.380823+0000","last_became_peered":"2023-11-28T11:08:17.380823+0000","last_unstale":"2023-11-28T11:08:19.073714+0000","last_undegraded":"2023-11-28T11:08:19.073714+0000","last_fullsized":"2023-11-28T11:08:19.073714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:56.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.364256+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:08:17.364256+0000","last_peered":"2023-11-28T11:08:17.364256+0000","last_clean":"2023-11-28T11:08:17.364256+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:08:17.364256+0000","last_undegraded":"2023-11-28T11:08:17.364256+0000","last_fullsized":"2023-11-28T11:08:17.364256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.360038+0000","last_change":"2023-11-28T11:08:16.978195+0000","last_active":"2023-11-28T11:08:17.360038+0000","last_peered":"2023-11-28T11:08:17.360038+0000","last_clean":"2023-11-28T11:08:17.360038+0000","last_became_active":"2023-11-28T11:08:16.977906+0000","last_became_peered":"2023-11-28T11:08:16.977906+0000","last_unstale":"2023-11-28T11:08:17.360038+0000","last_undegraded":"2023-11-28T11:08:17.360038+0000","last_fullsized":"2023-11-28T11:08:17.360038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673777+0000","last_change":"2023-11-28T11:08:17.369849+0000","last_active":"2023-11-28T11:08:18.673777+0000","last_peered":"2023-11-28T11:08:18.673777+0000","last_clean":"2023-11-28T11:08:18.673777+0000","last_became_active":"2023-11-28T11:08:17.369614+0000","last_became_peered":"2023-11-28T11:08:17.369614+0000","last_unstale":"2023-11-28T11:08:18.673777+0000","last_undegraded":"2023-11-28T11:08:18.673777+0000","last_fullsized":"2023-11-28T11:08:18.673777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.368123+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:08:17.368123+0000","last_peered":"2023-11-28T11:08:17.368123+0000","last_clean":"2023-11-28T11:08:17.368123+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:08:17.368123+0000","last_undegraded":"2023-11-28T11:08:17.368123+0000","last_fullsized":"2023-11-28T11:08:17.368123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.365891+0000","last_change":"2023-11-28T11:08:16.975131+0000","last_active":"2023-11-28T11:08:17.365891+0000","last_peered":"2023-11-28T11:08:17.365891+0000","last_clean":"2023-11-28T11:08:17.365891+0000","last_became_active":"2023-11-28T11:08:16.974846+0000","last_became_peered":"2023-11-28T11:08:16.974846+0000","last_unstale":"2023-11-28T11:08:17.365891+0000","last_undegraded":"2023-11-28T11:08:17.365891+0000","last_fullsized":"2023-11-28T11:08:17.365891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:51.300088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673513+0000","last_change":"2023-11-28T11:08:17.366819+0000","last_active":"2023-11-28T11:08:18.673513+0000","last_peered":"2023-11-28T11:08:18.673513+0000","last_clean":"2023-11-28T11:08:18.673513+0000","last_became_active":"2023-11-28T11:08:17.366576+0000","last_became_peered":"2023-11-28T11:08:17.366576+0000","last_unstale":"2023-11-28T11:08:18.673513+0000","last_undegraded":"2023-11-28T11:08:18.673513+0000","last_fullsized":"2023-11-28T11:08:18.673513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:27.769937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.379003+0000","last_change":"2023-11-28T11:08:16.973768+0000","last_active":"2023-11-28T11:08:17.379003+0000","last_peered":"2023-11-28T11:08:17.379003+0000","last_clean":"2023-11-28T11:08:17.379003+0000","last_became_active":"2023-11-28T11:08:16.973505+0000","last_became_peered":"2023-11-28T11:08:16.973505+0000","last_unstale":"2023-11-28T11:08:17.379003+0000","last_undegraded":"2023-11-28T11:08:17.379003+0000","last_fullsized":"2023-11-28T11:08:17.379003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.367956+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:08:17.367956+0000","last_peered":"2023-11-28T11:08:17.367956+0000","last_clean":"2023-11-28T11:08:17.367956+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:08:17.367956+0000","last_undegraded":"2023-11-28T11:08:17.367956+0000","last_fullsized":"2023-11-28T11:08:17.367956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365501+0000","last_change":"2023-11-28T11:08:17.375741+0000","last_active":"2023-11-28T11:08:18.365501+0000","last_peered":"2023-11-28T11:08:18.365501+0000","last_clean":"2023-11-28T11:08:18.365501+0000","last_became_active":"2023-11-28T11:08:17.375349+0000","last_became_peered":"2023-11-28T11:08:17.375349+0000","last_unstale":"2023-11-28T11:08:18.365501+0000","last_undegraded":"2023-11-28T11:08:18.365501+0000","last_fullsized":"2023-11-28T11:08:18.365501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.372744+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:08:17.372744+0000","last_peered":"2023-11-28T11:08:17.372744+0000","last_clean":"2023-11-28T11:08:17.372744+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:08:17.372744+0000","last_undegraded":"2023-11-28T11:08:17.372744+0000","last_fullsized":"2023-11-28T11:08:17.372744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.673785+0000","last_change":"2023-11-28T11:08:16.974692+0000","last_active":"2023-11-28T11:08:18.673785+0000","last_peered":"2023-11-28T11:08:18.673785+0000","last_clean":"2023-11-28T11:08:18.673785+0000","last_became_active":"2023-11-28T11:08:16.974255+0000","last_became_peered":"2023-11-28T11:08:16.974255+0000","last_unstale":"2023-11-28T11:08:18.673785+0000","last_undegraded":"2023-11-28T11:08:18.673785+0000","last_fullsized":"2023-11-28T11:08:18.673785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:50.382379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368842+0000","last_change":"2023-11-28T11:08:17.373312+0000","last_active":"2023-11-28T11:08:18.368842+0000","last_peered":"2023-11-28T11:08:18.368842+0000","last_clean":"2023-11-28T11:08:18.368842+0000","last_became_active":"2023-11-28T11:08:17.373033+0000","last_became_peered":"2023-11-28T11:08:17.373033+0000","last_unstale":"2023-11-28T11:08:18.368842+0000","last_undegraded":"2023-11-28T11:08:18.368842+0000","last_fullsized":"2023-11-28T11:08:18.368842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:25.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.377530+0000","last_change":"2023-11-28T11:08:16.976409+0000","last_active":"2023-11-28T11:08:17.377530+0000","last_peered":"2023-11-28T11:08:17.377530+0000","last_clean":"2023-11-28T11:08:17.377530+0000","last_became_active":"2023-11-28T11:08:16.976127+0000","last_became_peered":"2023-11-28T11:08:16.976127+0000","last_unstale":"2023-11-28T11:08:17.377530+0000","last_undegraded":"2023-11-28T11:08:17.377530+0000","last_fullsized":"2023-11-28T11:08:17.377530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:11.421587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.072815+0000","last_change":"2023-11-28T11:08:17.374219+0000","last_active":"2023-11-28T11:08:19.072815+0000","last_peered":"2023-11-28T11:08:19.072815+0000","last_clean":"2023-11-28T11:08:19.072815+0000","last_became_active":"2023-11-28T11:08:17.374010+0000","last_became_peered":"2023-11-28T11:08:17.374010+0000","last_unstale":"2023-11-28T11:08:19.072815+0000","last_undegraded":"2023-11-28T11:08:19.072815+0000","last_fullsized":"2023-11-28T11:08:19.072815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:50.027648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.375928+0000","last_change":"2023-11-28T11:08:16.978678+0000","last_active":"2023-11-28T11:08:17.375928+0000","last_peered":"2023-11-28T11:08:17.375928+0000","last_clean":"2023-11-28T11:08:17.375928+0000","last_became_active":"2023-11-28T11:08:16.978317+0000","last_became_peered":"2023-11-28T11:08:16.978317+0000","last_unstale":"2023-11-28T11:08:17.375928+0000","last_undegraded":"2023-11-28T11:08:17.375928+0000","last_fullsized":"2023-11-28T11:08:17.375928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.758805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368851+0000","last_change":"2023-11-28T11:08:17.373647+0000","last_active":"2023-11-28T11:08:18.368851+0000","last_peered":"2023-11-28T11:08:18.368851+0000","last_clean":"2023-11-28T11:08:18.368851+0000","last_became_active":"2023-11-28T11:08:17.373288+0000","last_became_peered":"2023-11-28T11:08:17.373288+0000","last_unstale":"2023-11-28T11:08:18.368851+0000","last_undegraded":"2023-11-28T11:08:18.368851+0000","last_fullsized":"2023-11-28T11:08:18.368851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:59.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.376913+0000","last_change":"2023-11-28T11:08:16.976048+0000","last_active":"2023-11-28T11:08:17.376913+0000","last_peered":"2023-11-28T11:08:17.376913+0000","last_clean":"2023-11-28T11:08:17.376913+0000","last_became_active":"2023-11-28T11:08:16.975600+0000","last_became_peered":"2023-11-28T11:08:16.975600+0000","last_unstale":"2023-11-28T11:08:17.376913+0000","last_undegraded":"2023-11-28T11:08:17.376913+0000","last_fullsized":"2023-11-28T11:08:17.376913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368940+0000","last_change":"2023-11-28T11:08:17.372086+0000","last_active":"2023-11-28T11:08:18.368940+0000","last_peered":"2023-11-28T11:08:18.368940+0000","last_clean":"2023-11-28T11:08:18.368940+0000","last_became_active":"2023-11-28T11:08:17.371833+0000","last_became_peered":"2023-11-28T11:08:17.371833+0000","last_unstale":"2023-11-28T11:08:18.368940+0000","last_undegraded":"2023-11-28T11:08:18.368940+0000","last_fullsized":"2023-11-28T11:08:18.368940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:46.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.380725+0000","last_change":"2023-11-28T11:08:16.964126+0000","last_active":"2023-11-28T11:08:17.380725+0000","last_peered":"2023-11-28T11:08:17.380725+0000","last_clean":"2023-11-28T11:08:17.380725+0000","last_became_active":"2023-11-28T11:08:16.963838+0000","last_became_peered":"2023-11-28T11:08:16.963838+0000","last_unstale":"2023-11-28T11:08:17.380725+0000","last_undegraded":"2023-11-28T11:08:17.380725+0000","last_fullsized":"2023-11-28T11:08:17.380725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:49.025763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365611+0000","last_change":"2023-11-28T11:08:17.361919+0000","last_active":"2023-11-28T11:08:18.365611+0000","last_peered":"2023-11-28T11:08:18.365611+0000","last_clean":"2023-11-28T11:08:18.365611+0000","last_became_active":"2023-11-28T11:08:17.361654+0000","last_became_peered":"2023-11-28T11:08:17.361654+0000","last_unstale":"2023-11-28T11:08:18.365611+0000","last_undegraded":"2023-11-28T11:08:18.365611+0000","last_fullsized":"2023-11-28T11:08:18.365611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:46.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:08:17.372346+0000","last_change":"2023-11-28T11:08:16.974232+0000","last_active":"2023-11-28T11:08:17.372346+0000","last_peered":"2023-11-28T11:08:17.372346+0000","last_clean":"2023-11-28T11:08:17.372346+0000","last_became_active":"2023-11-28T11:08:16.972983+0000","last_became_peered":"2023-11-28T11:08:16.972983+0000","last_unstale":"2023-11-28T11:08:17.372346+0000","last_undegraded":"2023-11-28T11:08:17.372346+0000","last_fullsized":"2023-11-28T11:08:17.372346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:15.953025+0000","last_clean_scrub_stamp":"2023-11-28T11:08:15.953025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:09.422524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368650+0000","last_change":"2023-11-28T11:08:17.386923+0000","last_active":"2023-11-28T11:08:18.368650+0000","last_peered":"2023-11-28T11:08:18.368650+0000","last_clean":"2023-11-28T11:08:18.368650+0000","last_became_active":"2023-11-28T11:08:17.386690+0000","last_became_peered":"2023-11-28T11:08:17.386690+0000","last_unstale":"2023-11-28T11:08:18.368650+0000","last_undegraded":"2023-11-28T11:08:18.368650+0000","last_fullsized":"2023-11-28T11:08:18.368650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:35.096193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:19.074332+0000","last_change":"2023-11-28T11:08:17.385964+0000","last_active":"2023-11-28T11:08:19.074332+0000","last_peered":"2023-11-28T11:08:19.074332+0000","last_clean":"2023-11-28T11:08:19.074332+0000","last_became_active":"2023-11-28T11:08:17.385673+0000","last_became_peered":"2023-11-28T11:08:17.385673+0000","last_unstale":"2023-11-28T11:08:19.074332+0000","last_undegraded":"2023-11-28T11:08:19.074332+0000","last_fullsized":"2023-11-28T11:08:19.074332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:37.983570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.365540+0000","last_change":"2023-11-28T11:08:17.372667+0000","last_active":"2023-11-28T11:08:18.365540+0000","last_peered":"2023-11-28T11:08:18.365540+0000","last_clean":"2023-11-28T11:08:18.365540+0000","last_became_active":"2023-11-28T11:08:17.372363+0000","last_became_peered":"2023-11-28T11:08:17.372363+0000","last_unstale":"2023-11-28T11:08:18.365540+0000","last_undegraded":"2023-11-28T11:08:18.365540+0000","last_fullsized":"2023-11-28T11:08:18.365540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:25.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368769+0000","last_change":"2023-11-28T11:08:17.386764+0000","last_active":"2023-11-28T11:08:18.368769+0000","last_peered":"2023-11-28T11:08:18.368769+0000","last_clean":"2023-11-28T11:08:18.368769+0000","last_became_active":"2023-11-28T11:08:17.386525+0000","last_became_peered":"2023-11-28T11:08:17.386525+0000","last_unstale":"2023-11-28T11:08:18.368769+0000","last_undegraded":"2023-11-28T11:08:18.368769+0000","last_fullsized":"2023-11-28T11:08:18.368769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:42.135077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.366116+0000","last_change":"2023-11-28T11:08:17.378211+0000","last_active":"2023-11-28T11:08:18.366116+0000","last_peered":"2023-11-28T11:08:18.366116+0000","last_clean":"2023-11-28T11:08:18.366116+0000","last_became_active":"2023-11-28T11:08:17.376746+0000","last_became_peered":"2023-11-28T11:08:17.376746+0000","last_unstale":"2023-11-28T11:08:18.366116+0000","last_undegraded":"2023-11-28T11:08:18.366116+0000","last_fullsized":"2023-11-28T11:08:18.366116+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:46.129189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T11:08:18.368303+0000","last_change":"2023-11-28T11:08:17.375172+0000","last_active":"2023-11-28T11:08:18.368303+0000","last_peered":"2023-11-28T11:08:18.368303+0000","last_clean":"2023-11-28T11:08:18.368303+0000","last_became_active":"2023-11-28T11:08:17.374911+0000","last_became_peered":"2023-11-28T11:08:17.374911+0000","last_unstale":"2023-11-28T11:08:18.368303+0000","last_undegraded":"2023-11-28T11:08:18.368303+0000","last_fullsized":"2023-11-28T11:08:18.368303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:16.954910+0000","last_clean_scrub_stamp":"2023-11-28T11:08:16.954910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:36.093523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9108,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362732,"statfs":{"total":96636764160,"available":96627437568,"internally_reserved":0,"allocated":864256,"data_stored":681731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7988,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363852,"statfs":{"total":96636764160,"available":96628584448,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1239999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.0250000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.0620000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7988,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363852,"statfs":{"total":96636764160,"available":96628584448,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.224}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":680533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7924,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363916,"statfs":{"total":96636764160,"available":96628649984,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8056,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363784,"statfs":{"total":96636764160,"available":96628514816,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":91203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7996,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363844,"statfs":{"total":96636764160,"available":96628576256,"internally_reserved":0,"allocated":249856,"data_stored":90235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:08:31.107 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T11:08:31.107 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T11:08:31.107 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T11:08:31.108 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:08:31.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.261+0000 7f4e7991a700 1 Processor -- start 2023-11-28T11:08:31.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.261+0000 7f4e7991a700 1 -- start start 2023-11-28T11:08:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.262+0000 7f4e7991a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4e7415c9b0 0x7f4e740b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.262+0000 7f4e7991a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e740d1950 con 0x7f4e74161340 2023-11-28T11:08:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.262+0000 7f4e7991a700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e740d24f0 con 0x7f4e740cd7c0 2023-11-28T11:08:31.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.262+0000 7f4e7991a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e740b5800 con 0x7f4e7415c9b0 2023-11-28T11:08:31.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.262+0000 7f4e727fc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4e74161340 0x7f4e740d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47212/0 (socket says 172.21.15.156:47212) 2023-11-28T11:08:31.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.262+0000 7f4e727fc700 1 -- 172.21.15.156:0/755622241 learned_addr learned my addr 172.21.15.156:0/755622241 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:31.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.262+0000 7f4e72ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4e7415c9b0 0x7f4e740b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.262+0000 7f4e72ffd700 1 -- 172.21.15.156:0/755622241 >> v1:172.21.15.186:6789/0 conn(0x7f4e740cd7c0 legacy=0x7f4e740b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e72ffd700 1 -- 172.21.15.156:0/755622241 >> v1:172.21.15.156:6789/0 conn(0x7f4e74161340 legacy=0x7f4e740d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.262+0000 7f4e71ffb700 1 -- 172.21.15.156:0/755622241 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961578966 0 0) 0x7f4e740d1950 con 0x7f4e74161340 2023-11-28T11:08:31.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e72ffd700 1 -- 172.21.15.156:0/755622241 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e740b5520 con 0x7f4e7415c9b0 2023-11-28T11:08:31.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e72ffd700 1 --2- 172.21.15.156:0/755622241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e7415c9b0 0x7f4e740b4bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009840 tx=0x7f4e68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca568ae8461c3f19 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e71ffb700 1 -- 172.21.15.156:0/755622241 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68011830 con 0x7f4e7415c9b0 2023-11-28T11:08:31.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e71ffb700 1 -- 172.21.15.156:0/755622241 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e68011990 con 0x7f4e7415c9b0 2023-11-28T11:08:31.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e71ffb700 1 -- 172.21.15.156:0/755622241 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68011d20 con 0x7f4e7415c9b0 2023-11-28T11:08:31.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e7991a700 1 -- 172.21.15.156:0/755622241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e7415c9b0 msgr2=0x7f4e740b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e7991a700 1 --2- 172.21.15.156:0/755622241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e7415c9b0 0x7f4e740b4bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009840 tx=0x7f4e68009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:31.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e7991a700 1 -- 172.21.15.156:0/755622241 shutdown_connections 2023-11-28T11:08:31.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e7991a700 1 --2- 172.21.15.156:0/755622241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e7415c9b0 0x7f4e740b4bf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e7991a700 1 -- 172.21.15.156:0/755622241 >> 172.21.15.156:0/755622241 conn(0x7f4e74072600 msgr2=0x7f4e740cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e7991a700 1 -- 172.21.15.156:0/755622241 shutdown_connections 2023-11-28T11:08:31.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.263+0000 7f4e7991a700 1 -- 172.21.15.156:0/755622241 wait complete. 2023-11-28T11:08:31.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.264+0000 7f4e7991a700 1 Processor -- start 2023-11-28T11:08:31.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.265+0000 7f4e7991a700 1 -- start start 2023-11-28T11:08:31.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.265+0000 7f4e7991a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e740cd7c0 0x7f4e741f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.265+0000 7f4e7991a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e7415c9b0 0x7f4e741f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.265+0000 7f4e7991a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e74161340 0x7f4e741f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.265+0000 7f4e7991a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7414e0b0 con 0x7f4e7415c9b0 2023-11-28T11:08:31.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.265+0000 7f4e7991a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e741f3450 con 0x7f4e740cd7c0 2023-11-28T11:08:31.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.265+0000 7f4e7991a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7415e280 con 0x7f4e74161340 2023-11-28T11:08:31.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e727fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e7415c9b0 0x7f4e741f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e727fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e7415c9b0 0x7f4e741f2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60570/0 (socket says 172.21.15.156:60570) 2023-11-28T11:08:31.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e727fc700 1 -- 172.21.15.156:0/2914838409 learned_addr learned my addr 172.21.15.156:0/2914838409 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:31.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e72ffd700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e740cd7c0 0x7f4e741f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e737fe700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e74161340 0x7f4e741f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e727fc700 1 -- 172.21.15.156:0/2914838409 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e74161340 msgr2=0x7f4e741f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e727fc700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e74161340 0x7f4e741f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e727fc700 1 -- 172.21.15.156:0/2914838409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e740cd7c0 msgr2=0x7f4e741f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e727fc700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e740cd7c0 0x7f4e741f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e727fc700 1 -- 172.21.15.156:0/2914838409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e680094a0 con 0x7f4e7415c9b0 2023-11-28T11:08:31.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e72ffd700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e740cd7c0 0x7f4e741f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:31.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e727fc700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e7415c9b0 0x7f4e741f2f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400c520 tx=0x7f4e6400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e5bfff700 1 -- 172.21.15.156:0/2914838409 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64004c80 con 0x7f4e7415c9b0 2023-11-28T11:08:31.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e5bfff700 1 -- 172.21.15.156:0/2914838409 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e64004e00 con 0x7f4e7415c9b0 2023-11-28T11:08:31.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e7991a700 1 -- 172.21.15.156:0/2914838409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e740d15e0 con 0x7f4e7415c9b0 2023-11-28T11:08:31.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e5bfff700 1 -- 172.21.15.156:0/2914838409 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64007310 con 0x7f4e7415c9b0 2023-11-28T11:08:31.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.266+0000 7f4e7991a700 1 -- 172.21.15.156:0/2914838409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e740d1300 con 0x7f4e7415c9b0 2023-11-28T11:08:31.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.267+0000 7f4e7991a700 1 -- 172.21.15.156:0/2914838409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e740b5130 con 0x7f4e7415c9b0 2023-11-28T11:08:31.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.267+0000 7f4e5bfff700 1 -- 172.21.15.156:0/2914838409 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e6401a020 con 0x7f4e7415c9b0 2023-11-28T11:08:31.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.268+0000 7f4e5bfff700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4e600682f0 0x7f4e6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.268+0000 7f4e5bfff700 1 -- 172.21.15.156:0/2914838409 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e640858a0 con 0x7f4e7415c9b0 2023-11-28T11:08:31.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.268+0000 7f4e72ffd700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4e600682f0 0x7f4e6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.268+0000 7f4e72ffd700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4e600682f0 0x7f4e6006a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4e68000c00 tx=0x7f4e6801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.272+0000 7f4e5bfff700 1 -- 172.21.15.156:0/2914838409 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e64052100 con 0x7f4e7415c9b0 2023-11-28T11:08:31.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.498+0000 7f4e7991a700 1 -- 172.21.15.156:0/2914838409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4e740745c0 con 0x7f4e7415c9b0 2023-11-28T11:08:31.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.498+0000 7f4e5bfff700 1 -- 172.21.15.156:0/2914838409 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4e64055720 con 0x7f4e7415c9b0 2023-11-28T11:08:31.499 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:31.499 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T11:08:31.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.500+0000 7f4e7991a700 1 -- 172.21.15.156:0/2914838409 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4e600682f0 msgr2=0x7f4e6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.500+0000 7f4e7991a700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4e600682f0 0x7f4e6006a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4e68000c00 tx=0x7f4e6801c040 comp rx=0 tx=0).stop 2023-11-28T11:08:31.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.500+0000 7f4e7991a700 1 -- 172.21.15.156:0/2914838409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e7415c9b0 msgr2=0x7f4e741f2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.500+0000 7f4e7991a700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e7415c9b0 0x7f4e741f2f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400c520 tx=0x7f4e6400c8e0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.500+0000 7f4e7991a700 1 -- 172.21.15.156:0/2914838409 shutdown_connections 2023-11-28T11:08:31.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.500+0000 7f4e7991a700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4e600682f0 0x7f4e6006a7a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.500+0000 7f4e7991a700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e74161340 0x7f4e741f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.500+0000 7f4e7991a700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e740cd7c0 0x7f4e741f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.500+0000 7f4e7991a700 1 --2- 172.21.15.156:0/2914838409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e7415c9b0 0x7f4e741f2f10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.500+0000 7f4e7991a700 1 -- 172.21.15.156:0/2914838409 >> 172.21.15.156:0/2914838409 conn(0x7f4e74072600 msgr2=0x7f4e7415ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.500+0000 7f4e7991a700 1 -- 172.21.15.156:0/2914838409 shutdown_connections 2023-11-28T11:08:31.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.500+0000 7f4e7991a700 1 -- 172.21.15.156:0/2914838409 wait complete. 2023-11-28T11:08:31.514 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T11:08:31.514 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:31.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.634+0000 7fa4f065c700 1 Processor -- start 2023-11-28T11:08:31.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.634+0000 7fa4f065c700 1 -- start start 2023-11-28T11:08:31.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4f065c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e80d1330 0x7fa4e80b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4f065c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa4e80b20b0 0x7fa4e80b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4f065c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e80b0230 con 0x7fa4e80d01b0 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4f065c700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e815cec0 con 0x7fa4e80b20b0 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4f065c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80b4570 con 0x7fa4e80d1330 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ee3f8700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa4e80d01b0 0x7fa4e80d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47224/0 (socket says 172.21.15.156:47224) 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ee3f8700 1 -- 172.21.15.156:0/3326065568 learned_addr learned my addr 172.21.15.156:0/3326065568 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4edbf7700 1 --2- 172.21.15.156:0/3326065568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e80d1330 0x7fa4e80b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4eebf9700 1 --2- 172.21.15.156:0/3326065568 >> v2:172.21.15.186:3300/0 conn(0x7fa4e80b20b0 0x7fa4e80b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541207968 0 0) 0x7fa4e80b0230 con 0x7fa4e80d01b0 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4d40035b0 con 0x7fa4e80d01b0 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4076272808 0 0) 0x7fa4d40035b0 con 0x7fa4e80d01b0 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e80d1330 msgr2=0x7fa4e80b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 --2- 172.21.15.156:0/3326065568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e80d1330 0x7fa4e80b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 >> v2:172.21.15.186:3300/0 conn(0x7fa4e80b20b0 msgr2=0x7fa4e80b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:31.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 --2- 172.21.15.156:0/3326065568 >> v2:172.21.15.186:3300/0 conn(0x7fa4e80b20b0 0x7fa4e80b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e80b4850 con 0x7fa4e80d01b0 2023-11-28T11:08:31.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fa4e4001670 con 0x7fa4e80d01b0 2023-11-28T11:08:31.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4eebf9700 1 --2- 172.21.15.156:0/3326065568 >> v2:172.21.15.186:3300/0 conn(0x7fa4e80b20b0 0x7fa4e80b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:31.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 >> v1:172.21.15.156:6789/0 conn(0x7fa4e80d01b0 legacy=0x7fa4e80d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 --2- 172.21.15.156:0/3326065568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e80d1330 0x7fa4d4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 --2- 172.21.15.156:0/3326065568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4d4003f80 0x7fa4d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 --2- 172.21.15.156:0/3326065568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4d4004d90 0x7fa4d4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e815cec0 con 0x7fa4d4003f80 2023-11-28T11:08:31.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80b4570 con 0x7fa4d4004d90 2023-11-28T11:08:31.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.635+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d400ba10 con 0x7fa4e80d1330 2023-11-28T11:08:31.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.636+0000 7fa4edbf7700 1 --2- 172.21.15.156:0/3326065568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4d4003f80 0x7fa4d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.636+0000 7fa4ee3f8700 1 --2- 172.21.15.156:0/3326065568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e80d1330 0x7fa4d4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.636+0000 7fa4eebf9700 1 --2- 172.21.15.156:0/3326065568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4d4004d90 0x7fa4d4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.636+0000 7fa4edbf7700 1 -- 172.21.15.156:0/3326065568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e80d1330 msgr2=0x7fa4d4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:31.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.636+0000 7fa4edbf7700 1 --2- 172.21.15.156:0/3326065568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e80d1330 0x7fa4d4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.636+0000 7fa4edbf7700 1 -- 172.21.15.156:0/3326065568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4d4004d90 msgr2=0x7fa4d4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:31.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.636+0000 7fa4edbf7700 1 --2- 172.21.15.156:0/3326065568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4d4004d90 0x7fa4d4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.636+0000 7fa4edbf7700 1 -- 172.21.15.156:0/3326065568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d40035b0 con 0x7fa4d4003f80 2023-11-28T11:08:31.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.636+0000 7fa4edbf7700 1 --2- 172.21.15.156:0/3326065568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4d4003f80 0x7fa4d4004850 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009b30 tx=0x7fa4d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb89cc9459adb0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.636+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800e6c0 con 0x7fa4d4003f80 2023-11-28T11:08:31.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.636+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d800e820 con 0x7fa4d4003f80 2023-11-28T11:08:31.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.636+0000 7fa4ed3f6700 1 -- 172.21.15.156:0/3326065568 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800eb90 con 0x7fa4d4003f80 2023-11-28T11:08:31.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.637+0000 7fa4f065c700 1 -- 172.21.15.156:0/3326065568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4d4003f80 msgr2=0x7fa4d4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.637+0000 7fa4f065c700 1 --2- 172.21.15.156:0/3326065568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4d4003f80 0x7fa4d4004850 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009b30 tx=0x7fa4d4005880 comp rx=0 tx=0).stop 2023-11-28T11:08:31.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.637+0000 7fa4eebf9700 1 -- 172.21.15.156:0/3326065568 reap_dead start 2023-11-28T11:08:31.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.637+0000 7fa4f065c700 1 -- 172.21.15.156:0/3326065568 shutdown_connections 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.637+0000 7fa4f065c700 1 -- 172.21.15.156:0/3326065568 >> 172.21.15.156:0/3326065568 conn(0x7fa4e809c3f0 msgr2=0x7fa4e80d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.637+0000 7fa4f065c700 1 -- 172.21.15.156:0/3326065568 shutdown_connections 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.637+0000 7fa4f065c700 1 -- 172.21.15.156:0/3326065568 wait complete. 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.638+0000 7fa4f065c700 1 Processor -- start 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.638+0000 7fa4f065c700 1 -- start start 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4f065c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4d4003f80 0x7fa4e815a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4f065c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e809d550 0x7fa4e80b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4f065c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e809d9c0 0x7fa4e80b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4f065c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81f56c0 con 0x7fa4e809d9c0 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4f065c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81f6240 con 0x7fa4d4003f80 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4f065c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81f5910 con 0x7fa4e809d550 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4eebf9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e809d9c0 0x7fa4e80b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4eebf9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e809d9c0 0x7fa4e80b83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60594/0 (socket says 172.21.15.156:60594) 2023-11-28T11:08:31.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4eebf9700 1 -- 172.21.15.156:0/2374265705 learned_addr learned my addr 172.21.15.156:0/2374265705 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:31.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4edbf7700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e809d550 0x7fa4e80b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4ee3f8700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4d4003f80 0x7fa4e815a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4eebf9700 1 -- 172.21.15.156:0/2374265705 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e809d550 msgr2=0x7fa4e80b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4eebf9700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e809d550 0x7fa4e80b7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4eebf9700 1 -- 172.21.15.156:0/2374265705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4d4003f80 msgr2=0x7fa4e815a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4eebf9700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4d4003f80 0x7fa4e815a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4eebf9700 1 -- 172.21.15.156:0/2374265705 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e80afec0 con 0x7fa4e809d9c0 2023-11-28T11:08:31.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4ee3f8700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4d4003f80 0x7fa4e815a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:31.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4eebf9700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e809d9c0 0x7fa4e80b83d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80b2530 tx=0x7fa4e000f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4deffd700 1 -- 172.21.15.156:0/2374265705 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e001c070 con 0x7fa4e809d9c0 2023-11-28T11:08:31.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4deffd700 1 -- 172.21.15.156:0/2374265705 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e000f040 con 0x7fa4e809d9c0 2023-11-28T11:08:31.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4f065c700 1 -- 172.21.15.156:0/2374265705 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e80afbe0 con 0x7fa4e809d9c0 2023-11-28T11:08:31.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4deffd700 1 -- 172.21.15.156:0/2374265705 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e000e040 con 0x7fa4e809d9c0 2023-11-28T11:08:31.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.639+0000 7fa4f065c700 1 -- 172.21.15.156:0/2374265705 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e80b2600 con 0x7fa4e809d9c0 2023-11-28T11:08:31.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.640+0000 7fa4f065c700 1 -- 172.21.15.156:0/2374265705 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e8068690 con 0x7fa4e809d9c0 2023-11-28T11:08:31.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.641+0000 7fa4deffd700 1 -- 172.21.15.156:0/2374265705 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa4e0015430 con 0x7fa4e809d9c0 2023-11-28T11:08:31.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.641+0000 7fa4deffd700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4cc0682f0 0x7fa4cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.641+0000 7fa4deffd700 1 -- 172.21.15.156:0/2374265705 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa4e0085cb0 con 0x7fa4e809d9c0 2023-11-28T11:08:31.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.641+0000 7fa4ee3f8700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4cc0682f0 0x7fa4cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.641+0000 7fa4ee3f8700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4cc0682f0 0x7fa4cc06a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4000fb0 tx=0x7fa4e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.645+0000 7fa4deffd700 1 -- 172.21.15.156:0/2374265705 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa4e00524b0 con 0x7fa4e809d9c0 2023-11-28T11:08:31.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.829+0000 7fa4f065c700 1 -- 172.21.15.156:0/2374265705 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4e81f6380 con 0x7fa4e809d9c0 2023-11-28T11:08:31.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.830+0000 7fa4deffd700 1 -- 172.21.15.156:0/2374265705 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17955 (secure 0 0 0) 0x7fa4e0055ad0 con 0x7fa4e809d9c0 2023-11-28T11:08:31.832 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:31.832 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:18.354009+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:31.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.832+0000 7fa4f065c700 1 -- 172.21.15.156:0/2374265705 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4cc0682f0 msgr2=0x7fa4cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.832+0000 7fa4f065c700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4cc0682f0 0x7fa4cc06a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4000fb0 tx=0x7fa4e4009040 comp rx=0 tx=0).stop 2023-11-28T11:08:31.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.832+0000 7fa4f065c700 1 -- 172.21.15.156:0/2374265705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e809d9c0 msgr2=0x7fa4e80b83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.832+0000 7fa4f065c700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e809d9c0 0x7fa4e80b83d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80b2530 tx=0x7fa4e000f490 comp rx=0 tx=0).stop 2023-11-28T11:08:31.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.833+0000 7fa4f065c700 1 -- 172.21.15.156:0/2374265705 shutdown_connections 2023-11-28T11:08:31.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.833+0000 7fa4f065c700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e809d9c0 0x7fa4e80b83d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.833+0000 7fa4f065c700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4d4003f80 0x7fa4e815a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.833+0000 7fa4f065c700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4cc0682f0 0x7fa4cc06a7a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.833+0000 7fa4f065c700 1 --2- 172.21.15.156:0/2374265705 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4e809d550 0x7fa4e80b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.833+0000 7fa4f065c700 1 -- 172.21.15.156:0/2374265705 >> 172.21.15.156:0/2374265705 conn(0x7fa4e809c3f0 msgr2=0x7fa4e80b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.833+0000 7fa4f065c700 1 -- 172.21.15.156:0/2374265705 shutdown_connections 2023-11-28T11:08:31.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.833+0000 7fa4f065c700 1 -- 172.21.15.156:0/2374265705 wait complete. 2023-11-28T11:08:31.846 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:15.021591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:16.332241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:31.847 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:31.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.969+0000 7f7a2f4f5700 1 Processor -- start 2023-11-28T11:08:31.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.969+0000 7f7a2f4f5700 1 -- start start 2023-11-28T11:08:31.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2f4f5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a28071040 0x7f7a28071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2f4f5700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7a28160540 0x7f7a281609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2f4f5700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a28071950 con 0x7f7a2806dee0 2023-11-28T11:08:31.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2f4f5700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a2815d060 con 0x7f7a28160540 2023-11-28T11:08:31.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2f4f5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2815d340 con 0x7f7a28071040 2023-11-28T11:08:31.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2da92700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7a28160540 0x7f7a281609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2ca90700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7a2806dee0 0x7f7a2806e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47244/0 (socket says 172.21.15.156:47244) 2023-11-28T11:08:31.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2ca90700 1 -- 172.21.15.156:0/2968684998 learned_addr learned my addr 172.21.15.156:0/2968684998 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:31.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2d291700 1 --2- 172.21.15.156:0/2968684998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a28071040 0x7f7a28071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2da92700 1 -- 172.21.15.156:0/2968684998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a28071040 msgr2=0x7f7a28071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:31.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2da92700 1 --2- 172.21.15.156:0/2968684998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a28071040 0x7f7a28071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2da92700 1 -- 172.21.15.156:0/2968684998 >> v1:172.21.15.156:6789/0 conn(0x7f7a2806dee0 legacy=0x7f7a2806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a1ffff700 1 -- 172.21.15.156:0/2968684998 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122994678 0 0) 0x7f7a28071950 con 0x7f7a2806dee0 2023-11-28T11:08:31.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2da92700 1 -- 172.21.15.156:0/2968684998 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2815d620 con 0x7f7a28160540 2023-11-28T11:08:31.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.970+0000 7f7a2d291700 1 --2- 172.21.15.156:0/2968684998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a28071040 0x7f7a28071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:31.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.971+0000 7f7a2da92700 1 --2- 172.21.15.156:0/2968684998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28160540 0x7f7a281609c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009d90 tx=0x7f7a2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2a8f7edfbdd515 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.971+0000 7f7a1ffff700 1 -- 172.21.15.156:0/2968684998 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000e070 con 0x7f7a28160540 2023-11-28T11:08:31.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.971+0000 7f7a1ffff700 1 -- 172.21.15.156:0/2968684998 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2000bbd0 con 0x7f7a28160540 2023-11-28T11:08:31.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.971+0000 7f7a1ffff700 1 -- 172.21.15.156:0/2968684998 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20012430 con 0x7f7a28160540 2023-11-28T11:08:31.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.971+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/2968684998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28160540 msgr2=0x7f7a281609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.971+0000 7f7a2f4f5700 1 --2- 172.21.15.156:0/2968684998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28160540 0x7f7a281609c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009d90 tx=0x7f7a2000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:31.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.971+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/2968684998 shutdown_connections 2023-11-28T11:08:31.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.971+0000 7f7a2f4f5700 1 --2- 172.21.15.156:0/2968684998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28160540 0x7f7a281609c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.971+0000 7f7a2f4f5700 1 --2- 172.21.15.156:0/2968684998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a28071040 0x7f7a28071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.971+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/2968684998 >> 172.21.15.156:0/2968684998 conn(0x7f7a280d0000 msgr2=0x7f7a280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.971+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/2968684998 shutdown_connections 2023-11-28T11:08:31.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.971+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/2968684998 wait complete. 2023-11-28T11:08:31.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.972+0000 7f7a2f4f5700 1 Processor -- start 2023-11-28T11:08:31.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.973+0000 7f7a2f4f5700 1 -- start start 2023-11-28T11:08:31.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.973+0000 7f7a2f4f5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a2806dee0 0x7f7a280b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.973+0000 7f7a2f4f5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28071040 0x7f7a280b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.973+0000 7f7a2f4f5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a28160540 0x7f7a281f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.973+0000 7f7a2f4f5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a280c67f0 con 0x7f7a28071040 2023-11-28T11:08:31.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.973+0000 7f7a2f4f5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a280b4bf0 con 0x7f7a2806dee0 2023-11-28T11:08:31.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.973+0000 7f7a2f4f5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a280b4000 con 0x7f7a28160540 2023-11-28T11:08:31.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.973+0000 7f7a2ca90700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28071040 0x7f7a280b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.973+0000 7f7a2ca90700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28071040 0x7f7a280b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60604/0 (socket says 172.21.15.156:60604) 2023-11-28T11:08:31.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.973+0000 7f7a2ca90700 1 -- 172.21.15.156:0/3979927902 learned_addr learned my addr 172.21.15.156:0/3979927902 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:31.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.973+0000 7f7a2d291700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a2806dee0 0x7f7a280b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.973+0000 7f7a2da92700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a28160540 0x7f7a281f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.974+0000 7f7a2ca90700 1 -- 172.21.15.156:0/3979927902 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a28160540 msgr2=0x7f7a281f1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.974+0000 7f7a2ca90700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a28160540 0x7f7a281f1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.974+0000 7f7a2ca90700 1 -- 172.21.15.156:0/3979927902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a2806dee0 msgr2=0x7f7a280b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.974+0000 7f7a2ca90700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a2806dee0 0x7f7a280b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.974+0000 7f7a2ca90700 1 -- 172.21.15.156:0/3979927902 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a20009a70 con 0x7f7a28071040 2023-11-28T11:08:31.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.974+0000 7f7a2da92700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a28160540 0x7f7a281f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:31.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.974+0000 7f7a2ca90700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28071040 0x7f7a280b0df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400c550 tx=0x7f7a2400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.974+0000 7f7a1dffb700 1 -- 172.21.15.156:0/3979927902 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a24004c80 con 0x7f7a28071040 2023-11-28T11:08:31.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.974+0000 7f7a1dffb700 1 -- 172.21.15.156:0/3979927902 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a24004e00 con 0x7f7a28071040 2023-11-28T11:08:31.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.974+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/3979927902 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2815ccd0 con 0x7f7a28071040 2023-11-28T11:08:31.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.974+0000 7f7a1dffb700 1 -- 172.21.15.156:0/3979927902 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a24007310 con 0x7f7a28071040 2023-11-28T11:08:31.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.974+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/3979927902 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2815d900 con 0x7f7a28071040 2023-11-28T11:08:31.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.975+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/3979927902 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2806a270 con 0x7f7a28071040 2023-11-28T11:08:31.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.976+0000 7f7a1dffb700 1 -- 172.21.15.156:0/3979927902 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a2401a020 con 0x7f7a28071040 2023-11-28T11:08:31.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.976+0000 7f7a1dffb700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7a0c0682f0 0x7f7a0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.976+0000 7f7a1dffb700 1 -- 172.21.15.156:0/3979927902 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7a240857a0 con 0x7f7a28071040 2023-11-28T11:08:31.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.976+0000 7f7a2d291700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7a0c0682f0 0x7f7a0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.976+0000 7f7a2d291700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7a0c0682f0 0x7f7a0c06a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800c940 tx=0x7f7a18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:31.980+0000 7f7a1dffb700 1 -- 172.21.15.156:0/3979927902 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7a24052000 con 0x7f7a28071040 2023-11-28T11:08:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.161+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/3979927902 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a280720a0 con 0x7f7a28071040 2023-11-28T11:08:32.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.162+0000 7f7a1dffb700 1 -- 172.21.15.156:0/3979927902 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a24055620 con 0x7f7a28071040 2023-11-28T11:08:32.164 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.164+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/3979927902 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7a0c0682f0 msgr2=0x7f7a0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.164+0000 7f7a2f4f5700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7a0c0682f0 0x7f7a0c06a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800c940 tx=0x7f7a18008040 comp rx=0 tx=0).stop 2023-11-28T11:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.164+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/3979927902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28071040 msgr2=0x7f7a280b0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.164+0000 7f7a2f4f5700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28071040 0x7f7a280b0df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400c550 tx=0x7f7a2400c860 comp rx=0 tx=0).stop 2023-11-28T11:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.164+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/3979927902 shutdown_connections 2023-11-28T11:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.164+0000 7f7a2f4f5700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7a0c0682f0 0x7f7a0c06a7a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.164+0000 7f7a2f4f5700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a28160540 0x7f7a281f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.164+0000 7f7a2f4f5700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a2806dee0 0x7f7a280b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.164+0000 7f7a2f4f5700 1 --2- 172.21.15.156:0/3979927902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28071040 0x7f7a280b0df0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.164+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/3979927902 >> 172.21.15.156:0/3979927902 conn(0x7f7a280d0000 msgr2=0x7f7a2806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.164+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/3979927902 shutdown_connections 2023-11-28T11:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.164+0000 7f7a2f4f5700 1 -- 172.21.15.156:0/3979927902 wait complete. 2023-11-28T11:08:32.177 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:32.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.302+0000 7f0c3a999700 1 Processor -- start 2023-11-28T11:08:32.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.302+0000 7f0c3a999700 1 -- start start 2023-11-28T11:08:32.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.303+0000 7f0c3a999700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0c340ce040 0x7f0c3415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.303+0000 7f0c3a999700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0c340d0a00 0x7f0c3406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.303+0000 7f0c3a999700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3406b420 con 0x7f0c340b2930 2023-11-28T11:08:32.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.303+0000 7f0c3a999700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c3406b700 con 0x7f0c340d0a00 2023-11-28T11:08:32.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.303+0000 7f0c3a999700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c3406b9e0 con 0x7f0c340ce040 2023-11-28T11:08:32.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.303+0000 7f0c337fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0c340d0a00 0x7f0c3406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.303+0000 7f0c337fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0c340d0a00 0x7f0c3406f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60610/0 (socket says 172.21.15.156:60610) 2023-11-28T11:08:32.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.303+0000 7f0c337fe700 1 -- 172.21.15.156:0/2970940614 learned_addr learned my addr 172.21.15.156:0/2970940614 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:32.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.304+0000 7f0c33fff700 1 --2- 172.21.15.156:0/2970940614 >> v2:172.21.15.186:3300/0 conn(0x7f0c340ce040 0x7f0c3415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.304+0000 7f0c337fe700 1 -- 172.21.15.156:0/2970940614 >> v2:172.21.15.186:3300/0 conn(0x7f0c340ce040 msgr2=0x7f0c3415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.304+0000 7f0c337fe700 1 --2- 172.21.15.156:0/2970940614 >> v2:172.21.15.186:3300/0 conn(0x7f0c340ce040 0x7f0c3415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.304+0000 7f0c337fe700 1 -- 172.21.15.156:0/2970940614 >> v1:172.21.15.186:6789/0 conn(0x7f0c340b2930 legacy=0x7f0c3406c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.304+0000 7f0c337fe700 1 -- 172.21.15.156:0/2970940614 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3406bcc0 con 0x7f0c340d0a00 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.304+0000 7f0c337fe700 1 --2- 172.21.15.156:0/2970940614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c340d0a00 0x7f0c3406f850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c3406eab0 tx=0x7f0c24009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7d5546b2e31b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.304+0000 7f0c32ffd700 1 -- 172.21.15.156:0/2970940614 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400b870 con 0x7f0c340d0a00 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.304+0000 7f0c32ffd700 1 -- 172.21.15.156:0/2970940614 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2400b9d0 con 0x7f0c340d0a00 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.304+0000 7f0c32ffd700 1 -- 172.21.15.156:0/2970940614 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400bd60 con 0x7f0c340d0a00 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.305+0000 7f0c3a999700 1 -- 172.21.15.156:0/2970940614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c340d0a00 msgr2=0x7f0c3406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.305+0000 7f0c3a999700 1 --2- 172.21.15.156:0/2970940614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c340d0a00 0x7f0c3406f850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c3406eab0 tx=0x7f0c24009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.305+0000 7f0c3a999700 1 -- 172.21.15.156:0/2970940614 shutdown_connections 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.305+0000 7f0c3a999700 1 --2- 172.21.15.156:0/2970940614 >> v2:172.21.15.186:3300/0 conn(0x7f0c340ce040 0x7f0c3415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.305+0000 7f0c3a999700 1 --2- 172.21.15.156:0/2970940614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c340d0a00 0x7f0c3406f850 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.305+0000 7f0c3a999700 1 -- 172.21.15.156:0/2970940614 >> 172.21.15.156:0/2970940614 conn(0x7f0c3406aed0 msgr2=0x7f0c3406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:32.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.305+0000 7f0c3a999700 1 -- 172.21.15.156:0/2970940614 shutdown_connections 2023-11-28T11:08:32.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.305+0000 7f0c3a999700 1 -- 172.21.15.156:0/2970940614 wait complete. 2023-11-28T11:08:32.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.306+0000 7f0c3a999700 1 Processor -- start 2023-11-28T11:08:32.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.306+0000 7f0c3a999700 1 -- start start 2023-11-28T11:08:32.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c3a999700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c340b2930 0x7f0c34151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c3a999700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c340ce040 0x7f0c341520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c3a999700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c340d0a00 0x7f0c340b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c3a999700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c340b6520 con 0x7f0c340ce040 2023-11-28T11:08:32.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c3a999700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c340c7200 con 0x7f0c340d0a00 2023-11-28T11:08:32.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c3a999700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c340c74e0 con 0x7f0c340b2930 2023-11-28T11:08:32.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c337fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c340ce040 0x7f0c341520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c337fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c340ce040 0x7f0c341520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60616/0 (socket says 172.21.15.156:60616) 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c337fe700 1 -- 172.21.15.156:0/1806803753 learned_addr learned my addr 172.21.15.156:0/1806803753 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c33fff700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c340b2930 0x7f0c34151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c38f36700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c340d0a00 0x7f0c340b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c337fe700 1 -- 172.21.15.156:0/1806803753 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c340b2930 msgr2=0x7f0c34151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c337fe700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c340b2930 0x7f0c34151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c337fe700 1 -- 172.21.15.156:0/1806803753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c340d0a00 msgr2=0x7f0c340b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c337fe700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c340d0a00 0x7f0c340b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c337fe700 1 -- 172.21.15.156:0/1806803753 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c24009550 con 0x7f0c340ce040 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c337fe700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c340ce040 0x7f0c341520b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c24000c00 tx=0x7f0c24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.307+0000 7f0c30ff9700 1 -- 172.21.15.156:0/1806803753 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c24004480 con 0x7f0c340ce040 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.308+0000 7f0c30ff9700 1 -- 172.21.15.156:0/1806803753 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c240045e0 con 0x7f0c340ce040 2023-11-28T11:08:32.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.308+0000 7f0c3a999700 1 -- 172.21.15.156:0/1806803753 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3406fd90 con 0x7f0c340ce040 2023-11-28T11:08:32.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.308+0000 7f0c30ff9700 1 -- 172.21.15.156:0/1806803753 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c240038e0 con 0x7f0c340ce040 2023-11-28T11:08:32.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.308+0000 7f0c3a999700 1 -- 172.21.15.156:0/1806803753 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3406bfa0 con 0x7f0c340ce040 2023-11-28T11:08:32.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.308+0000 7f0c3a999700 1 -- 172.21.15.156:0/1806803753 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3406ff20 con 0x7f0c340ce040 2023-11-28T11:08:32.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.310+0000 7f0c30ff9700 1 -- 172.21.15.156:0/1806803753 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c2400f050 con 0x7f0c340ce040 2023-11-28T11:08:32.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.310+0000 7f0c30ff9700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c200682f0 0x7f0c2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.310+0000 7f0c30ff9700 1 -- 172.21.15.156:0/1806803753 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0c24087760 con 0x7f0c340ce040 2023-11-28T11:08:32.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.310+0000 7f0c33fff700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c200682f0 0x7f0c2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.311+0000 7f0c33fff700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c200682f0 0x7f0c2006a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0c28006fd0 tx=0x7f0c28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.313+0000 7f0c30ff9700 1 -- 172.21.15.156:0/1806803753 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0c24053f40 con 0x7f0c340ce040 2023-11-28T11:08:32.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.496+0000 7f0c3a999700 1 -- 172.21.15.156:0/1806803753 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0c340b6660 con 0x7f0c340ce040 2023-11-28T11:08:32.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.497+0000 7f0c30ff9700 1 -- 172.21.15.156:0/1806803753 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0c24015020 con 0x7f0c340ce040 2023-11-28T11:08:32.498 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:08:32.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.499+0000 7f0c3a999700 1 -- 172.21.15.156:0/1806803753 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c200682f0 msgr2=0x7f0c2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.499+0000 7f0c3a999700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c200682f0 0x7f0c2006a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0c28006fd0 tx=0x7f0c28008040 comp rx=0 tx=0).stop 2023-11-28T11:08:32.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.499+0000 7f0c3a999700 1 -- 172.21.15.156:0/1806803753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c340ce040 msgr2=0x7f0c341520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.499+0000 7f0c3a999700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c340ce040 0x7f0c341520b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c24000c00 tx=0x7f0c24004750 comp rx=0 tx=0).stop 2023-11-28T11:08:32.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.499+0000 7f0c3a999700 1 -- 172.21.15.156:0/1806803753 shutdown_connections 2023-11-28T11:08:32.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.499+0000 7f0c3a999700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c340d0a00 0x7f0c340b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.499+0000 7f0c3a999700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c200682f0 0x7f0c2006a7a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.499+0000 7f0c3a999700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c340b2930 0x7f0c34151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.499+0000 7f0c3a999700 1 --2- 172.21.15.156:0/1806803753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c340ce040 0x7f0c341520b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.499+0000 7f0c3a999700 1 -- 172.21.15.156:0/1806803753 >> 172.21.15.156:0/1806803753 conn(0x7f0c3406aed0 msgr2=0x7f0c3415e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:32.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.499+0000 7f0c3a999700 1 -- 172.21.15.156:0/1806803753 shutdown_connections 2023-11-28T11:08:32.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.499+0000 7f0c3a999700 1 -- 172.21.15.156:0/1806803753 wait complete. 2023-11-28T11:08:32.512 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:32.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.635+0000 7f6b5a010700 1 Processor -- start 2023-11-28T11:08:32.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.635+0000 7f6b5a010700 1 -- start start 2023-11-28T11:08:32.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b5a010700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6b54071040 0x7f6b54071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b5a010700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6b5406dee0 0x7f6b5406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b5a010700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b54071950 con 0x7f6b5406cdc0 2023-11-28T11:08:32.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b5a010700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b54161550 con 0x7f6b5406dee0 2023-11-28T11:08:32.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b5a010700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b5415d060 con 0x7f6b54071040 2023-11-28T11:08:32.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b52ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6b5406dee0 0x7f6b5406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b52ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6b5406dee0 0x7f6b5406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60624/0 (socket says 172.21.15.156:60624) 2023-11-28T11:08:32.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b52ffd700 1 -- 172.21.15.156:0/1451597265 learned_addr learned my addr 172.21.15.156:0/1451597265 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:32.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b537fe700 1 --2- 172.21.15.156:0/1451597265 >> v2:172.21.15.186:3300/0 conn(0x7f6b54071040 0x7f6b54071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b52ffd700 1 -- 172.21.15.156:0/1451597265 >> v2:172.21.15.186:3300/0 conn(0x7f6b54071040 msgr2=0x7f6b54071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:32.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b52ffd700 1 --2- 172.21.15.156:0/1451597265 >> v2:172.21.15.186:3300/0 conn(0x7f6b54071040 0x7f6b54071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b52ffd700 1 -- 172.21.15.156:0/1451597265 >> v1:172.21.15.186:6789/0 conn(0x7f6b5406cdc0 legacy=0x7f6b5406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b52ffd700 1 -- 172.21.15.156:0/1451597265 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5415d340 con 0x7f6b5406dee0 2023-11-28T11:08:32.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b52ffd700 1 --2- 172.21.15.156:0/1451597265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b5406dee0 0x7f6b5406e330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b540d0f50 tx=0x7f6b44009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7c3214df8a9a49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.636+0000 7f6b527fc700 1 -- 172.21.15.156:0/1451597265 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4400b870 con 0x7f6b5406dee0 2023-11-28T11:08:32.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.637+0000 7f6b527fc700 1 -- 172.21.15.156:0/1451597265 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4400b9d0 con 0x7f6b5406dee0 2023-11-28T11:08:32.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.637+0000 7f6b527fc700 1 -- 172.21.15.156:0/1451597265 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4400bd60 con 0x7f6b5406dee0 2023-11-28T11:08:32.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.637+0000 7f6b5a010700 1 -- 172.21.15.156:0/1451597265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b5406dee0 msgr2=0x7f6b5406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.637+0000 7f6b5a010700 1 --2- 172.21.15.156:0/1451597265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b5406dee0 0x7f6b5406e330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b540d0f50 tx=0x7f6b44009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.637+0000 7f6b5a010700 1 -- 172.21.15.156:0/1451597265 shutdown_connections 2023-11-28T11:08:32.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.637+0000 7f6b5a010700 1 --2- 172.21.15.156:0/1451597265 >> v2:172.21.15.186:3300/0 conn(0x7f6b54071040 0x7f6b54071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.637+0000 7f6b5a010700 1 --2- 172.21.15.156:0/1451597265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b5406dee0 0x7f6b5406e330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.637+0000 7f6b5a010700 1 -- 172.21.15.156:0/1451597265 >> 172.21.15.156:0/1451597265 conn(0x7f6b540d0000 msgr2=0x7f6b540d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:32.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.637+0000 7f6b5a010700 1 -- 172.21.15.156:0/1451597265 shutdown_connections 2023-11-28T11:08:32.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.637+0000 7f6b5a010700 1 -- 172.21.15.156:0/1451597265 wait complete. 2023-11-28T11:08:32.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.639+0000 7f6b5a010700 1 Processor -- start 2023-11-28T11:08:32.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.639+0000 7f6b5a010700 1 -- start start 2023-11-28T11:08:32.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b5a010700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b5406cdc0 0x7f6b540b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b5a010700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b5406dee0 0x7f6b540cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b5a010700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b54071040 0x7f6b540cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b5a010700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b541f3620 con 0x7f6b54071040 2023-11-28T11:08:32.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b5a010700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b541f2cf0 con 0x7f6b5406dee0 2023-11-28T11:08:32.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b5a010700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b541f2fd0 con 0x7f6b5406cdc0 2023-11-28T11:08:32.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b53fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b54071040 0x7f6b540cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b53fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b54071040 0x7f6b540cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60632/0 (socket says 172.21.15.156:60632) 2023-11-28T11:08:32.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b53fff700 1 -- 172.21.15.156:0/3305577605 learned_addr learned my addr 172.21.15.156:0/3305577605 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b537fe700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b5406cdc0 0x7f6b540b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b53fff700 1 -- 172.21.15.156:0/3305577605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b5406cdc0 msgr2=0x7f6b540b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b53fff700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b5406cdc0 0x7f6b540b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b52ffd700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b5406dee0 0x7f6b540cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b53fff700 1 -- 172.21.15.156:0/3305577605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b5406dee0 msgr2=0x7f6b540cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b53fff700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b5406dee0 0x7f6b540cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b53fff700 1 -- 172.21.15.156:0/3305577605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b44009550 con 0x7f6b54071040 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b53fff700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b54071040 0x7f6b540cf920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00cb50 tx=0x7f6b4c00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b3bfff700 1 -- 172.21.15.156:0/3305577605 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c003c00 con 0x7f6b54071040 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b3bfff700 1 -- 172.21.15.156:0/3305577605 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c003d60 con 0x7f6b54071040 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b5a010700 1 -- 172.21.15.156:0/3305577605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5415d900 con 0x7f6b54071040 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b3bfff700 1 -- 172.21.15.156:0/3305577605 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c00ab70 con 0x7f6b54071040 2023-11-28T11:08:32.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.640+0000 7f6b5a010700 1 -- 172.21.15.156:0/3305577605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5415d620 con 0x7f6b54071040 2023-11-28T11:08:32.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.641+0000 7f6b5a010700 1 -- 172.21.15.156:0/3305577605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5406e870 con 0x7f6b54071040 2023-11-28T11:08:32.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.642+0000 7f6b3bfff700 1 -- 172.21.15.156:0/3305577605 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6b4c010050 con 0x7f6b54071040 2023-11-28T11:08:32.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.642+0000 7f6b3bfff700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6b400682f0 0x7f6b4006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.642+0000 7f6b3bfff700 1 -- 172.21.15.156:0/3305577605 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6b4c085c60 con 0x7f6b54071040 2023-11-28T11:08:32.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.642+0000 7f6b537fe700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6b400682f0 0x7f6b4006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.643+0000 7f6b537fe700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6b400682f0 0x7f6b4006a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009770 tx=0x7f6b48008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.646+0000 7f6b3bfff700 1 -- 172.21.15.156:0/3305577605 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6b4c0523d0 con 0x7f6b54071040 2023-11-28T11:08:32.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.830+0000 7f6b5a010700 1 -- 172.21.15.156:0/3305577605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b541f3760 con 0x7f6b54071040 2023-11-28T11:08:32.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.831+0000 7f6b3bfff700 1 -- 172.21.15.156:0/3305577605 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6b4c0559f0 con 0x7f6b54071040 2023-11-28T11:08:32.832 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:08:32.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.833+0000 7f6b5a010700 1 -- 172.21.15.156:0/3305577605 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6b400682f0 msgr2=0x7f6b4006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.833+0000 7f6b5a010700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6b400682f0 0x7f6b4006a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009770 tx=0x7f6b48008040 comp rx=0 tx=0).stop 2023-11-28T11:08:32.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.833+0000 7f6b5a010700 1 -- 172.21.15.156:0/3305577605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b54071040 msgr2=0x7f6b540cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.833+0000 7f6b5a010700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b54071040 0x7f6b540cf920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00cb50 tx=0x7f6b4c00cf10 comp rx=0 tx=0).stop 2023-11-28T11:08:32.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.833+0000 7f6b5a010700 1 -- 172.21.15.156:0/3305577605 shutdown_connections 2023-11-28T11:08:32.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.833+0000 7f6b5a010700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b54071040 0x7f6b540cf920 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.833+0000 7f6b5a010700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6b400682f0 0x7f6b4006a7a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.833+0000 7f6b5a010700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b5406cdc0 0x7f6b540b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.833+0000 7f6b5a010700 1 --2- 172.21.15.156:0/3305577605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b5406dee0 0x7f6b540cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.833+0000 7f6b5a010700 1 -- 172.21.15.156:0/3305577605 >> 172.21.15.156:0/3305577605 conn(0x7f6b540d0000 msgr2=0x7f6b5406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:32.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.833+0000 7f6b5a010700 1 -- 172.21.15.156:0/3305577605 shutdown_connections 2023-11-28T11:08:32.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.833+0000 7f6b5a010700 1 -- 172.21.15.156:0/3305577605 wait complete. 2023-11-28T11:08:32.846 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:32.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.968+0000 7fdfce016700 1 Processor -- start 2023-11-28T11:08:32.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.968+0000 7fdfce016700 1 -- start start 2023-11-28T11:08:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.969+0000 7fdfce016700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdfc8071040 0x7fdfc8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.969+0000 7fdfce016700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc8160540 0x7fdfc81609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.969+0000 7fdfce016700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc8071950 con 0x7fdfc806dee0 2023-11-28T11:08:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.969+0000 7fdfce016700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc815d060 con 0x7fdfc8071040 2023-11-28T11:08:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.969+0000 7fdfce016700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc815d340 con 0x7fdfc8160540 2023-11-28T11:08:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.969+0000 7fdfc77fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdfc8071040 0x7fdfc8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.969+0000 7fdfc77fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdfc8071040 0x7fdfc8071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60636/0 (socket says 172.21.15.156:60636) 2023-11-28T11:08:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.969+0000 7fdfc77fe700 1 -- 172.21.15.156:0/3762435254 learned_addr learned my addr 172.21.15.156:0/3762435254 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.969+0000 7fdfc7fff700 1 --2- 172.21.15.156:0/3762435254 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc8160540 0x7fdfc81609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfc77fe700 1 -- 172.21.15.156:0/3762435254 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc8160540 msgr2=0x7fdfc81609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfc77fe700 1 --2- 172.21.15.156:0/3762435254 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc8160540 0x7fdfc81609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfc77fe700 1 -- 172.21.15.156:0/3762435254 >> v1:172.21.15.186:6789/0 conn(0x7fdfc806dee0 legacy=0x7fdfc806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfc77fe700 1 -- 172.21.15.156:0/3762435254 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc815d620 con 0x7fdfc8071040 2023-11-28T11:08:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfc7fff700 1 --2- 172.21.15.156:0/3762435254 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc8160540 0x7fdfc81609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfc77fe700 1 --2- 172.21.15.156:0/3762435254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 0x7fdfc8071410 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80098f0 tx=0x7fdfb8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f5a531ca8d736e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfc67fc700 1 -- 172.21.15.156:0/3762435254 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb800b8a0 con 0x7fdfc8071040 2023-11-28T11:08:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfc67fc700 1 -- 172.21.15.156:0/3762435254 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb800ba00 con 0x7fdfc8071040 2023-11-28T11:08:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfc67fc700 1 -- 172.21.15.156:0/3762435254 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb800bd90 con 0x7fdfc8071040 2023-11-28T11:08:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfce016700 1 -- 172.21.15.156:0/3762435254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 msgr2=0x7fdfc8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfce016700 1 --2- 172.21.15.156:0/3762435254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 0x7fdfc8071410 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80098f0 tx=0x7fdfb8009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfce016700 1 -- 172.21.15.156:0/3762435254 shutdown_connections 2023-11-28T11:08:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfce016700 1 --2- 172.21.15.156:0/3762435254 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc8160540 0x7fdfc81609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfce016700 1 --2- 172.21.15.156:0/3762435254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 0x7fdfc8071410 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.970+0000 7fdfce016700 1 -- 172.21.15.156:0/3762435254 >> 172.21.15.156:0/3762435254 conn(0x7fdfc80d0000 msgr2=0x7fdfc80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.971+0000 7fdfce016700 1 -- 172.21.15.156:0/3762435254 shutdown_connections 2023-11-28T11:08:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.971+0000 7fdfce016700 1 -- 172.21.15.156:0/3762435254 wait complete. 2023-11-28T11:08:32.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.972+0000 7fdfce016700 1 Processor -- start 2023-11-28T11:08:32.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.972+0000 7fdfce016700 1 -- start start 2023-11-28T11:08:32.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfce016700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc806dee0 0x7fdfc80b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfce016700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc8071040 0x7fdfc80c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfce016700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8160540 0x7fdfc81f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfce016700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80b4700 con 0x7fdfc8160540 2023-11-28T11:08:32.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfce016700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc81fb690 con 0x7fdfc806dee0 2023-11-28T11:08:32.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfce016700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80b4930 con 0x7fdfc8071040 2023-11-28T11:08:32.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfc7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8160540 0x7fdfc81f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfc7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8160540 0x7fdfc81f6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60646/0 (socket says 172.21.15.156:60646) 2023-11-28T11:08:32.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfc7fff700 1 -- 172.21.15.156:0/2201091579 learned_addr learned my addr 172.21.15.156:0/2201091579 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfc6ffd700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc8071040 0x7fdfc80c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfc77fe700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc806dee0 0x7fdfc80b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfc7fff700 1 -- 172.21.15.156:0/2201091579 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc8071040 msgr2=0x7fdfc80c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfc7fff700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc8071040 0x7fdfc80c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfc7fff700 1 -- 172.21.15.156:0/2201091579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc806dee0 msgr2=0x7fdfc80b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfc7fff700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc806dee0 0x7fdfc80b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfc7fff700 1 -- 172.21.15.156:0/2201091579 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb8009550 con 0x7fdfc8160540 2023-11-28T11:08:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfc7fff700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8160540 0x7fdfc81f6a50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000cef0 tx=0x7fdfc000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.973+0000 7fdfaffff700 1 -- 172.21.15.156:0/2201091579 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000ad00 con 0x7fdfc8160540 2023-11-28T11:08:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.974+0000 7fdfce016700 1 -- 172.21.15.156:0/2201091579 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc815ccd0 con 0x7fdfc8160540 2023-11-28T11:08:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.974+0000 7fdfce016700 1 -- 172.21.15.156:0/2201091579 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc815d900 con 0x7fdfc8160540 2023-11-28T11:08:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.974+0000 7fdfaffff700 1 -- 172.21.15.156:0/2201091579 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc000ae60 con 0x7fdfc8160540 2023-11-28T11:08:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.974+0000 7fdfaffff700 1 -- 172.21.15.156:0/2201091579 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0015680 con 0x7fdfc8160540 2023-11-28T11:08:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.975+0000 7fdfce016700 1 -- 172.21.15.156:0/2201091579 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa80052f0 con 0x7fdfc8160540 2023-11-28T11:08:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.975+0000 7fdfaffff700 1 -- 172.21.15.156:0/2201091579 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdfc001c030 con 0x7fdfc8160540 2023-11-28T11:08:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.976+0000 7fdfaffff700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdfb40682f0 0x7fdfb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.976+0000 7fdfc77fe700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdfb40682f0 0x7fdfb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.976+0000 7fdfaffff700 1 -- 172.21.15.156:0/2201091579 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdfc0085f90 con 0x7fdfc8160540 2023-11-28T11:08:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.976+0000 7fdfc77fe700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdfb40682f0 0x7fdfb406a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8014040 tx=0x7fdfb8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:32.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:32.980+0000 7fdfaffff700 1 -- 172.21.15.156:0/2201091579 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdfc0052790 con 0x7fdfc8160540 2023-11-28T11:08:33.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.163+0000 7fdfce016700 1 -- 172.21.15.156:0/2201091579 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdfa8005b20 con 0x7fdfc8160540 2023-11-28T11:08:33.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.163+0000 7fdfaffff700 1 -- 172.21.15.156:0/2201091579 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdfc001a0a0 con 0x7fdfc8160540 2023-11-28T11:08:33.165 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:08:33.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.165+0000 7fdfce016700 1 -- 172.21.15.156:0/2201091579 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdfb40682f0 msgr2=0x7fdfb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.165+0000 7fdfce016700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdfb40682f0 0x7fdfb406a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8014040 tx=0x7fdfb8008030 comp rx=0 tx=0).stop 2023-11-28T11:08:33.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.165+0000 7fdfce016700 1 -- 172.21.15.156:0/2201091579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8160540 msgr2=0x7fdfc81f6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.165+0000 7fdfce016700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8160540 0x7fdfc81f6a50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000cef0 tx=0x7fdfc000cf20 comp rx=0 tx=0).stop 2023-11-28T11:08:33.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.165+0000 7fdfce016700 1 -- 172.21.15.156:0/2201091579 shutdown_connections 2023-11-28T11:08:33.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.165+0000 7fdfce016700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8160540 0x7fdfc81f6a50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.165+0000 7fdfce016700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc806dee0 0x7fdfc80b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.165+0000 7fdfce016700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdfb40682f0 0x7fdfb406a7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.165+0000 7fdfce016700 1 --2- 172.21.15.156:0/2201091579 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc8071040 0x7fdfc80c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.165+0000 7fdfce016700 1 -- 172.21.15.156:0/2201091579 >> 172.21.15.156:0/2201091579 conn(0x7fdfc80d0000 msgr2=0x7fdfc80aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:33.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.165+0000 7fdfce016700 1 -- 172.21.15.156:0/2201091579 shutdown_connections 2023-11-28T11:08:33.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.165+0000 7fdfce016700 1 -- 172.21.15.156:0/2201091579 wait complete. 2023-11-28T11:08:33.178 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:08:33.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.297+0000 7fce669ca700 1 Processor -- start 2023-11-28T11:08:33.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.297+0000 7fce669ca700 1 -- start start 2023-11-28T11:08:33.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.298+0000 7fce669ca700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce6006c990 0x7fce6006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.298+0000 7fce669ca700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fce600ca240 0x7fce600ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.298+0000 7fce669ca700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6006d320 con 0x7fce6006e9f0 2023-11-28T11:08:33.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.298+0000 7fce669ca700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6006ae60 con 0x7fce600ca240 2023-11-28T11:08:33.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.298+0000 7fce669ca700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6006a270 con 0x7fce6006c990 2023-11-28T11:08:33.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.298+0000 7fce64f67700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fce600ca240 0x7fce600ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.298+0000 7fce64f67700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fce600ca240 0x7fce600ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60654/0 (socket says 172.21.15.156:60654) 2023-11-28T11:08:33.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.298+0000 7fce64f67700 1 -- 172.21.15.156:0/2271872118 learned_addr learned my addr 172.21.15.156:0/2271872118 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:33.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.299+0000 7fce5f7fe700 1 --2- 172.21.15.156:0/2271872118 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce6006c990 0x7fce6006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.299+0000 7fce64f67700 1 -- 172.21.15.156:0/2271872118 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce6006c990 msgr2=0x7fce6006cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:33.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.299+0000 7fce64f67700 1 --2- 172.21.15.156:0/2271872118 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce6006c990 0x7fce6006cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.299+0000 7fce64f67700 1 -- 172.21.15.156:0/2271872118 >> v1:172.21.15.186:6789/0 conn(0x7fce6006e9f0 legacy=0x7fce6006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.299+0000 7fce64f67700 1 -- 172.21.15.156:0/2271872118 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce600cabe0 con 0x7fce600ca240 2023-11-28T11:08:33.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.299+0000 7fce64f67700 1 --2- 172.21.15.156:0/2271872118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce600ca240 0x7fce600ca6a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e50 tx=0x7fce5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cb8285d8a11881 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:33.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.299+0000 7fce5effd700 1 -- 172.21.15.156:0/2271872118 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5800e070 con 0x7fce600ca240 2023-11-28T11:08:33.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.299+0000 7fce5effd700 1 -- 172.21.15.156:0/2271872118 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5800b9c0 con 0x7fce600ca240 2023-11-28T11:08:33.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.299+0000 7fce5effd700 1 -- 172.21.15.156:0/2271872118 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5800bd70 con 0x7fce600ca240 2023-11-28T11:08:33.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.299+0000 7fce669ca700 1 -- 172.21.15.156:0/2271872118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce600ca240 msgr2=0x7fce600ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.299+0000 7fce669ca700 1 --2- 172.21.15.156:0/2271872118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce600ca240 0x7fce600ca6a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e50 tx=0x7fce5800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:33.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.300+0000 7fce669ca700 1 -- 172.21.15.156:0/2271872118 shutdown_connections 2023-11-28T11:08:33.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.300+0000 7fce669ca700 1 --2- 172.21.15.156:0/2271872118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce600ca240 0x7fce600ca6a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.300+0000 7fce669ca700 1 --2- 172.21.15.156:0/2271872118 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce6006c990 0x7fce6006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.300+0000 7fce669ca700 1 -- 172.21.15.156:0/2271872118 >> 172.21.15.156:0/2271872118 conn(0x7fce600b12b0 msgr2=0x7fce600b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:33.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.300+0000 7fce669ca700 1 -- 172.21.15.156:0/2271872118 shutdown_connections 2023-11-28T11:08:33.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.300+0000 7fce669ca700 1 -- 172.21.15.156:0/2271872118 wait complete. 2023-11-28T11:08:33.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.301+0000 7fce669ca700 1 Processor -- start 2023-11-28T11:08:33.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.301+0000 7fce669ca700 1 -- start start 2023-11-28T11:08:33.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce669ca700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce6006c990 0x7fce601540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce669ca700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6006e9f0 0x7fce60154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce669ca700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce600ca240 0x7fce60163f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce669ca700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce600c9060 con 0x7fce6006e9f0 2023-11-28T11:08:33.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce669ca700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6014e3e0 con 0x7fce6006c990 2023-11-28T11:08:33.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce669ca700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce600c61c0 con 0x7fce600ca240 2023-11-28T11:08:33.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6006e9f0 0x7fce60154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6006e9f0 0x7fce60154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60658/0 (socket says 172.21.15.156:60658) 2023-11-28T11:08:33.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5f7fe700 1 -- 172.21.15.156:0/276656115 learned_addr learned my addr 172.21.15.156:0/276656115 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5ffff700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce6006c990 0x7fce601540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce64f67700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce600ca240 0x7fce60163f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5f7fe700 1 -- 172.21.15.156:0/276656115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce600ca240 msgr2=0x7fce60163f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5f7fe700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce600ca240 0x7fce60163f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5f7fe700 1 -- 172.21.15.156:0/276656115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce6006c990 msgr2=0x7fce601540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5f7fe700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce6006c990 0x7fce601540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5f7fe700 1 -- 172.21.15.156:0/276656115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58009b30 con 0x7fce6006e9f0 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5f7fe700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6006e9f0 0x7fce60154600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fce5400d800 tx=0x7fce5400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5cff9700 1 -- 172.21.15.156:0/276656115 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce540125f0 con 0x7fce6006e9f0 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5cff9700 1 -- 172.21.15.156:0/276656115 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce54012750 con 0x7fce6006e9f0 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce669ca700 1 -- 172.21.15.156:0/276656115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6006a810 con 0x7fce6006e9f0 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce5cff9700 1 -- 172.21.15.156:0/276656115 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54012b00 con 0x7fce6006e9f0 2023-11-28T11:08:33.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.302+0000 7fce669ca700 1 -- 172.21.15.156:0/276656115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6006a530 con 0x7fce6006e9f0 2023-11-28T11:08:33.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.303+0000 7fce669ca700 1 -- 172.21.15.156:0/276656115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6006aaf0 con 0x7fce6006e9f0 2023-11-28T11:08:33.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.304+0000 7fce5cff9700 1 -- 172.21.15.156:0/276656115 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fce54012c60 con 0x7fce6006e9f0 2023-11-28T11:08:33.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.304+0000 7fce5cff9700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fce4c0682f0 0x7fce4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:33.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.304+0000 7fce5cff9700 1 -- 172.21.15.156:0/276656115 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fce54086130 con 0x7fce6006e9f0 2023-11-28T11:08:33.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.304+0000 7fce5ffff700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fce4c0682f0 0x7fce4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:33.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.304+0000 7fce5ffff700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fce4c0682f0 0x7fce4c06a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce50007fd0 tx=0x7fce50009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:33.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.308+0000 7fce5cff9700 1 -- 172.21.15.156:0/276656115 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fce54052990 con 0x7fce6006e9f0 2023-11-28T11:08:33.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.536+0000 7fce669ca700 1 -- 172.21.15.156:0/276656115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce600cd5b0 con 0x7fce6006e9f0 2023-11-28T11:08:33.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.536+0000 7fce5cff9700 1 -- 172.21.15.156:0/276656115 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7fce54055fb0 con 0x7fce6006e9f0 2023-11-28T11:08:33.538 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:33.538 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1221397150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1221397150},{"type":"v1","addr":"172.21.15.186:6834","nonce":1221397150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/22494485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":22494485},{"type":"v1","addr":"172.21.15.156:6835","nonce":22494485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3382922708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3382922708},{"type":"v1","addr":"172.21.15.156:6837","nonce":3382922708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:17.352335+0000","modified":"2023-11-28T11:08:20.752191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/4134790300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":4134790300},{"type":"v1","addr":"172.21.15.186:6835","nonce":4134790300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:08:33.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.538+0000 7fce669ca700 1 -- 172.21.15.156:0/276656115 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fce4c0682f0 msgr2=0x7fce4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.538+0000 7fce669ca700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fce4c0682f0 0x7fce4c06a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce50007fd0 tx=0x7fce50009040 comp rx=0 tx=0).stop 2023-11-28T11:08:33.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.538+0000 7fce669ca700 1 -- 172.21.15.156:0/276656115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6006e9f0 msgr2=0x7fce60154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:33.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.538+0000 7fce669ca700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6006e9f0 0x7fce60154600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fce5400d800 tx=0x7fce5400dbc0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.538+0000 7fce669ca700 1 -- 172.21.15.156:0/276656115 shutdown_connections 2023-11-28T11:08:33.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.538+0000 7fce669ca700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fce4c0682f0 0x7fce4c06a7a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.538+0000 7fce669ca700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce600ca240 0x7fce60163f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.538+0000 7fce669ca700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce6006c990 0x7fce601540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.538+0000 7fce669ca700 1 --2- 172.21.15.156:0/276656115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6006e9f0 0x7fce60154600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:33.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.538+0000 7fce669ca700 1 -- 172.21.15.156:0/276656115 >> 172.21.15.156:0/276656115 conn(0x7fce600b12b0 msgr2=0x7fce60065ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:33.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.539+0000 7fce669ca700 1 -- 172.21.15.156:0/276656115 shutdown_connections 2023-11-28T11:08:33.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:33.539+0000 7fce669ca700 1 -- 172.21.15.156:0/276656115 wait complete. 2023-11-28T11:08:33.541 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 8 2023-11-28T11:08:33.957 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:08:17.352335+0000', 'modified': '2023-11-28T11:08:20.752191+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6835/4134790300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6833', 'nonce': 4134790300}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 4134790300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:08:33.957 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:08:33.957 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T11:08:34.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.079+0000 7f8e2e4bb700 1 Processor -- start 2023-11-28T11:08:34.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.079+0000 7f8e2e4bb700 1 -- start start 2023-11-28T11:08:34.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.079+0000 7f8e2e4bb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8e2806dee0 0x7f8e2806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.079+0000 7f8e2e4bb700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2815d060 con 0x7f8e28160540 2023-11-28T11:08:34.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.079+0000 7f8e2e4bb700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2815d340 con 0x7f8e28071040 2023-11-28T11:08:34.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.079+0000 7f8e2e4bb700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2815d620 con 0x7f8e2806dee0 2023-11-28T11:08:34.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.080+0000 7f8e277fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8e2806dee0 0x7f8e2806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.080+0000 7f8e2ca58700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f8e28160540 0x7f8e28160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47314/0 (socket says 172.21.15.156:47314) 2023-11-28T11:08:34.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.080+0000 7f8e2ca58700 1 -- 172.21.15.156:0/1346070570 learned_addr learned my addr 172.21.15.156:0/1346070570 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:34.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.080+0000 7f8e277fe700 1 -- 172.21.15.156:0/1346070570 >> v1:172.21.15.186:6789/0 conn(0x7f8e28071040 legacy=0x7f8e28071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:34.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.080+0000 7f8e277fe700 1 -- 172.21.15.156:0/1346070570 >> v1:172.21.15.156:6789/0 conn(0x7f8e28160540 legacy=0x7f8e28160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.080+0000 7f8e277fe700 1 -- 172.21.15.156:0/1346070570 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2815d900 con 0x7f8e2806dee0 2023-11-28T11:08:34.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.080+0000 7f8e277fe700 1 --2- 172.21.15.156:0/1346070570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2806dee0 0x7f8e2806e330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009840 tx=0x7f8e1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a2c8aa17bdba7b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.080+0000 7f8e26ffd700 1 -- 172.21.15.156:0/1346070570 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c004230 con 0x7f8e2806dee0 2023-11-28T11:08:34.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.080+0000 7f8e26ffd700 1 -- 172.21.15.156:0/1346070570 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1c004390 con 0x7f8e2806dee0 2023-11-28T11:08:34.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.080+0000 7f8e26ffd700 1 -- 172.21.15.156:0/1346070570 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0036a0 con 0x7f8e2806dee0 2023-11-28T11:08:34.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.081+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/1346070570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2806dee0 msgr2=0x7f8e2806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.081+0000 7f8e2e4bb700 1 --2- 172.21.15.156:0/1346070570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2806dee0 0x7f8e2806e330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009840 tx=0x7f8e1c009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:34.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.081+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/1346070570 shutdown_connections 2023-11-28T11:08:34.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.081+0000 7f8e2e4bb700 1 --2- 172.21.15.156:0/1346070570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2806dee0 0x7f8e2806e330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.081+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/1346070570 >> 172.21.15.156:0/1346070570 conn(0x7f8e280d0000 msgr2=0x7f8e280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:34.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.081+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/1346070570 shutdown_connections 2023-11-28T11:08:34.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.081+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/1346070570 wait complete. 2023-11-28T11:08:34.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.082+0000 7f8e2e4bb700 1 Processor -- start 2023-11-28T11:08:34.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.082+0000 7f8e2e4bb700 1 -- start start 2023-11-28T11:08:34.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e2e4bb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2806dee0 0x7f8e280b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e2e4bb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e28071040 0x7f8e2814e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e2e4bb700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e28160540 0x7f8e280b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e2e4bb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2814ea50 con 0x7f8e2806dee0 2023-11-28T11:08:34.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e2e4bb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280b43b0 con 0x7f8e28071040 2023-11-28T11:08:34.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e2e4bb700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280b37a0 con 0x7f8e28160540 2023-11-28T11:08:34.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2806dee0 0x7f8e280b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2806dee0 0x7f8e280b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60670/0 (socket says 172.21.15.156:60670) 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e27fff700 1 -- 172.21.15.156:0/2601361375 learned_addr learned my addr 172.21.15.156:0/2601361375 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e2ca58700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e28160540 0x7f8e280b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e277fe700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e28071040 0x7f8e2814e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e27fff700 1 -- 172.21.15.156:0/2601361375 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e28160540 msgr2=0x7f8e280b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e27fff700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e28160540 0x7f8e280b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e27fff700 1 -- 172.21.15.156:0/2601361375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e28071040 msgr2=0x7f8e2814e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e27fff700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e28071040 0x7f8e2814e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e27fff700 1 -- 172.21.15.156:0/2601361375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c0094a0 con 0x7f8e2806dee0 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e277fe700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e28071040 0x7f8e2814e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e2ca58700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e28160540 0x7f8e280b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e27fff700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2806dee0 0x7f8e280b4840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800b4a0 tx=0x7f8e1800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.083+0000 7f8e24ff9700 1 -- 172.21.15.156:0/2601361375 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e18011830 con 0x7f8e2806dee0 2023-11-28T11:08:34.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.084+0000 7f8e24ff9700 1 -- 172.21.15.156:0/2601361375 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e18011990 con 0x7f8e2806dee0 2023-11-28T11:08:34.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.084+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/2601361375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2815ccd0 con 0x7f8e2806dee0 2023-11-28T11:08:34.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.084+0000 7f8e24ff9700 1 -- 172.21.15.156:0/2601361375 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e18011d40 con 0x7f8e2806dee0 2023-11-28T11:08:34.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.084+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/2601361375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2806e870 con 0x7f8e2806dee0 2023-11-28T11:08:34.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.085+0000 7f8e24ff9700 1 -- 172.21.15.156:0/2601361375 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8e1800f050 con 0x7f8e2806dee0 2023-11-28T11:08:34.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.085+0000 7f8e24ff9700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8e140682f0 0x7f8e1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.086+0000 7f8e277fe700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8e140682f0 0x7f8e1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.086+0000 7f8e24ff9700 1 -- 172.21.15.156:0/2601361375 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8e18085960 con 0x7f8e2806dee0 2023-11-28T11:08:34.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.086+0000 7f8e277fe700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8e140682f0 0x7f8e1406a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8e280b4fe0 tx=0x7f8e1c013040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.087+0000 7f8e3a644b80 1 Processor -- start 2023-11-28T11:08:34.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.087+0000 7f8e3a644b80 1 -- start start 2023-11-28T11:08:34.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e3a644b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0910810 0x5592c08e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e3a644b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c090e130 0x5592c0910be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e3a644b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0911120 0x5592c092c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e3a644b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c07d0bc0 con 0x5592c0910810 2023-11-28T11:08:34.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e3a644b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c083f350 con 0x5592c090e130 2023-11-28T11:08:34.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e3a644b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592c0838880 con 0x5592c0911120 2023-11-28T11:08:34.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0910810 0x5592c08e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0910810 0x5592c08e6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60678/0 (socket says 172.21.15.156:60678) 2023-11-28T11:08:34.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e27fff700 1 -- 172.21.15.156:0/1378486576 learned_addr learned my addr 172.21.15.156:0/1378486576 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:34.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e27fff700 1 -- 172.21.15.156:0/1378486576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0911120 msgr2=0x5592c092c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:34.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e27fff700 1 --2- 172.21.15.156:0/1378486576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0911120 0x5592c092c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e27fff700 1 -- 172.21.15.156:0/1378486576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c090e130 msgr2=0x5592c0910be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:34.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.088+0000 7f8e277fe700 1 --2- 172.21.15.156:0/1378486576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c090e130 0x5592c0910be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e27fff700 1 --2- 172.21.15.156:0/1378486576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c090e130 0x5592c0910be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e27fff700 1 -- 172.21.15.156:0/1378486576 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592c06ba5e0 con 0x5592c0910810 2023-11-28T11:08:34.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e27fff700 1 --2- 172.21.15.156:0/1378486576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0910810 0x5592c08e6a20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8e1805df30 tx=0x7f8e1805df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7478e1a98a1956 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e0dffb700 1 -- 172.21.15.156:0/1378486576 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e18055420 con 0x5592c0910810 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e0dffb700 1 -- 172.21.15.156:0/1378486576 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e18054190 con 0x5592c0910810 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e0dffb700 1 -- 172.21.15.156:0/1378486576 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1805dc40 con 0x5592c0910810 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e3a644b80 1 -- 172.21.15.156:0/1378486576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0910810 msgr2=0x5592c08e6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/1378486576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0910810 0x5592c08e6a20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8e1805df30 tx=0x7f8e1805df60 comp rx=0 tx=0).stop 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e3a644b80 1 -- 172.21.15.156:0/1378486576 shutdown_connections 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/1378486576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0911120 0x5592c092c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/1378486576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0910810 0x5592c08e6a20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/1378486576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c090e130 0x5592c0910be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e3a644b80 1 -- 172.21.15.156:0/1378486576 >> 172.21.15.156:0/1378486576 conn(0x5592c08e3d60 msgr2=0x5592c08e46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.089+0000 7f8e3a644b80 1 -- 172.21.15.156:0/1378486576 shutdown_connections 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.090+0000 7f8e3a644b80 1 -- 172.21.15.156:0/1378486576 wait complete. 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.090+0000 7f8e3a644b80 1 -- start start 2023-11-28T11:08:34.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.090+0000 7f8e3a644b80 1 Processor -- start 2023-11-28T11:08:34.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e3a644b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 0x5592c09088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e3a644b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 0x5592c090f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e3a644b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c0911120 0x5592c090fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e3a644b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c06bc8d0 con 0x5592c090e130 2023-11-28T11:08:34.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e3a644b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c0932490 con 0x5592c0911120 2023-11-28T11:08:34.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e3a644b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592c0924960 con 0x5592c0910810 2023-11-28T11:08:34.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e3a644b80 10 client.0 _ll_register_callbacks cb 0x5592c06de240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:08:34.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 0x5592c09088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 0x5592c09088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60684/0 (socket says 172.21.15.156:60684) 2023-11-28T11:08:34.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e27fff700 1 -- 172.21.15.156:0/4008012429 learned_addr learned my addr 172.21.15.156:0/4008012429 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:34.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e277fe700 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 0x5592c090f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e2ca58700 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c0911120 0x5592c090fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e27fff700 1 -- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 msgr2=0x5592c090f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:34.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e27fff700 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 0x5592c090f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e27fff700 1 -- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c0911120 msgr2=0x5592c090fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:34.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e27fff700 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c0911120 0x5592c090fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e27fff700 1 -- 172.21.15.156:0/4008012429 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2806e870 con 0x5592c090e130 2023-11-28T11:08:34.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.091+0000 7f8e2ca58700 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c0911120 0x5592c090fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:34.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e27fff700 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 0x5592c09088b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800bf40 tx=0x7f8e1800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eeb40cf722ca480 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:08:34.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e06ffd700 1 -- 172.21.15.156:0/4008012429 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e18069120 con 0x5592c090e130 2023-11-28T11:08:34.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e06ffd700 1 -- 172.21.15.156:0/4008012429 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8e18059040 con 0x5592c090e130 2023-11-28T11:08:34.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e3a644b80 1 -- 172.21.15.156:0/4008012429 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592c0806140 con 0x5592c090e130 2023-11-28T11:08:34.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e06ffd700 1 -- 172.21.15.156:0/4008012429 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e18069280 con 0x5592c090e130 2023-11-28T11:08:34.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e3a644b80 10 client.4635 fetch_fsmap learned FSMap version 8 2023-11-28T11:08:34.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e3a644b80 1 -- 172.21.15.156:0/4008012429 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5592c0666d90 con 0x5592c090e130 2023-11-28T11:08:34.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e06ffd700 1 -- 172.21.15.156:0/4008012429 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5592c0806140 con 0x5592c090e130 2023-11-28T11:08:34.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e06ffd700 1 -- 172.21.15.156:0/4008012429 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8e18051b20 con 0x5592c090e130 2023-11-28T11:08:34.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e3a644b80 10 client.4635 fetch_fsmap finished waiting for FSMap version 8 2023-11-28T11:08:34.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e3a644b80 10 client.4635 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T11:08:34.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e3a644b80 20 client.4635 populate_metadata read hostname 'smithi156' 2023-11-28T11:08:34.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:6833/4134790300,v1:172.21.15.186:6835/4134790300] conn(0x5592c093eac0 0x5592c093ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e3a644b80 4 client.4635 mds_command: new command op to 4287 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:08:34.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e3a644b80 1 -- 172.21.15.156:0/4008012429 --> [v2:172.21.15.186:6833/4134790300,v1:172.21.15.186:6835/4134790300] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592c06bcf10 con 0x5592c093eac0 2023-11-28T11:08:34.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.092+0000 7f8e277fe700 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:6833/4134790300,v1:172.21.15.186:6835/4134790300] conn(0x5592c093eac0 0x5592c093ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.093+0000 7f8e06ffd700 10 client.4635 ms_handle_connect on v2:172.21.15.186:6833/4134790300 2023-11-28T11:08:34.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.093+0000 7f8e277fe700 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:6833/4134790300,v1:172.21.15.186:6835/4134790300] conn(0x5592c093eac0 0x5592c093ee90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4206ec7f2106a409 server_cookie=e473299f779a6313 in_seq=0 out_seq=0 2023-11-28T11:08:34.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.093+0000 7f8e06ffd700 1 -- 172.21.15.156:0/4008012429 <== mds.0 v2:172.21.15.186:6833/4134790300 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5592c06bcf10 con 0x5592c093eac0 2023-11-28T11:08:34.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.093+0000 7f8e06ffd700 10 client.4635 handle_command_reply: tid=0 2023-11-28T11:08:34.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.093+0000 7f8e3a644b80 1 client.4635 shutdown 2023-11-28T11:08:34.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.094+0000 7f8e3a644b80 1 -- 172.21.15.156:0/4008012429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 msgr2=0x5592c09088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.094+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 0x5592c09088b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800bf40 tx=0x7f8e1800bed0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.094+0000 7f8e3a644b80 1 -- 172.21.15.156:0/4008012429 shutdown_connections 2023-11-28T11:08:34.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.094+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c0911120 0x5592c090fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.094+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:6833/4134790300,v1:172.21.15.186:6835/4134790300] conn(0x5592c093eac0 0x5592c093ee90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.094+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 0x5592c09088b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.094+0000 7f8e06ffd700 0 client.4635 ms_handle_reset on v2:172.21.15.186:6833/4134790300 2023-11-28T11:08:34.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.094+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/4008012429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 0x5592c090f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.094+0000 7f8e3a644b80 1 -- 172.21.15.156:0/4008012429 >> 172.21.15.156:0/4008012429 conn(0x5592c08e3d60 msgr2=0x5592c08e6990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:34.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.094+0000 7f8e3a644b80 1 -- 172.21.15.156:0/4008012429 shutdown_connections 2023-11-28T11:08:34.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.094+0000 7f8e3a644b80 1 -- 172.21.15.156:0/4008012429 wait complete. 2023-11-28T11:08:34.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.094+0000 7f8e3a644b80 20 client.4635 trim_cache size 0 max 16384 2023-11-28T11:08:34.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.111+0000 7f8e3a644b80 1 Processor -- start 2023-11-28T11:08:34.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.111+0000 7f8e3a644b80 1 -- start start 2023-11-28T11:08:34.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e3a644b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 0x5592c098a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e3a644b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 0x5592c08df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e3a644b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c0911120 0x5592c08df990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e3a644b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c0907c00 con 0x5592c090e130 2023-11-28T11:08:34.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e3a644b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c08e5d70 con 0x5592c0911120 2023-11-28T11:08:34.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e3a644b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592c0908860 con 0x5592c0910810 2023-11-28T11:08:34.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 0x5592c098a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 0x5592c098a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60692/0 (socket says 172.21.15.156:60692) 2023-11-28T11:08:34.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e27fff700 1 -- 172.21.15.156:0/3718153543 learned_addr learned my addr 172.21.15.156:0/3718153543 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:34.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e27fff700 1 -- 172.21.15.156:0/3718153543 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 msgr2=0x5592c08df450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:34.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e27fff700 1 --2- 172.21.15.156:0/3718153543 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 0x5592c08df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e27fff700 1 -- 172.21.15.156:0/3718153543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c0911120 msgr2=0x5592c08df990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:34.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e27fff700 1 --2- 172.21.15.156:0/3718153543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c0911120 0x5592c08df990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.112+0000 7f8e27fff700 1 -- 172.21.15.156:0/3718153543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592c07ff140 con 0x5592c090e130 2023-11-28T11:08:34.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e27fff700 1 --2- 172.21.15.156:0/3718153543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 0x5592c098a610 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8e1805df50 tx=0x7f8e18082050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d232ded44abe8f07 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e0e7fc700 1 -- 172.21.15.156:0/3718153543 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e18053ef0 con 0x5592c090e130 2023-11-28T11:08:34.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e0e7fc700 1 -- 172.21.15.156:0/3718153543 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1800d040 con 0x5592c090e130 2023-11-28T11:08:34.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e0e7fc700 1 -- 172.21.15.156:0/3718153543 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e18083ec0 con 0x5592c090e130 2023-11-28T11:08:34.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3718153543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 msgr2=0x5592c098a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/3718153543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 0x5592c098a610 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8e1805df50 tx=0x7f8e18082050 comp rx=0 tx=0).stop 2023-11-28T11:08:34.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3718153543 shutdown_connections 2023-11-28T11:08:34.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/3718153543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c0911120 0x5592c08df990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/3718153543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c090e130 0x5592c098a610 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/3718153543 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 0x5592c08df450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3718153543 >> 172.21.15.156:0/3718153543 conn(0x5592c08e3d60 msgr2=0x5592c09091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:34.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3718153543 shutdown_connections 2023-11-28T11:08:34.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.113+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3718153543 wait complete. 2023-11-28T11:08:34.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.114+0000 7f8e3a644b80 1 -- start start 2023-11-28T11:08:34.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.114+0000 7f8e3a644b80 1 Processor -- start 2023-11-28T11:08:34.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e3a644b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c090e130 0x5592c0932410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e3a644b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 0x5592c0932950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e3a644b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0911120 0x5592c092e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e3a644b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c09899d0 con 0x5592c0911120 2023-11-28T11:08:34.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e3a644b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c0933610 con 0x5592c090e130 2023-11-28T11:08:34.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e3a644b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592c0949190 con 0x5592c0910810 2023-11-28T11:08:34.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e3a644b80 10 client.0 _ll_register_callbacks cb 0x7f8e18055720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:08:34.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e2ca58700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0911120 0x5592c092e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e2ca58700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0911120 0x5592c092e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60702/0 (socket says 172.21.15.156:60702) 2023-11-28T11:08:34.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e2ca58700 1 -- 172.21.15.156:0/3210572876 learned_addr learned my addr 172.21.15.156:0/3210572876 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:34.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e277fe700 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 0x5592c0932950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e2ca58700 1 -- 172.21.15.156:0/3210572876 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 msgr2=0x5592c0932950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:34.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e2ca58700 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 0x5592c0932950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e2ca58700 1 -- 172.21.15.156:0/3210572876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c090e130 msgr2=0x5592c0932410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:34.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e27fff700 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c090e130 0x5592c0932410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e2ca58700 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c090e130 0x5592c0932410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e2ca58700 1 -- 172.21.15.156:0/3210572876 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592c0666d90 con 0x5592c0911120 2023-11-28T11:08:34.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e2ca58700 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0911120 0x5592c092e850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000f540 tx=0x7f8e2000f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36296a2fd94ffcbb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:08:34.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e06ffd700 1 -- 172.21.15.156:0/3210572876 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e200072e0 con 0x5592c0911120 2023-11-28T11:08:34.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e06ffd700 1 -- 172.21.15.156:0/3210572876 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8e20007440 con 0x5592c0911120 2023-11-28T11:08:34.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.115+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3210572876 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592c07e0170 con 0x5592c0911120 2023-11-28T11:08:34.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e06ffd700 1 -- 172.21.15.156:0/3210572876 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e20001930 con 0x5592c0911120 2023-11-28T11:08:34.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e3a644b80 10 client.4641 fetch_fsmap learned FSMap version 8 2023-11-28T11:08:34.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3210572876 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5592c068d200 con 0x5592c0911120 2023-11-28T11:08:34.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e06ffd700 1 -- 172.21.15.156:0/3210572876 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5592c07e0170 con 0x5592c0911120 2023-11-28T11:08:34.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e06ffd700 1 -- 172.21.15.156:0/3210572876 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8e20012070 con 0x5592c0911120 2023-11-28T11:08:34.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e3a644b80 10 client.4641 fetch_fsmap finished waiting for FSMap version 8 2023-11-28T11:08:34.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e3a644b80 10 client.4641 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T11:08:34.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e3a644b80 20 client.4641 populate_metadata read hostname 'smithi156' 2023-11-28T11:08:34.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.186:6833/4134790300,v1:172.21.15.186:6835/4134790300] conn(0x5592c0930e00 0x5592c09311d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e3a644b80 4 client.4641 mds_command: new command op to 4287 tid=0[{"prefix": "status"}] 2023-11-28T11:08:34.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3210572876 --> [v2:172.21.15.186:6833/4134790300,v1:172.21.15.186:6835/4134790300] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5592c0807550 con 0x5592c0930e00 2023-11-28T11:08:34.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e27fff700 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.186:6833/4134790300,v1:172.21.15.186:6835/4134790300] conn(0x5592c0930e00 0x5592c09311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e06ffd700 10 client.4641 ms_handle_connect on v2:172.21.15.186:6833/4134790300 2023-11-28T11:08:34.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.116+0000 7f8e27fff700 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.186:6833/4134790300,v1:172.21.15.186:6835/4134790300] conn(0x5592c0930e00 0x5592c09311d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=972750104f57fcdc server_cookie=972a5938dde85c14 in_seq=0 out_seq=0 2023-11-28T11:08:34.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e06ffd700 1 -- 172.21.15.156:0/3210572876 <== mds.0 v2:172.21.15.186:6833/4134790300 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5592c0807550 con 0x5592c0930e00 2023-11-28T11:08:34.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e06ffd700 10 client.4641 handle_command_reply: tid=0 2023-11-28T11:08:34.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e3a644b80 1 client.4641 shutdown 2023-11-28T11:08:34.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3210572876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0911120 msgr2=0x5592c092e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0911120 0x5592c092e850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000f540 tx=0x7f8e2000f900 comp rx=0 tx=0).stop 2023-11-28T11:08:34.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3210572876 shutdown_connections 2023-11-28T11:08:34.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.186:6833/4134790300,v1:172.21.15.186:6835/4134790300] conn(0x5592c0930e00 0x5592c09311d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592c0911120 0x5592c092e850 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e06ffd700 0 client.4641 ms_handle_reset on v2:172.21.15.186:6833/4134790300 2023-11-28T11:08:34.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592c090e130 0x5592c0932410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e3a644b80 1 --2- 172.21.15.156:0/3210572876 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592c0910810 0x5592c0932950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3210572876 >> 172.21.15.156:0/3210572876 conn(0x5592c08e3d60 msgr2=0x5592c08df450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:34.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3210572876 shutdown_connections 2023-11-28T11:08:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e3a644b80 1 -- 172.21.15.156:0/3210572876 wait complete. 2023-11-28T11:08:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e3a644b80 20 client.4641 trim_cache size 0 max 16384 2023-11-28T11:08:34.128 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:08:34.128 INFO:teuthology.orchestra.run.smithi156.stdout: "cluster_fsid": "351da896-8dbf-473b-9472-c3454d9368bf", 2023-11-28T11:08:34.128 INFO:teuthology.orchestra.run.smithi156.stdout: "whoami": 0, 2023-11-28T11:08:34.128 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 4287, 2023-11-28T11:08:34.128 INFO:teuthology.orchestra.run.smithi156.stdout: "want_state": "up:active", 2023-11-28T11:08:34.128 INFO:teuthology.orchestra.run.smithi156.stdout: "state": "up:active", 2023-11-28T11:08:34.128 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs", 2023-11-28T11:08:34.128 INFO:teuthology.orchestra.run.smithi156.stdout: "rank_uptime": 16.764796173000001, 2023-11-28T11:08:34.129 INFO:teuthology.orchestra.run.smithi156.stdout: "mdsmap_epoch": 8, 2023-11-28T11:08:34.129 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch": 23, 2023-11-28T11:08:34.129 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch_barrier": 23, 2023-11-28T11:08:34.129 INFO:teuthology.orchestra.run.smithi156.stdout: "uptime": 22.055464157999999 2023-11-28T11:08:34.129 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:08:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/2601361375 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8e140682f0 msgr2=0x7f8e1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e2e4bb700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8e140682f0 0x7f8e1406a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8e280b4fe0 tx=0x7f8e1c013040 comp rx=0 tx=0).stop 2023-11-28T11:08:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/2601361375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2806dee0 msgr2=0x7f8e280b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e2e4bb700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2806dee0 0x7f8e280b4840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800b4a0 tx=0x7f8e1800b860 comp rx=0 tx=0).stop 2023-11-28T11:08:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/2601361375 shutdown_connections 2023-11-28T11:08:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e2e4bb700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8e140682f0 0x7f8e1406a7a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e2e4bb700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e28160540 0x7f8e280b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e2e4bb700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2806dee0 0x7f8e280b4840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e2e4bb700 1 --2- 172.21.15.156:0/2601361375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e28071040 0x7f8e2814e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.117+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/2601361375 >> 172.21.15.156:0/2601361375 conn(0x7f8e280d0000 msgr2=0x7f8e280cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.118+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/2601361375 shutdown_connections 2023-11-28T11:08:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.118+0000 7f8e2e4bb700 1 -- 172.21.15.156:0/2601361375 wait complete. 2023-11-28T11:08:34.132 INFO:teuthology.run_tasks:Running task kclient... 2023-11-28T11:08:34.154 INFO:tasks.kclient:Mounting kernel clients... 2023-11-28T11:08:34.154 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-28T11:08:34.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:34.155 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T11:08:34.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:34.174 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-11-28T11:08:34.242 INFO:teuthology.orchestra.run.smithi156.stderr:Cannot find device "ceph-brx" 2023-11-28T11:08:34.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:08:34.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:34.244 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T11:08:34.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:34.302 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-11-28T11:08:34.369 INFO:teuthology.orchestra.run.smithi156.stderr:Cannot find device "ceph-brx" 2023-11-28T11:08:34.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:08:34.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:34.371 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-11-28T11:08:34.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:34.392 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2023-11-28T11:08:34.467 INFO:teuthology.orchestra.run.smithi186.stderr:Cannot find device "ceph-brx" 2023-11-28T11:08:34.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:08:34.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:34.469 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-11-28T11:08:34.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:34.529 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2023-11-28T11:08:34.600 INFO:teuthology.orchestra.run.smithi186.stderr:Cannot find device "ceph-brx" 2023-11-28T11:08:34.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:08:34.601 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-11-28T11:08:34.601 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:08:34.602 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:08:34.602 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:08:34.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.721+0000 7f81922fc700 1 Processor -- start 2023-11-28T11:08:34.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.721+0000 7f81922fc700 1 -- start start 2023-11-28T11:08:34.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.721+0000 7f81922fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f818c071040 0x7f818c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f81922fc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f818c06dee0 0x7f818c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f81922fc700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c071950 con 0x7f818c06cdc0 2023-11-28T11:08:34.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f81922fc700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c161550 con 0x7f818c071040 2023-11-28T11:08:34.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f81922fc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c15d060 con 0x7f818c06dee0 2023-11-28T11:08:34.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f818bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f818c071040 0x7f818c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f818bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f818c071040 0x7f818c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60706/0 (socket says 172.21.15.156:60706) 2023-11-28T11:08:34.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f818bfff700 1 -- 172.21.15.156:0/2270732354 learned_addr learned my addr 172.21.15.156:0/2270732354 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:34.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f818bfff700 1 -- 172.21.15.156:0/2270732354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f818c06dee0 msgr2=0x7f818c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:34.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f818bfff700 1 --2- 172.21.15.156:0/2270732354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f818c06dee0 0x7f818c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f818bfff700 1 -- 172.21.15.156:0/2270732354 >> v1:172.21.15.186:6789/0 conn(0x7f818c06cdc0 legacy=0x7f818c06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:34.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f818bfff700 1 -- 172.21.15.156:0/2270732354 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c15d340 con 0x7f818c071040 2023-11-28T11:08:34.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f818bfff700 1 --2- 172.21.15.156:0/2270732354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818c071040 0x7f818c071410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f817c0098f0 tx=0x7f817c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2e6385385789a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f818affd700 1 -- 172.21.15.156:0/2270732354 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c004230 con 0x7f818c071040 2023-11-28T11:08:34.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f818affd700 1 -- 172.21.15.156:0/2270732354 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c004390 con 0x7f818c071040 2023-11-28T11:08:34.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.722+0000 7f818affd700 1 -- 172.21.15.156:0/2270732354 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c0036a0 con 0x7f818c071040 2023-11-28T11:08:34.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.723+0000 7f81922fc700 1 -- 172.21.15.156:0/2270732354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818c071040 msgr2=0x7f818c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.723+0000 7f81922fc700 1 --2- 172.21.15.156:0/2270732354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818c071040 0x7f818c071410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f817c0098f0 tx=0x7f817c009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:34.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.723+0000 7f81922fc700 1 -- 172.21.15.156:0/2270732354 shutdown_connections 2023-11-28T11:08:34.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.723+0000 7f81922fc700 1 --2- 172.21.15.156:0/2270732354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818c071040 0x7f818c071410 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.723+0000 7f81922fc700 1 --2- 172.21.15.156:0/2270732354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f818c06dee0 0x7f818c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.723+0000 7f81922fc700 1 -- 172.21.15.156:0/2270732354 >> 172.21.15.156:0/2270732354 conn(0x7f818c0d0000 msgr2=0x7f818c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:34.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.723+0000 7f81922fc700 1 -- 172.21.15.156:0/2270732354 shutdown_connections 2023-11-28T11:08:34.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.723+0000 7f81922fc700 1 -- 172.21.15.156:0/2270732354 wait complete. 2023-11-28T11:08:34.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.724+0000 7f81922fc700 1 Processor -- start 2023-11-28T11:08:34.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.724+0000 7f81922fc700 1 -- start start 2023-11-28T11:08:34.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f81922fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f818c06cdc0 0x7f818c154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f81922fc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f818c06dee0 0x7f818c1e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f81922fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818c071040 0x7f818c1e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f81922fc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0b46a0 con 0x7f818c071040 2023-11-28T11:08:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f81922fc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0b4b30 con 0x7f818c06cdc0 2023-11-28T11:08:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f81922fc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0b37f0 con 0x7f818c06dee0 2023-11-28T11:08:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f8190899700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818c071040 0x7f818c1e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f8190899700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818c071040 0x7f818c1e68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60716/0 (socket says 172.21.15.156:60716) 2023-11-28T11:08:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f818bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f818c06cdc0 0x7f818c154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f818b7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f818c06dee0 0x7f818c1e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f8190899700 1 -- 172.21.15.156:0/3642218769 learned_addr learned my addr 172.21.15.156:0/3642218769 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f8190899700 1 -- 172.21.15.156:0/3642218769 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f818c06dee0 msgr2=0x7f818c1e63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f8190899700 1 --2- 172.21.15.156:0/3642218769 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f818c06dee0 0x7f818c1e63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f8190899700 1 -- 172.21.15.156:0/3642218769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f818c06cdc0 msgr2=0x7f818c154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f8190899700 1 --2- 172.21.15.156:0/3642218769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f818c06cdc0 0x7f818c154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.725+0000 7f8190899700 1 -- 172.21.15.156:0/3642218769 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c009550 con 0x7f818c071040 2023-11-28T11:08:34.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.726+0000 7f8190899700 1 --2- 172.21.15.156:0/3642218769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818c071040 0x7f818c1e68e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f818400b970 tx=0x7f818400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.726+0000 7f8188ff9700 1 -- 172.21.15.156:0/3642218769 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818400d860 con 0x7f818c071040 2023-11-28T11:08:34.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.726+0000 7f8188ff9700 1 -- 172.21.15.156:0/3642218769 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818400d9c0 con 0x7f818c071040 2023-11-28T11:08:34.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.726+0000 7f81922fc700 1 -- 172.21.15.156:0/3642218769 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c15d900 con 0x7f818c071040 2023-11-28T11:08:34.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.726+0000 7f8188ff9700 1 -- 172.21.15.156:0/3642218769 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818400dd70 con 0x7f818c071040 2023-11-28T11:08:34.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.726+0000 7f81922fc700 1 -- 172.21.15.156:0/3642218769 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c15d620 con 0x7f818c071040 2023-11-28T11:08:34.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.727+0000 7f81922fc700 1 -- 172.21.15.156:0/3642218769 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818c06e870 con 0x7f818c071040 2023-11-28T11:08:34.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.727+0000 7f8188ff9700 1 -- 172.21.15.156:0/3642218769 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f818400e430 con 0x7f818c071040 2023-11-28T11:08:34.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.727+0000 7f8188ff9700 1 --2- 172.21.15.156:0/3642218769 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f81780682f0 0x7f817806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:34.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.728+0000 7f8188ff9700 1 -- 172.21.15.156:0/3642218769 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8184085f40 con 0x7f818c071040 2023-11-28T11:08:34.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.728+0000 7f818bfff700 1 --2- 172.21.15.156:0/3642218769 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f81780682f0 0x7f817806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:34.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.728+0000 7f818bfff700 1 --2- 172.21.15.156:0/3642218769 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f81780682f0 0x7f817806a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f817c009130 tx=0x7f817c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:34.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.732+0000 7f8188ff9700 1 -- 172.21.15.156:0/3642218769 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8184052740 con 0x7f818c071040 2023-11-28T11:08:34.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.959+0000 7f81922fc700 1 -- 172.21.15.156:0/3642218769 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f818c0b47e0 con 0x7f818c071040 2023-11-28T11:08:34.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.959+0000 7f8188ff9700 1 -- 172.21.15.156:0/3642218769 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8184055d60 con 0x7f818c071040 2023-11-28T11:08:34.962 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:08:34.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.961+0000 7f81922fc700 1 -- 172.21.15.156:0/3642218769 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f81780682f0 msgr2=0x7f817806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.961+0000 7f81922fc700 1 --2- 172.21.15.156:0/3642218769 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f81780682f0 0x7f817806a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f817c009130 tx=0x7f817c008030 comp rx=0 tx=0).stop 2023-11-28T11:08:34.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.961+0000 7f81922fc700 1 -- 172.21.15.156:0/3642218769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818c071040 msgr2=0x7f818c1e68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:34.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.961+0000 7f81922fc700 1 --2- 172.21.15.156:0/3642218769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818c071040 0x7f818c1e68e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f818400b970 tx=0x7f818400bc80 comp rx=0 tx=0).stop 2023-11-28T11:08:34.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.961+0000 7f81922fc700 1 -- 172.21.15.156:0/3642218769 shutdown_connections 2023-11-28T11:08:34.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.961+0000 7f81922fc700 1 --2- 172.21.15.156:0/3642218769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818c071040 0x7f818c1e68e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.961+0000 7f81922fc700 1 --2- 172.21.15.156:0/3642218769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f818c06cdc0 0x7f818c154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.961+0000 7f81922fc700 1 --2- 172.21.15.156:0/3642218769 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f81780682f0 0x7f817806a7a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.961+0000 7f81922fc700 1 --2- 172.21.15.156:0/3642218769 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f818c06dee0 0x7f818c1e63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:34.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.961+0000 7f81922fc700 1 -- 172.21.15.156:0/3642218769 >> 172.21.15.156:0/3642218769 conn(0x7f818c0d0000 msgr2=0x7f818c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:34.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.962+0000 7f81922fc700 1 -- 172.21.15.156:0/3642218769 shutdown_connections 2023-11-28T11:08:34.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:34.962+0000 7f81922fc700 1 -- 172.21.15.156:0/3642218769 wait complete. 2023-11-28T11:08:34.976 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:08:34.977 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:08:34.977 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-28T11:08:34.977 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:08:34.977 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:08:34.977 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:08:34.977 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:08:34.977 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:08:34.977 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:08:34.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:34.978 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-28T11:08:34.995 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:08:34.995 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:08:34.995 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:08:34.995 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:34.995 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T11:08:34.995 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:34.996 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:34.996 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:34.996 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:34.996 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:08:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:08:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42368sec preferred_lft 42368sec 2023-11-28T11:08:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::c260:d776:a692:b0cc/64 scope link noprefixroute 2023-11-28T11:08:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:34.997 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:08:34.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:34.997 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T11:08:34.997 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T11:08:34.997 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:08:34.997 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip addr flush dev ceph-brx 2023-11-28T11:08:34.997 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set ceph-brx up 2023-11-28T11:08:34.997 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:08:34.997 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T11:08:35.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:35.165 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:08:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-28T11:08:35.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:35.195 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-28T11:08:35.264 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-28T11:08:35.265 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:08:35.265 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:08:35.265 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:08:35.265 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:08:35.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:35.265 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T11:08:35.266 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T11:08:35.266 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:08:35.266 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:08:35.266 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:08:35.266 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T11:08:35.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:35.501 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T11:08:35.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:35.519 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list-id 2023-11-28T11:08:35.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:35.576 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T11:08:35.576 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T11:08:35.576 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:08:35.576 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T11:08:35.576 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T11:08:35.671 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T11:08:35.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:35.671 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T11:08:35.671 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T11:08:35.671 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T11:08:35.671 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:08:35.671 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T11:08:35.672 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T11:08:35.672 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T11:08:35.672 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T11:08:35.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:35.930 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T11:08:35.930 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T11:08:35.930 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-28T11:08:35.930 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:08:35.930 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T11:08:36.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:08:36.031 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:08:36.052 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:08:36.052 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:08:36.052 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:08:36.109 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:08:36.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:36.109 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:08:36.207 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:08:36.207 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:08:36.208 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:08:36.304 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:08:36.304 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:08:36.304 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:08:36.304 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:08:36.304 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:08:36.330 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:08:36.330 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:08:36.330 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:08:36.330 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:08:36.331 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:08:36.331 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T11:08:36.331 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T11:08:36.331 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=351da896-8dbf-473b-9472-c3454d9368bf" will pass to kernel 2023-11-28T11:08:36.334 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:08:36.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:36.334 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:08:36.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:36.365 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:08:36.450 INFO:teuthology.orchestra.run.smithi156.stdout:client4549 2023-11-28T11:08:36.451 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:08:36.451 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:36.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:36.469 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:08:36.554 INFO:teuthology.orchestra.run.smithi156.stdout:client4549 2023-11-28T11:08:36.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:36.555 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:08:36.600 INFO:teuthology.orchestra.run.smithi156.stdout:client4549 2023-11-28T11:08:36.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:36.601 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/351da896-8dbf-473b-9472-c3454d9368bf.client4549/status') 2023-11-28T11:08:36.691 INFO:teuthology.orchestra.run.smithi156.stdout:instance: client.4549 (3)172.21.15.156:0/1838085959 2023-11-28T11:08:36.691 INFO:teuthology.orchestra.run.smithi156.stdout:blocklisted: false 2023-11-28T11:08:36.692 INFO:teuthology.orchestra.run.smithi156.stderr:0+1 records in 2023-11-28T11:08:36.693 INFO:teuthology.orchestra.run.smithi156.stderr:0+1 records out 2023-11-28T11:08:36.693 INFO:teuthology.orchestra.run.smithi156.stderr:71 bytes copied, 2.1508e-05 s, 3.3 MB/s 2023-11-28T11:08:36.693 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-11-28T11:08:36.693 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:08:36.694 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T11:08:36.694 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:08:36.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.814+0000 7ffb85da2700 1 Processor -- start 2023-11-28T11:08:36.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.814+0000 7ffb85da2700 1 -- start start 2023-11-28T11:08:36.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.815+0000 7ffb85da2700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ffb8006a980 0x7ffb8006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.815+0000 7ffb85da2700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb800b4960 0x7ffb800b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.815+0000 7ffb85da2700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8006d170 con 0x7ffb800af0c0 2023-11-28T11:08:36.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.815+0000 7ffb85da2700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8015d900 con 0x7ffb8006a980 2023-11-28T11:08:36.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.815+0000 7ffb85da2700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb80071cb0 con 0x7ffb800b4960 2023-11-28T11:08:36.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.815+0000 7ffb7ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ffb8006a980 0x7ffb8006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.815+0000 7ffb7ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ffb8006a980 0x7ffb8006cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60724/0 (socket says 172.21.15.156:60724) 2023-11-28T11:08:36.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.815+0000 7ffb7ffff700 1 -- 172.21.15.156:0/2642769702 learned_addr learned my addr 172.21.15.156:0/2642769702 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:36.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb855a1700 1 --2- 172.21.15.156:0/2642769702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb800b4960 0x7ffb800b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb7ffff700 1 -- 172.21.15.156:0/2642769702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb800b4960 msgr2=0x7ffb800b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:36.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb7ffff700 1 --2- 172.21.15.156:0/2642769702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb800b4960 0x7ffb800b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb7ffff700 1 -- 172.21.15.156:0/2642769702 >> v1:172.21.15.186:6789/0 conn(0x7ffb800af0c0 legacy=0x7ffb8006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:36.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb7ffff700 1 -- 172.21.15.156:0/2642769702 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8015dd20 con 0x7ffb8006a980 2023-11-28T11:08:36.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb7ffff700 1 --2- 172.21.15.156:0/2642769702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb8006a980 0x7ffb8006cc30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0098f0 tx=0x7ffb6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5c63994daf2753 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:36.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb7f7fe700 1 -- 172.21.15.156:0/2642769702 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c004230 con 0x7ffb8006a980 2023-11-28T11:08:36.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb7f7fe700 1 -- 172.21.15.156:0/2642769702 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c004390 con 0x7ffb8006a980 2023-11-28T11:08:36.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb7f7fe700 1 -- 172.21.15.156:0/2642769702 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c0036a0 con 0x7ffb8006a980 2023-11-28T11:08:36.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb85da2700 1 -- 172.21.15.156:0/2642769702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb8006a980 msgr2=0x7ffb8006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:36.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb85da2700 1 --2- 172.21.15.156:0/2642769702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb8006a980 0x7ffb8006cc30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0098f0 tx=0x7ffb6c009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:36.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb85da2700 1 -- 172.21.15.156:0/2642769702 shutdown_connections 2023-11-28T11:08:36.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb85da2700 1 --2- 172.21.15.156:0/2642769702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb800b4960 0x7ffb800b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb85da2700 1 --2- 172.21.15.156:0/2642769702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb8006a980 0x7ffb8006cc30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb85da2700 1 -- 172.21.15.156:0/2642769702 >> 172.21.15.156:0/2642769702 conn(0x7ffb8006b380 msgr2=0x7ffb8006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:36.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb85da2700 1 -- 172.21.15.156:0/2642769702 shutdown_connections 2023-11-28T11:08:36.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.816+0000 7ffb85da2700 1 -- 172.21.15.156:0/2642769702 wait complete. 2023-11-28T11:08:36.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.818+0000 7ffb85da2700 1 Processor -- start 2023-11-28T11:08:36.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.818+0000 7ffb85da2700 1 -- start start 2023-11-28T11:08:36.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb85da2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb8006a980 0x7ffb80152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb85da2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb800af0c0 0x7ffb800ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb85da2700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb800b4960 0x7ffb800adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb85da2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb80152d70 con 0x7ffb8006a980 2023-11-28T11:08:36.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb85da2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800c5160 con 0x7ffb800af0c0 2023-11-28T11:08:36.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb85da2700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800cd490 con 0x7ffb800b4960 2023-11-28T11:08:36.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb84da0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb8006a980 0x7ffb80152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb84da0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb8006a980 0x7ffb80152830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60730/0 (socket says 172.21.15.156:60730) 2023-11-28T11:08:36.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb84da0700 1 -- 172.21.15.156:0/2704998625 learned_addr learned my addr 172.21.15.156:0/2704998625 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:36.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb7ffff700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb800af0c0 0x7ffb800ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb855a1700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb800b4960 0x7ffb800adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb84da0700 1 -- 172.21.15.156:0/2704998625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb800b4960 msgr2=0x7ffb800adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:36.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb84da0700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb800b4960 0x7ffb800adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb84da0700 1 -- 172.21.15.156:0/2704998625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb800af0c0 msgr2=0x7ffb800ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:36.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb84da0700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb800af0c0 0x7ffb800ad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:36.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb84da0700 1 -- 172.21.15.156:0/2704998625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c009550 con 0x7ffb8006a980 2023-11-28T11:08:36.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb84da0700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb8006a980 0x7ffb80152830 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400c500 tx=0x7ffb7400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:36.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb7d7fa700 1 -- 172.21.15.156:0/2704998625 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74011790 con 0x7ffb8006a980 2023-11-28T11:08:36.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb7d7fa700 1 -- 172.21.15.156:0/2704998625 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb740118f0 con 0x7ffb8006a980 2023-11-28T11:08:36.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb7d7fa700 1 -- 172.21.15.156:0/2704998625 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74011ca0 con 0x7ffb8006a980 2023-11-28T11:08:36.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb85da2700 1 -- 172.21.15.156:0/2704998625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8015cf60 con 0x7ffb8006a980 2023-11-28T11:08:36.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.819+0000 7ffb85da2700 1 -- 172.21.15.156:0/2704998625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8006d3a0 con 0x7ffb8006a980 2023-11-28T11:08:36.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.820+0000 7ffb85da2700 1 -- 172.21.15.156:0/2704998625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8015d240 con 0x7ffb8006a980 2023-11-28T11:08:36.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.821+0000 7ffb7d7fa700 1 -- 172.21.15.156:0/2704998625 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffb74010050 con 0x7ffb8006a980 2023-11-28T11:08:36.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.821+0000 7ffb7d7fa700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ffb640682f0 0x7ffb6406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:36.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.821+0000 7ffb7d7fa700 1 -- 172.21.15.156:0/2704998625 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb740859c0 con 0x7ffb8006a980 2023-11-28T11:08:36.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.821+0000 7ffb7ffff700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ffb640682f0 0x7ffb6406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:36.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.821+0000 7ffb7ffff700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ffb640682f0 0x7ffb6406a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0098f0 tx=0x7ffb6c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:36.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:36.825+0000 7ffb7d7fa700 1 -- 172.21.15.156:0/2704998625 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffb74052220 con 0x7ffb8006a980 2023-11-28T11:08:37.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.052+0000 7ffb85da2700 1 -- 172.21.15.156:0/2704998625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffb80061e80 con 0x7ffb8006a980 2023-11-28T11:08:37.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.053+0000 7ffb7d7fa700 1 -- 172.21.15.156:0/2704998625 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffb74055840 con 0x7ffb8006a980 2023-11-28T11:08:37.054 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:08:37.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.055+0000 7ffb85da2700 1 -- 172.21.15.156:0/2704998625 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ffb640682f0 msgr2=0x7ffb6406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:37.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.055+0000 7ffb85da2700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ffb640682f0 0x7ffb6406a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0098f0 tx=0x7ffb6c008030 comp rx=0 tx=0).stop 2023-11-28T11:08:37.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.055+0000 7ffb85da2700 1 -- 172.21.15.156:0/2704998625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb8006a980 msgr2=0x7ffb80152830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:37.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.055+0000 7ffb85da2700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb8006a980 0x7ffb80152830 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400c500 tx=0x7ffb7400c8c0 comp rx=0 tx=0).stop 2023-11-28T11:08:37.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.055+0000 7ffb85da2700 1 -- 172.21.15.156:0/2704998625 shutdown_connections 2023-11-28T11:08:37.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.055+0000 7ffb85da2700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ffb640682f0 0x7ffb6406a7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:37.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.055+0000 7ffb85da2700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb800b4960 0x7ffb800adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:37.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.055+0000 7ffb85da2700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb8006a980 0x7ffb80152830 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:37.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.055+0000 7ffb85da2700 1 --2- 172.21.15.156:0/2704998625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb800af0c0 0x7ffb800ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:37.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.055+0000 7ffb85da2700 1 -- 172.21.15.156:0/2704998625 >> 172.21.15.156:0/2704998625 conn(0x7ffb8006b380 msgr2=0x7ffb800af490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:37.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.055+0000 7ffb85da2700 1 -- 172.21.15.156:0/2704998625 shutdown_connections 2023-11-28T11:08:37.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:37.055+0000 7ffb85da2700 1 -- 172.21.15.156:0/2704998625 wait complete. 2023-11-28T11:08:37.068 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:08:37.068 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:08:37.068 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-28T11:08:37.069 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-28T11:08:37.069 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T11:08:37.069 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:08:37.069 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:08:37.069 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:08:37.070 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-28T11:08:37.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:37.071 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-28T11:08:37.089 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:08:37.089 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:08:37.089 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:08:37.089 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.090 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T11:08:37.090 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.090 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:37.091 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:37.091 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:37.092 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:37.092 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:37.092 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:37.092 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:08:37.092 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:37.092 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:08:37.092 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42366sec preferred_lft 42366sec 2023-11-28T11:08:37.092 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::c260:d776:a692:b0cc/64 scope link noprefixroute 2023-11-28T11:08:37.092 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.092 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:08:37.092 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 8e:07:21:2f:e8:e5 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:37.092 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:08:37.093 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.093 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::a871:90ff:feb9:397d/64 scope link 2023-11-28T11:08:37.093 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.093 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:08:37.093 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 8e:07:21:2f:e8:e5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:08:37.093 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::8c07:21ff:fe2f:e8e5/64 scope link tentative 2023-11-28T11:08:37.093 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:37.094 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:08:37.155 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-28T11:08:37.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:37.157 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-28T11:08:37.215 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-28T11:08:37.216 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:08:37.216 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:08:37.216 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:08:37.216 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:08:37.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:37.216 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T11:08:37.216 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T11:08:37.216 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:08:37.216 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:08:37.216 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:08:37.217 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T11:08:37.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:37.321 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T11:08:37.338 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:08:37.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:37.339 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list-id 2023-11-28T11:08:37.396 INFO:teuthology.orchestra.run.smithi156.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-28T11:08:37.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:37.397 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T11:08:37.397 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T11:08:37.397 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:08:37.397 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-28T11:08:37.397 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T11:08:37.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:37.490 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T11:08:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:08:37.535 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:08:37.535 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:08:37.535 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.535 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T11:08:37.536 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:08:37.536 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether fa:0f:88:53:78:88 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:08:37.536 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:08:37.536 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.536 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::f80f:88ff:fe53:7888/64 scope link 2023-11-28T11:08:37.536 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:37.550 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T11:08:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:08:37.587 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:08:37.587 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:08:37.587 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.587 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T11:08:37.587 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:08:37.587 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether fa:0f:88:53:78:88 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:08:37.587 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:08:37.587 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.588 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::f80f:88ff:fe53:7888/64 scope link 2023-11-28T11:08:37.588 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:37.598 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-28T11:08:37.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:37.599 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T11:08:37.599 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T11:08:37.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-28T11:08:37.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T11:08:37.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-28T11:08:37.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-28T11:08:37.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-28T11:08:37.599 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T11:08:37.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:37.861 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T11:08:37.861 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T11:08:37.861 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.1 up 2023-11-28T11:08:37.862 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T11:08:37.862 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T11:08:37.973 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:08:37.973 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-28T11:08:37.998 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-28T11:08:37.999 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:08:37.999 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-28T11:08:38.057 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:08:38.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:38.058 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:08:38.144 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:08:38.144 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T11:08:38.144 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:08:38.152 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:08:38.152 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T11:08:38.153 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:08:38.153 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:08:38.153 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:08:38.227 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:08:38.227 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T11:08:38.227 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:08:38.227 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:08:38.227 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:08:38.227 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T11:08:38.227 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T11:08:38.227 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=351da896-8dbf-473b-9472-c3454d9368bf" will pass to kernel 2023-11-28T11:08:38.229 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:08:38.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:38.229 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-28T11:08:38.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:38.256 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:08:38.329 INFO:teuthology.orchestra.run.smithi156.stdout:client4552 2023-11-28T11:08:38.329 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:08:38.329 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:38.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:38.387 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:08:38.459 INFO:teuthology.orchestra.run.smithi156.stdout:client4552 2023-11-28T11:08:38.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:38.460 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:08:38.532 INFO:teuthology.orchestra.run.smithi156.stdout:client4552 2023-11-28T11:08:38.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:38.533 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/351da896-8dbf-473b-9472-c3454d9368bf.client4552/status') 2023-11-28T11:08:38.613 INFO:teuthology.orchestra.run.smithi156.stdout:instance: client.4552 (3)172.21.15.156:0/3718613354 2023-11-28T11:08:38.613 INFO:teuthology.orchestra.run.smithi156.stdout:blocklisted: false 2023-11-28T11:08:38.613 INFO:teuthology.orchestra.run.smithi156.stderr:0+1 records in 2023-11-28T11:08:38.613 INFO:teuthology.orchestra.run.smithi156.stderr:0+1 records out 2023-11-28T11:08:38.613 INFO:teuthology.orchestra.run.smithi156.stderr:71 bytes copied, 2.4036e-05 s, 3.0 MB/s 2023-11-28T11:08:38.614 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-11-28T11:08:38.614 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:08:38.615 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T11:08:38.615 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:08:38.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.775+0000 7f0f526fb700 1 Processor -- start 2023-11-28T11:08:38.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.775+0000 7f0f526fb700 1 -- start start 2023-11-28T11:08:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.775+0000 7f0f526fb700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f4c06dee0 0x7f0f4c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.775+0000 7f0f526fb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f4c160540 0x7f0f4c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.775+0000 7f0f526fb700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4c15d060 con 0x7f0f4c071040 2023-11-28T11:08:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.775+0000 7f0f526fb700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4c15d340 con 0x7f0f4c160540 2023-11-28T11:08:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.775+0000 7f0f526fb700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c15d620 con 0x7f0f4c06dee0 2023-11-28T11:08:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f50c98700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f4c160540 0x7f0f4c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f4bfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f0f4c071040 0x7f0f4c071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47388/0 (socket says 172.21.15.156:47388) 2023-11-28T11:08:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f4bfff700 1 -- 172.21.15.156:0/1847358355 learned_addr learned my addr 172.21.15.156:0/1847358355 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f4b7fe700 1 --2- 172.21.15.156:0/1847358355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f4c06dee0 0x7f0f4c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f50c98700 1 -- 172.21.15.156:0/1847358355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f4c06dee0 msgr2=0x7f0f4c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f50c98700 1 --2- 172.21.15.156:0/1847358355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f4c06dee0 0x7f0f4c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f50c98700 1 -- 172.21.15.156:0/1847358355 >> v1:172.21.15.156:6789/0 conn(0x7f0f4c071040 legacy=0x7f0f4c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f50c98700 1 -- 172.21.15.156:0/1847358355 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c15d900 con 0x7f0f4c160540 2023-11-28T11:08:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f4affd700 1 -- 172.21.15.156:0/1847358355 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426279698 0 0) 0x7f0f4c15d060 con 0x7f0f4c071040 2023-11-28T11:08:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f50c98700 1 --2- 172.21.15.156:0/1847358355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c160540 0x7f0f4c1609c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009d90 tx=0x7f0f4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c3e5dd9544864b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f4affd700 1 -- 172.21.15.156:0/1847358355 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4400e070 con 0x7f0f4c160540 2023-11-28T11:08:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f4affd700 1 -- 172.21.15.156:0/1847358355 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4400bbd0 con 0x7f0f4c160540 2023-11-28T11:08:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.776+0000 7f0f4affd700 1 -- 172.21.15.156:0/1847358355 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44012430 con 0x7f0f4c160540 2023-11-28T11:08:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.777+0000 7f0f526fb700 1 -- 172.21.15.156:0/1847358355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c160540 msgr2=0x7f0f4c1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.777+0000 7f0f526fb700 1 --2- 172.21.15.156:0/1847358355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c160540 0x7f0f4c1609c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009d90 tx=0x7f0f4400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.777+0000 7f0f526fb700 1 -- 172.21.15.156:0/1847358355 shutdown_connections 2023-11-28T11:08:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.777+0000 7f0f526fb700 1 --2- 172.21.15.156:0/1847358355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c160540 0x7f0f4c1609c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.777+0000 7f0f526fb700 1 --2- 172.21.15.156:0/1847358355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f4c06dee0 0x7f0f4c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.777+0000 7f0f526fb700 1 -- 172.21.15.156:0/1847358355 >> 172.21.15.156:0/1847358355 conn(0x7f0f4c0d0000 msgr2=0x7f0f4c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:38.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.777+0000 7f0f526fb700 1 -- 172.21.15.156:0/1847358355 shutdown_connections 2023-11-28T11:08:38.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.777+0000 7f0f526fb700 1 -- 172.21.15.156:0/1847358355 wait complete. 2023-11-28T11:08:38.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.778+0000 7f0f526fb700 1 Processor -- start 2023-11-28T11:08:38.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.778+0000 7f0f526fb700 1 -- start start 2023-11-28T11:08:38.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f526fb700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f4c06dee0 0x7f0f4c14a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f526fb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f4c071040 0x7f0f4c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f526fb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c160540 0x7f0f4c0ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f526fb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c0c4990 con 0x7f0f4c160540 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f526fb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c156b60 con 0x7f0f4c071040 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f526fb700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c0c3aa0 con 0x7f0f4c06dee0 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f50c98700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c160540 0x7f0f4c0ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f50c98700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c160540 0x7f0f4c0ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60752/0 (socket says 172.21.15.156:60752) 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f50c98700 1 -- 172.21.15.156:0/3239982136 learned_addr learned my addr 172.21.15.156:0/3239982136 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f4b7fe700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f4c071040 0x7f0f4c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f4bfff700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f4c06dee0 0x7f0f4c14a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f50c98700 1 -- 172.21.15.156:0/3239982136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f4c06dee0 msgr2=0x7f0f4c14a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f50c98700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f4c06dee0 0x7f0f4c14a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f50c98700 1 -- 172.21.15.156:0/3239982136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f4c071040 msgr2=0x7f0f4c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f50c98700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f4c071040 0x7f0f4c0b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f50c98700 1 -- 172.21.15.156:0/3239982136 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f44009a70 con 0x7f0f4c160540 2023-11-28T11:08:38.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f4b7fe700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f4c071040 0x7f0f4c0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:38.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f4bfff700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f4c06dee0 0x7f0f4c14a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:38.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.779+0000 7f0f50c98700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c160540 0x7f0f4c0ade40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400b730 tx=0x7f0f44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:38.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.780+0000 7f0f48ff9700 1 -- 172.21.15.156:0/3239982136 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44012430 con 0x7f0f4c160540 2023-11-28T11:08:38.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.780+0000 7f0f48ff9700 1 -- 172.21.15.156:0/3239982136 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f44020070 con 0x7f0f4c160540 2023-11-28T11:08:38.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.780+0000 7f0f526fb700 1 -- 172.21.15.156:0/3239982136 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4c15ccd0 con 0x7f0f4c160540 2023-11-28T11:08:38.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.780+0000 7f0f48ff9700 1 -- 172.21.15.156:0/3239982136 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44004280 con 0x7f0f4c160540 2023-11-28T11:08:38.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.780+0000 7f0f526fb700 1 -- 172.21.15.156:0/3239982136 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4c06e870 con 0x7f0f4c160540 2023-11-28T11:08:38.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.781+0000 7f0f48ff9700 1 -- 172.21.15.156:0/3239982136 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0f440043e0 con 0x7f0f4c160540 2023-11-28T11:08:38.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.781+0000 7f0f526fb700 1 -- 172.21.15.156:0/3239982136 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4c06a270 con 0x7f0f4c160540 2023-11-28T11:08:38.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.781+0000 7f0f48ff9700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f380682f0 0x7f0f3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.781+0000 7f0f4bfff700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f380682f0 0x7f0f3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.781+0000 7f0f48ff9700 1 -- 172.21.15.156:0/3239982136 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f44022070 con 0x7f0f4c160540 2023-11-28T11:08:38.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.782+0000 7f0f4bfff700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f380682f0 0x7f0f3806a7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c007fd0 tx=0x7f0f3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:38.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:38.786+0000 7f0f48ff9700 1 -- 172.21.15.156:0/3239982136 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0f440549c0 con 0x7f0f4c160540 2023-11-28T11:08:39.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.011+0000 7f0f526fb700 1 -- 172.21.15.156:0/3239982136 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0f4c0c4b40 con 0x7f0f4c160540 2023-11-28T11:08:39.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.011+0000 7f0f48ff9700 1 -- 172.21.15.156:0/3239982136 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0f4c0c4b40 con 0x7f0f4c160540 2023-11-28T11:08:39.013 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:08:39.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.013+0000 7f0f526fb700 1 -- 172.21.15.156:0/3239982136 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f380682f0 msgr2=0x7f0f3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.013+0000 7f0f526fb700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f380682f0 0x7f0f3806a7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c007fd0 tx=0x7f0f3c009040 comp rx=0 tx=0).stop 2023-11-28T11:08:39.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.013+0000 7f0f526fb700 1 -- 172.21.15.156:0/3239982136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c160540 msgr2=0x7f0f4c0ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.013+0000 7f0f526fb700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c160540 0x7f0f4c0ade40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400b730 tx=0x7f0f44004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.013+0000 7f0f526fb700 1 -- 172.21.15.156:0/3239982136 shutdown_connections 2023-11-28T11:08:39.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.013+0000 7f0f526fb700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c160540 0x7f0f4c0ade40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.013+0000 7f0f526fb700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f380682f0 0x7f0f3806a7a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.013+0000 7f0f526fb700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f4c06dee0 0x7f0f4c14a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.013+0000 7f0f526fb700 1 --2- 172.21.15.156:0/3239982136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f4c071040 0x7f0f4c0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.013+0000 7f0f526fb700 1 -- 172.21.15.156:0/3239982136 >> 172.21.15.156:0/3239982136 conn(0x7f0f4c0d0000 msgr2=0x7f0f4c071410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:39.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.013+0000 7f0f526fb700 1 -- 172.21.15.156:0/3239982136 shutdown_connections 2023-11-28T11:08:39.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:39.013+0000 7f0f526fb700 1 -- 172.21.15.156:0/3239982136 wait complete. 2023-11-28T11:08:39.027 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:08:39.027 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:08:39.027 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi186.front.sepia.ceph.com 2023-11-28T11:08:39.027 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-28T11:08:39.027 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T11:08:39.027 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:08:39.028 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:08:39.028 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:08:39.028 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-28T11:08:39.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:39.028 DEBUG:teuthology.orchestra.run.smithi186:> ip addr 2023-11-28T11:08:39.050 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:08:39.050 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:08:39.050 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:08:39.050 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:39.050 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2023-11-28T11:08:39.051 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:39.051 INFO:teuthology.orchestra.run.smithi186.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:39.051 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2c brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:39.051 INFO:teuthology.orchestra.run.smithi186.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:39.051 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2d brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:39.051 INFO:teuthology.orchestra.run.smithi186.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:39.051 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f4 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:39.052 INFO:teuthology.orchestra.run.smithi186.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:08:39.052 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f5 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:39.052 INFO:teuthology.orchestra.run.smithi186.stdout: inet 172.21.15.186/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:08:39.052 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft 42371sec preferred_lft 42371sec 2023-11-28T11:08:39.052 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::193f:3827:310d:5280/64 scope link noprefixroute 2023-11-28T11:08:39.052 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:39.052 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:08:39.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:39.053 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T11:08:39.053 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T11:08:39.053 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:08:39.053 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip addr flush dev ceph-brx 2023-11-28T11:08:39.053 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set ceph-brx up 2023-11-28T11:08:39.053 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:08:39.054 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T11:08:39.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:39.232 DEBUG:teuthology.orchestra.run.smithi186:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:08:39.260 INFO:teuthology.orchestra.run.smithi186.stdout:1 2023-11-28T11:08:39.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:39.262 DEBUG:teuthology.orchestra.run.smithi186:> route 2023-11-28T11:08:39.327 INFO:teuthology.orchestra.run.smithi186.stdout:Kernel IP routing table 2023-11-28T11:08:39.327 INFO:teuthology.orchestra.run.smithi186.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:08:39.327 INFO:teuthology.orchestra.run.smithi186.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:08:39.327 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:08:39.328 INFO:teuthology.orchestra.run.smithi186.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:08:39.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:39.328 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T11:08:39.328 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T11:08:39.329 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:08:39.329 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:08:39.329 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:08:39.329 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T11:08:39.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:39.572 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-11-28T11:08:39.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:39.590 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list-id 2023-11-28T11:08:39.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:39.646 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T11:08:39.646 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T11:08:39.646 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:08:39.647 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-28T11:08:39.647 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T11:08:39.742 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-28T11:08:39.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:39.742 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T11:08:39.742 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T11:08:39.742 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-28T11:08:39.742 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:08:39.742 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-28T11:08:39.742 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-28T11:08:39.743 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-28T11:08:39.743 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T11:08:39.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:39.976 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T11:08:39.976 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T11:08:39.977 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set brx.0 up 2023-11-28T11:08:39.977 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:08:39.977 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T11:08:40.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:08:40.086 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-28T11:08:40.107 INFO:teuthology.orchestra.run.smithi186.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-28T11:08:40.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:08:40.108 DEBUG:teuthology.orchestra.run.smithi186:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-28T11:08:40.164 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:08:40.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:40.165 DEBUG:teuthology.orchestra.run.smithi186:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:08:40.257 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:08:40.257 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T11:08:40.257 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2023-11-28T11:08:40.353 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:08:40.353 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T11:08:40.353 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2023-11-28T11:08:40.353 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:08:40.353 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:08:40.429 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:08:40.430 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T11:08:40.430 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2023-11-28T11:08:40.430 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:08:40.430 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:08:40.430 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T11:08:40.431 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T11:08:40.431 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=351da896-8dbf-473b-9472-c3454d9368bf" will pass to kernel 2023-11-28T11:08:40.432 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:08:40.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:40.432 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-28T11:08:40.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:40.462 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:08:40.533 INFO:teuthology.orchestra.run.smithi186.stdout:client4665 2023-11-28T11:08:40.534 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:08:40.534 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:40.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:40.590 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:08:40.661 INFO:teuthology.orchestra.run.smithi186.stdout:client4665 2023-11-28T11:08:40.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:40.661 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:08:40.734 INFO:teuthology.orchestra.run.smithi186.stdout:client4665 2023-11-28T11:08:40.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:40.735 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/351da896-8dbf-473b-9472-c3454d9368bf.client4665/status') 2023-11-28T11:08:40.817 INFO:teuthology.orchestra.run.smithi186.stdout:instance: client.4665 (3)172.21.15.186:0/2996368282 2023-11-28T11:08:40.818 INFO:teuthology.orchestra.run.smithi186.stdout:blocklisted: false 2023-11-28T11:08:40.818 INFO:teuthology.orchestra.run.smithi186.stderr:0+1 records in 2023-11-28T11:08:40.818 INFO:teuthology.orchestra.run.smithi186.stderr:0+1 records out 2023-11-28T11:08:40.818 INFO:teuthology.orchestra.run.smithi186.stderr:71 bytes copied, 3.7256e-05 s, 1.9 MB/s 2023-11-28T11:08:40.819 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-11-28T11:08:40.819 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:08:40.819 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T11:08:40.819 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:08:40.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.951+0000 7fba95572700 1 Processor -- start 2023-11-28T11:08:40.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.951+0000 7fba95572700 1 -- start start 2023-11-28T11:08:40.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.952+0000 7fba95572700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fba9015cef0 0x7fba900af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.952+0000 7fba95572700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba900cc5f0 con 0x7fba900cbdf0 2023-11-28T11:08:40.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.952+0000 7fba95572700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9015d470 con 0x7fba9006ba90 2023-11-28T11:08:40.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.952+0000 7fba95572700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9015d750 con 0x7fba9015cef0 2023-11-28T11:08:40.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.952+0000 7fba8e7fc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fba900cbdf0 0x7fba9006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47402/0 (socket says 172.21.15.156:47402) 2023-11-28T11:08:40.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.952+0000 7fba8e7fc700 1 -- 172.21.15.156:0/1575035823 learned_addr learned my addr 172.21.15.156:0/1575035823 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:40.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.953+0000 7fba8effd700 1 --2- 172.21.15.156:0/1575035823 >> v2:172.21.15.186:3300/0 conn(0x7fba9015cef0 0x7fba900af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.953+0000 7fba8dffb700 1 -- 172.21.15.156:0/1575035823 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540550101 0 0) 0x7fba900cc5f0 con 0x7fba900cbdf0 2023-11-28T11:08:40.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.953+0000 7fba8dffb700 1 -- 172.21.15.156:0/1575035823 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba780035b0 con 0x7fba900cbdf0 2023-11-28T11:08:40.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.953+0000 7fba8dffb700 1 -- 172.21.15.156:0/1575035823 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633943092 0 0) 0x7fba9015d470 con 0x7fba9006ba90 2023-11-28T11:08:40.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.953+0000 7fba8dffb700 1 -- 172.21.15.156:0/1575035823 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba900cc5f0 con 0x7fba9006ba90 2023-11-28T11:08:40.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.953+0000 7fba8effd700 1 -- 172.21.15.156:0/1575035823 >> v1:172.21.15.186:6789/0 conn(0x7fba9006ba90 legacy=0x7fba900ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.953+0000 7fba8effd700 1 -- 172.21.15.156:0/1575035823 >> v1:172.21.15.156:6789/0 conn(0x7fba900cbdf0 legacy=0x7fba9006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.953+0000 7fba8effd700 1 -- 172.21.15.156:0/1575035823 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9015da30 con 0x7fba9015cef0 2023-11-28T11:08:40.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.953+0000 7fba8effd700 1 --2- 172.21.15.156:0/1575035823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba9015cef0 0x7fba900af400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fba84009870 tx=0x7fba84009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6e23575fed50bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.953+0000 7fba8dffb700 1 -- 172.21.15.156:0/1575035823 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84011830 con 0x7fba9015cef0 2023-11-28T11:08:40.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.953+0000 7fba8dffb700 1 -- 172.21.15.156:0/1575035823 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba84011990 con 0x7fba9015cef0 2023-11-28T11:08:40.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.953+0000 7fba8dffb700 1 -- 172.21.15.156:0/1575035823 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84011d20 con 0x7fba9015cef0 2023-11-28T11:08:40.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.954+0000 7fba95572700 1 -- 172.21.15.156:0/1575035823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba9015cef0 msgr2=0x7fba900af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.954+0000 7fba95572700 1 --2- 172.21.15.156:0/1575035823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba9015cef0 0x7fba900af400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fba84009870 tx=0x7fba84009b80 comp rx=0 tx=0).stop 2023-11-28T11:08:40.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.954+0000 7fba95572700 1 -- 172.21.15.156:0/1575035823 shutdown_connections 2023-11-28T11:08:40.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.954+0000 7fba95572700 1 --2- 172.21.15.156:0/1575035823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba9015cef0 0x7fba900af400 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.954+0000 7fba95572700 1 -- 172.21.15.156:0/1575035823 >> 172.21.15.156:0/1575035823 conn(0x7fba900d0000 msgr2=0x7fba900d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:40.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.954+0000 7fba95572700 1 -- 172.21.15.156:0/1575035823 shutdown_connections 2023-11-28T11:08:40.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.954+0000 7fba95572700 1 -- 172.21.15.156:0/1575035823 wait complete. 2023-11-28T11:08:40.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.955+0000 7fba95572700 1 Processor -- start 2023-11-28T11:08:40.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.955+0000 7fba95572700 1 -- start start 2023-11-28T11:08:40.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba95572700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9006ba90 0x7fba901efb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba95572700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba900cbdf0 0x7fba90099a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba95572700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba9015cef0 0x7fba900c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba95572700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba90151010 con 0x7fba9006ba90 2023-11-28T11:08:40.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba95572700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba90099f80 con 0x7fba9015cef0 2023-11-28T11:08:40.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba95572700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9009cc30 con 0x7fba900cbdf0 2023-11-28T11:08:40.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba8effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9006ba90 0x7fba901efb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba8effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9006ba90 0x7fba901efb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60762/0 (socket says 172.21.15.156:60762) 2023-11-28T11:08:40.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba8effd700 1 -- 172.21.15.156:0/385211030 learned_addr learned my addr 172.21.15.156:0/385211030 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:40.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba8e7fc700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba900cbdf0 0x7fba90099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba8f7fe700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba9015cef0 0x7fba900c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba8effd700 1 -- 172.21.15.156:0/385211030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba900cbdf0 msgr2=0x7fba90099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba8effd700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba900cbdf0 0x7fba90099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba8effd700 1 -- 172.21.15.156:0/385211030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba9015cef0 msgr2=0x7fba900c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba8effd700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba9015cef0 0x7fba900c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.956+0000 7fba8effd700 1 -- 172.21.15.156:0/385211030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba840094a0 con 0x7fba9006ba90 2023-11-28T11:08:40.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.957+0000 7fba8effd700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9006ba90 0x7fba901efb10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba84014040 tx=0x7fba84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.957+0000 7fba777fe700 1 -- 172.21.15.156:0/385211030 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8401a970 con 0x7fba9006ba90 2023-11-28T11:08:40.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.957+0000 7fba777fe700 1 -- 172.21.15.156:0/385211030 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8401aad0 con 0x7fba9006ba90 2023-11-28T11:08:40.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.957+0000 7fba777fe700 1 -- 172.21.15.156:0/385211030 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8401ae40 con 0x7fba9006ba90 2023-11-28T11:08:40.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.957+0000 7fba95572700 1 -- 172.21.15.156:0/385211030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9015dff0 con 0x7fba9006ba90 2023-11-28T11:08:40.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.957+0000 7fba95572700 1 -- 172.21.15.156:0/385211030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9015dd10 con 0x7fba9006ba90 2023-11-28T11:08:40.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.958+0000 7fba95572700 1 -- 172.21.15.156:0/385211030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba900cad60 con 0x7fba9006ba90 2023-11-28T11:08:40.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.958+0000 7fba777fe700 1 -- 172.21.15.156:0/385211030 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fba8400f050 con 0x7fba9006ba90 2023-11-28T11:08:40.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.958+0000 7fba777fe700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fba7c0682f0 0x7fba7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.959+0000 7fba777fe700 1 -- 172.21.15.156:0/385211030 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba84086e50 con 0x7fba9006ba90 2023-11-28T11:08:40.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.959+0000 7fba8e7fc700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fba7c0682f0 0x7fba7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.959+0000 7fba8e7fc700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fba7c0682f0 0x7fba7c06a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fba900c6d10 tx=0x7fba80007e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:40.963+0000 7fba777fe700 1 -- 172.21.15.156:0/385211030 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fba840536b0 con 0x7fba9006ba90 2023-11-28T11:08:41.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.190+0000 7fba95572700 1 -- 172.21.15.156:0/385211030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fba900d0e60 con 0x7fba9006ba90 2023-11-28T11:08:41.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.190+0000 7fba777fe700 1 -- 172.21.15.156:0/385211030 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fba84056cd0 con 0x7fba9006ba90 2023-11-28T11:08:41.191 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:08:41.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.192+0000 7fba95572700 1 -- 172.21.15.156:0/385211030 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fba7c0682f0 msgr2=0x7fba7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.192+0000 7fba95572700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fba7c0682f0 0x7fba7c06a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fba900c6d10 tx=0x7fba80007e00 comp rx=0 tx=0).stop 2023-11-28T11:08:41.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.192+0000 7fba95572700 1 -- 172.21.15.156:0/385211030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9006ba90 msgr2=0x7fba901efb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.192+0000 7fba95572700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9006ba90 0x7fba901efb10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba84014040 tx=0x7fba84004750 comp rx=0 tx=0).stop 2023-11-28T11:08:41.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.192+0000 7fba95572700 1 -- 172.21.15.156:0/385211030 shutdown_connections 2023-11-28T11:08:41.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.192+0000 7fba95572700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba9015cef0 0x7fba900c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.192+0000 7fba95572700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba9006ba90 0x7fba901efb10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.192+0000 7fba95572700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fba7c0682f0 0x7fba7c06a7a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.192+0000 7fba95572700 1 --2- 172.21.15.156:0/385211030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba900cbdf0 0x7fba90099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.192+0000 7fba95572700 1 -- 172.21.15.156:0/385211030 >> 172.21.15.156:0/385211030 conn(0x7fba900d0000 msgr2=0x7fba900afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.192+0000 7fba95572700 1 -- 172.21.15.156:0/385211030 shutdown_connections 2023-11-28T11:08:41.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:41.192+0000 7fba95572700 1 -- 172.21.15.156:0/385211030 wait complete. 2023-11-28T11:08:41.206 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:08:41.206 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:08:41.206 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi186.front.sepia.ceph.com 2023-11-28T11:08:41.206 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-28T11:08:41.206 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T11:08:41.206 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:08:41.206 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:08:41.206 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:08:41.206 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-28T11:08:41.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:41.207 DEBUG:teuthology.orchestra.run.smithi186:> ip addr 2023-11-28T11:08:41.224 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:08:41.224 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:08:41.224 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:08:41.224 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.225 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2023-11-28T11:08:41.225 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.225 INFO:teuthology.orchestra.run.smithi186.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:41.225 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2c brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:41.225 INFO:teuthology.orchestra.run.smithi186.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:41.225 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2d brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:41.225 INFO:teuthology.orchestra.run.smithi186.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:41.225 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f4 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:41.225 INFO:teuthology.orchestra.run.smithi186.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:08:41.225 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f5 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:41.225 INFO:teuthology.orchestra.run.smithi186.stdout: inet 172.21.15.186/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:08:41.225 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft 42368sec preferred_lft 42368sec 2023-11-28T11:08:41.226 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::193f:3827:310d:5280/64 scope link noprefixroute 2023-11-28T11:08:41.226 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.226 INFO:teuthology.orchestra.run.smithi186.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:08:41.226 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0e:30:c3:88:c5:57 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:41.226 INFO:teuthology.orchestra.run.smithi186.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:08:41.226 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.226 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::709a:78ff:fee2:6aa1/64 scope link 2023-11-28T11:08:41.226 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.226 INFO:teuthology.orchestra.run.smithi186.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:08:41.226 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0e:30:c3:88:c5:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:08:41.226 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::c30:c3ff:fe88:c557/64 scope link tentative 2023-11-28T11:08:41.226 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:41.227 DEBUG:teuthology.orchestra.run.smithi186:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:08:41.289 INFO:teuthology.orchestra.run.smithi186.stdout:1 2023-11-28T11:08:41.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:41.292 DEBUG:teuthology.orchestra.run.smithi186:> route 2023-11-28T11:08:41.351 INFO:teuthology.orchestra.run.smithi186.stdout:Kernel IP routing table 2023-11-28T11:08:41.351 INFO:teuthology.orchestra.run.smithi186.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:08:41.351 INFO:teuthology.orchestra.run.smithi186.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:08:41.352 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:08:41.352 INFO:teuthology.orchestra.run.smithi186.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:08:41.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:41.352 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T11:08:41.352 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T11:08:41.353 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:08:41.353 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:08:41.353 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:08:41.353 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T11:08:41.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:41.459 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-11-28T11:08:41.476 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T11:08:41.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:41.477 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list-id 2023-11-28T11:08:41.533 INFO:teuthology.orchestra.run.smithi186.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-28T11:08:41.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:41.534 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T11:08:41.534 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T11:08:41.534 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T11:08:41.534 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-28T11:08:41.534 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T11:08:41.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:41.629 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T11:08:41.671 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:08:41.671 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:08:41.671 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:08:41.671 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.671 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2023-11-28T11:08:41.672 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.672 INFO:teuthology.orchestra.run.smithi186.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:08:41.672 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 5a:9c:10:ee:9f:1c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:08:41.672 INFO:teuthology.orchestra.run.smithi186.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:08:41.672 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.672 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::589c:10ff:feee:9f1c/64 scope link 2023-11-28T11:08:41.672 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:41.683 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T11:08:41.758 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:08:41.758 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:08:41.758 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:08:41.759 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.759 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2023-11-28T11:08:41.759 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.759 INFO:teuthology.orchestra.run.smithi186.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:08:41.759 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 5a:9c:10:ee:9f:1c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:08:41.759 INFO:teuthology.orchestra.run.smithi186.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:08:41.759 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.759 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::589c:10ff:feee:9f1c/64 scope link 2023-11-28T11:08:41.759 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:41.771 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-28T11:08:41.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:41.772 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T11:08:41.772 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T11:08:41.772 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-28T11:08:41.772 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T11:08:41.772 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-28T11:08:41.772 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-28T11:08:41.772 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-28T11:08:41.772 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T11:08:42.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.027 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T11:08:42.027 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T11:08:42.027 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set brx.1 up 2023-11-28T11:08:42.027 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T11:08:42.027 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T11:08:42.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:08:42.134 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-28T11:08:42.155 INFO:teuthology.orchestra.run.smithi186.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-28T11:08:42.155 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:08:42.155 DEBUG:teuthology.orchestra.run.smithi186:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-28T11:08:42.212 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:08:42.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.212 DEBUG:teuthology.orchestra.run.smithi186:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:08:42.297 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:08:42.297 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T11:08:42.298 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2023-11-28T11:08:42.306 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:08:42.306 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T11:08:42.306 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2023-11-28T11:08:42.306 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:08:42.307 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:08:42.332 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:08:42.333 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T11:08:42.333 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2023-11-28T11:08:42.333 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:08:42.333 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:08:42.333 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T11:08:42.333 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T11:08:42.333 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=351da896-8dbf-473b-9472-c3454d9368bf" will pass to kernel 2023-11-28T11:08:42.336 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:08:42.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.336 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-28T11:08:42.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.366 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:08:42.438 INFO:teuthology.orchestra.run.smithi186.stdout:client4564 2023-11-28T11:08:42.438 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:08:42.438 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:42.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.496 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:08:42.568 INFO:teuthology.orchestra.run.smithi186.stdout:client4564 2023-11-28T11:08:42.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.569 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:08:42.639 INFO:teuthology.orchestra.run.smithi186.stdout:client4564 2023-11-28T11:08:42.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:42.640 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/351da896-8dbf-473b-9472-c3454d9368bf.client4564/status') 2023-11-28T11:08:42.719 INFO:teuthology.orchestra.run.smithi186.stdout:instance: client.4564 (3)192.168.0.2:0/1418935058 2023-11-28T11:08:42.720 INFO:teuthology.orchestra.run.smithi186.stdout:blocklisted: false 2023-11-28T11:08:42.720 INFO:teuthology.orchestra.run.smithi186.stderr:0+1 records in 2023-11-28T11:08:42.721 INFO:teuthology.orchestra.run.smithi186.stderr:0+1 records out 2023-11-28T11:08:42.721 INFO:teuthology.orchestra.run.smithi186.stderr:69 bytes copied, 2.249e-05 s, 3.1 MB/s 2023-11-28T11:08:42.721 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-28T11:08:42.829 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:42.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.961+0000 7f6864398700 1 Processor -- start 2023-11-28T11:08:42.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.961+0000 7f6864398700 1 -- start start 2023-11-28T11:08:42.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.962+0000 7f6864398700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f685c0cd8c0 0x7f685c140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.962+0000 7f6864398700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f685c06f850 0x7f685c06fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.962+0000 7f6864398700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c13ff00 con 0x7f685c0cdc90 2023-11-28T11:08:42.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.962+0000 7f6864398700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c1404c0 con 0x7f685c06f850 2023-11-28T11:08:42.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.962+0000 7f6864398700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c15aa80 con 0x7f685c0cd8c0 2023-11-28T11:08:42.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.962+0000 7f6862935700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f685c06f850 0x7f685c06fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.963+0000 7f6862935700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f685c06f850 0x7f685c06fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60774/0 (socket says 172.21.15.156:60774) 2023-11-28T11:08:42.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.963+0000 7f6862935700 1 -- 172.21.15.156:0/2064070358 learned_addr learned my addr 172.21.15.156:0/2064070358 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:42.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.963+0000 7f6862134700 1 --2- 172.21.15.156:0/2064070358 >> v2:172.21.15.186:3300/0 conn(0x7f685c0cd8c0 0x7f685c140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.963+0000 7f6862935700 1 -- 172.21.15.156:0/2064070358 >> v2:172.21.15.186:3300/0 conn(0x7f685c0cd8c0 msgr2=0x7f685c140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.963+0000 7f6862935700 1 --2- 172.21.15.156:0/2064070358 >> v2:172.21.15.186:3300/0 conn(0x7f685c0cd8c0 0x7f685c140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.963+0000 7f6862935700 1 -- 172.21.15.156:0/2064070358 >> v1:172.21.15.186:6789/0 conn(0x7f685c0cdc90 legacy=0x7f685c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.963+0000 7f6862935700 1 -- 172.21.15.156:0/2064070358 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0b3040 con 0x7f685c06f850 2023-11-28T11:08:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.963+0000 7f6862935700 1 --2- 172.21.15.156:0/2064070358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f685c06f850 0x7f685c06fcf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f685c070a20 tx=0x7f685400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578b2c1385ca1888 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.963+0000 7f6861132700 1 -- 172.21.15.156:0/2064070358 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685400e070 con 0x7f685c06f850 2023-11-28T11:08:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.963+0000 7f6861132700 1 -- 172.21.15.156:0/2064070358 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685400b910 con 0x7f685c06f850 2023-11-28T11:08:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.963+0000 7f6861132700 1 -- 172.21.15.156:0/2064070358 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685400bcc0 con 0x7f685c06f850 2023-11-28T11:08:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.964+0000 7f6864398700 1 -- 172.21.15.156:0/2064070358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f685c06f850 msgr2=0x7f685c06fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.964+0000 7f6864398700 1 --2- 172.21.15.156:0/2064070358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f685c06f850 0x7f685c06fcf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f685c070a20 tx=0x7f685400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.964+0000 7f6864398700 1 -- 172.21.15.156:0/2064070358 shutdown_connections 2023-11-28T11:08:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.964+0000 7f6864398700 1 --2- 172.21.15.156:0/2064070358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f685c06f850 0x7f685c06fcf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.964+0000 7f6864398700 1 --2- 172.21.15.156:0/2064070358 >> v2:172.21.15.186:3300/0 conn(0x7f685c0cd8c0 0x7f685c140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.964+0000 7f6864398700 1 -- 172.21.15.156:0/2064070358 >> 172.21.15.156:0/2064070358 conn(0x7f685c06aed0 msgr2=0x7f685c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.964+0000 7f6864398700 1 -- 172.21.15.156:0/2064070358 shutdown_connections 2023-11-28T11:08:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.964+0000 7f6864398700 1 -- 172.21.15.156:0/2064070358 wait complete. 2023-11-28T11:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.965+0000 7f6864398700 1 Processor -- start 2023-11-28T11:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.965+0000 7f6864398700 1 -- start start 2023-11-28T11:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6864398700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f685c06f850 0x7f685c154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6864398700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f685c0cd8c0 0x7f685c0b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6864398700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f685c0cdc90 0x7f685c143340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6864398700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c142eb0 con 0x7f685c0cdc90 2023-11-28T11:08:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6864398700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c15dc10 con 0x7f685c06f850 2023-11-28T11:08:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6864398700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c1422a0 con 0x7f685c0cd8c0 2023-11-28T11:08:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6862935700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f685c0cdc90 0x7f685c143340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6862134700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f685c06f850 0x7f685c154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6862935700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f685c0cdc90 0x7f685c143340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60780/0 (socket says 172.21.15.156:60780) 2023-11-28T11:08:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6862935700 1 -- 172.21.15.156:0/764881377 learned_addr learned my addr 172.21.15.156:0/764881377 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6861933700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f685c0cd8c0 0x7f685c0b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6862935700 1 -- 172.21.15.156:0/764881377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f685c0cd8c0 msgr2=0x7f685c0b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6862935700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f685c0cd8c0 0x7f685c0b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6862935700 1 -- 172.21.15.156:0/764881377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f685c06f850 msgr2=0x7f685c154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6862935700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f685c06f850 0x7f685c154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.966+0000 7f6862935700 1 -- 172.21.15.156:0/764881377 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6854009b30 con 0x7f685c0cdc90 2023-11-28T11:08:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.967+0000 7f6862935700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f685c0cdc90 0x7f685c143340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f685400b370 tx=0x7f6854004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.967+0000 7f6852ffd700 1 -- 172.21.15.156:0/764881377 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854004880 con 0x7f685c0cdc90 2023-11-28T11:08:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.967+0000 7f6852ffd700 1 -- 172.21.15.156:0/764881377 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68540099b0 con 0x7f685c0cdc90 2023-11-28T11:08:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.967+0000 7f6864398700 1 -- 172.21.15.156:0/764881377 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c0b35f0 con 0x7f685c0cdc90 2023-11-28T11:08:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.967+0000 7f6852ffd700 1 -- 172.21.15.156:0/764881377 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685401c680 con 0x7f685c0cdc90 2023-11-28T11:08:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.967+0000 7f6864398700 1 -- 172.21.15.156:0/764881377 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c0b3310 con 0x7f685c0cdc90 2023-11-28T11:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.968+0000 7f6864398700 1 -- 172.21.15.156:0/764881377 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c140150 con 0x7f685c0cdc90 2023-11-28T11:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.968+0000 7f6852ffd700 1 -- 172.21.15.156:0/764881377 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f685402c070 con 0x7f685c0cdc90 2023-11-28T11:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.968+0000 7f6852ffd700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f68400682f0 0x7f684006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.969+0000 7f6852ffd700 1 -- 172.21.15.156:0/764881377 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f68540877a0 con 0x7f685c0cdc90 2023-11-28T11:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.969+0000 7f6862134700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f68400682f0 0x7f684006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.969+0000 7f6862134700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f68400682f0 0x7f684006a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f685c13fbd0 tx=0x7f684c0091c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:42.973+0000 7f6852ffd700 1 -- 172.21.15.156:0/764881377 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6854054000 con 0x7f685c0cdc90 2023-11-28T11:08:43.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.159+0000 7f6864398700 1 -- 172.21.15.156:0/764881377 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f685c068690 con 0x7f685c0cdc90 2023-11-28T11:08:43.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.160+0000 7f6852ffd700 1 -- 172.21.15.156:0/764881377 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f6854057620 con 0x7f685c0cdc90 2023-11-28T11:08:43.162 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:43.162 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:18.354009+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:43.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.162+0000 7f6864398700 1 -- 172.21.15.156:0/764881377 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f68400682f0 msgr2=0x7f684006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.162+0000 7f6864398700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f68400682f0 0x7f684006a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f685c13fbd0 tx=0x7f684c0091c0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.162+0000 7f6864398700 1 -- 172.21.15.156:0/764881377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f685c0cdc90 msgr2=0x7f685c143340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.162+0000 7f6864398700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f685c0cdc90 0x7f685c143340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f685400b370 tx=0x7f6854004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.162+0000 7f6864398700 1 -- 172.21.15.156:0/764881377 shutdown_connections 2023-11-28T11:08:43.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.162+0000 7f6864398700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f685c0cdc90 0x7f685c143340 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.162+0000 7f6864398700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f685c06f850 0x7f685c154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.162+0000 7f6864398700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f68400682f0 0x7f684006a7a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.162+0000 7f6864398700 1 --2- 172.21.15.156:0/764881377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f685c0cd8c0 0x7f685c0b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.162+0000 7f6864398700 1 -- 172.21.15.156:0/764881377 >> 172.21.15.156:0/764881377 conn(0x7f685c06aed0 msgr2=0x7f685c0b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:43.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.162+0000 7f6864398700 1 -- 172.21.15.156:0/764881377 shutdown_connections 2023-11-28T11:08:43.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.162+0000 7f6864398700 1 -- 172.21.15.156:0/764881377 wait complete. 2023-11-28T11:08:43.176 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:15.021591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:16.332241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:43.177 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:43.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.300+0000 7f65f576b700 1 Processor -- start 2023-11-28T11:08:43.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.300+0000 7f65f576b700 1 -- start start 2023-11-28T11:08:43.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65f576b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f65f006ba90 0x7f65f00cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65f576b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f00cf5e0 con 0x7f65f00ccc40 2023-11-28T11:08:43.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65f576b700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f00cecb0 con 0x7f65f015ca50 2023-11-28T11:08:43.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65f576b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f00cd470 con 0x7f65f006ba90 2023-11-28T11:08:43.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65ee7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f65f006ba90 0x7f65f00cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65ef7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f65f00ccc40 0x7f65f00cfa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47434/0 (socket says 172.21.15.156:47434) 2023-11-28T11:08:43.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65ef7fe700 1 -- 172.21.15.156:0/3459055735 learned_addr learned my addr 172.21.15.156:0/3459055735 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:43.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65ee7fc700 1 -- 172.21.15.156:0/3459055735 >> v1:172.21.15.186:6789/0 conn(0x7f65f015ca50 legacy=0x7f65f006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:43.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65ee7fc700 1 -- 172.21.15.156:0/3459055735 >> v1:172.21.15.156:6789/0 conn(0x7f65f00ccc40 legacy=0x7f65f00cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65ee7fc700 1 -- 172.21.15.156:0/3459055735 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f00cef90 con 0x7f65f006ba90 2023-11-28T11:08:43.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65ee7fc700 1 --2- 172.21.15.156:0/3459055735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f006ba90 0x7f65f00cc700 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009840 tx=0x7f65e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51edcf2cdfc344a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65edffb700 1 -- 172.21.15.156:0/3459055735 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004230 con 0x7f65f006ba90 2023-11-28T11:08:43.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65edffb700 1 -- 172.21.15.156:0/3459055735 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e4004390 con 0x7f65f006ba90 2023-11-28T11:08:43.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.301+0000 7f65edffb700 1 -- 172.21.15.156:0/3459055735 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e40036a0 con 0x7f65f006ba90 2023-11-28T11:08:43.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.302+0000 7f65f576b700 1 -- 172.21.15.156:0/3459055735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f006ba90 msgr2=0x7f65f00cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.302+0000 7f65f576b700 1 --2- 172.21.15.156:0/3459055735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f006ba90 0x7f65f00cc700 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009840 tx=0x7f65e4009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:43.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.302+0000 7f65f576b700 1 -- 172.21.15.156:0/3459055735 shutdown_connections 2023-11-28T11:08:43.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.302+0000 7f65f576b700 1 --2- 172.21.15.156:0/3459055735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f006ba90 0x7f65f00cc700 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.302+0000 7f65f576b700 1 -- 172.21.15.156:0/3459055735 >> 172.21.15.156:0/3459055735 conn(0x7f65f009a230 msgr2=0x7f65f008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:43.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.302+0000 7f65f576b700 1 -- 172.21.15.156:0/3459055735 shutdown_connections 2023-11-28T11:08:43.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.302+0000 7f65f576b700 1 -- 172.21.15.156:0/3459055735 wait complete. 2023-11-28T11:08:43.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.303+0000 7f65f576b700 1 Processor -- start 2023-11-28T11:08:43.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.303+0000 7f65f576b700 1 -- start start 2023-11-28T11:08:43.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65f576b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f006ba90 0x7f65f00af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65f576b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65f00ccc40 0x7f65f015f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65f576b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65f015ca50 0x7f65f0145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65f576b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f0144ea0 con 0x7f65f006ba90 2023-11-28T11:08:43.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65f576b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f015fc40 con 0x7f65f00ccc40 2023-11-28T11:08:43.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65f576b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f00b30a0 con 0x7f65f015ca50 2023-11-28T11:08:43.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65eeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f006ba90 0x7f65f00af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65eeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f006ba90 0x7f65f00af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60790/0 (socket says 172.21.15.156:60790) 2023-11-28T11:08:43.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65eeffd700 1 -- 172.21.15.156:0/1816803852 learned_addr learned my addr 172.21.15.156:0/1816803852 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:43.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65ee7fc700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65f00ccc40 0x7f65f015f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65ef7fe700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65f015ca50 0x7f65f0145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65eeffd700 1 -- 172.21.15.156:0/1816803852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65f015ca50 msgr2=0x7f65f0145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65eeffd700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65f015ca50 0x7f65f0145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65eeffd700 1 -- 172.21.15.156:0/1816803852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65f00ccc40 msgr2=0x7f65f015f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65eeffd700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65f00ccc40 0x7f65f015f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65eeffd700 1 -- 172.21.15.156:0/1816803852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40094a0 con 0x7f65f006ba90 2023-11-28T11:08:43.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.304+0000 7f65ee7fc700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65f00ccc40 0x7f65f015f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:43.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.305+0000 7f65eeffd700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f006ba90 0x7f65f00af360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f65e000b4a0 tx=0x7f65e000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.305+0000 7f65d77fe700 1 -- 172.21.15.156:0/1816803852 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e0011830 con 0x7f65f006ba90 2023-11-28T11:08:43.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.305+0000 7f65d77fe700 1 -- 172.21.15.156:0/1816803852 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e0011990 con 0x7f65f006ba90 2023-11-28T11:08:43.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.305+0000 7f65f576b700 1 -- 172.21.15.156:0/1816803852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f00cd060 con 0x7f65f006ba90 2023-11-28T11:08:43.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.305+0000 7f65d77fe700 1 -- 172.21.15.156:0/1816803852 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e0011d40 con 0x7f65f006ba90 2023-11-28T11:08:43.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.305+0000 7f65f576b700 1 -- 172.21.15.156:0/1816803852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f00cf270 con 0x7f65f006ba90 2023-11-28T11:08:43.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.306+0000 7f65f576b700 1 -- 172.21.15.156:0/1816803852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f00cd1f0 con 0x7f65f006ba90 2023-11-28T11:08:43.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.306+0000 7f65d77fe700 1 -- 172.21.15.156:0/1816803852 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f65e000f050 con 0x7f65f006ba90 2023-11-28T11:08:43.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.307+0000 7f65d77fe700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65dc0682f0 0x7f65dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.307+0000 7f65d77fe700 1 -- 172.21.15.156:0/1816803852 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f65e0085d20 con 0x7f65f006ba90 2023-11-28T11:08:43.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.307+0000 7f65ee7fc700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65dc0682f0 0x7f65dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.307+0000 7f65ee7fc700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65dc0682f0 0x7f65dc06a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f65e4000c00 tx=0x7f65e401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.310+0000 7f65d77fe700 1 -- 172.21.15.156:0/1816803852 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f65e0052620 con 0x7f65f006ba90 2023-11-28T11:08:43.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.492+0000 7f65f576b700 1 -- 172.21.15.156:0/1816803852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65f00745c0 con 0x7f65f006ba90 2023-11-28T11:08:43.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.492+0000 7f65d77fe700 1 -- 172.21.15.156:0/1816803852 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65e0055c40 con 0x7f65f006ba90 2023-11-28T11:08:43.493 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:08:43.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.494+0000 7f65f576b700 1 -- 172.21.15.156:0/1816803852 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65dc0682f0 msgr2=0x7f65dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.494+0000 7f65f576b700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65dc0682f0 0x7f65dc06a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f65e4000c00 tx=0x7f65e401c040 comp rx=0 tx=0).stop 2023-11-28T11:08:43.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.494+0000 7f65f576b700 1 -- 172.21.15.156:0/1816803852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f006ba90 msgr2=0x7f65f00af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.494+0000 7f65f576b700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f006ba90 0x7f65f00af360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f65e000b4a0 tx=0x7f65e000b860 comp rx=0 tx=0).stop 2023-11-28T11:08:43.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.494+0000 7f65f576b700 1 -- 172.21.15.156:0/1816803852 shutdown_connections 2023-11-28T11:08:43.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.494+0000 7f65f576b700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65dc0682f0 0x7f65dc06a7a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.494+0000 7f65f576b700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65f015ca50 0x7f65f0145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.494+0000 7f65f576b700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f006ba90 0x7f65f00af360 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.494+0000 7f65f576b700 1 --2- 172.21.15.156:0/1816803852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65f00ccc40 0x7f65f015f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.494+0000 7f65f576b700 1 -- 172.21.15.156:0/1816803852 >> 172.21.15.156:0/1816803852 conn(0x7f65f009a230 msgr2=0x7f65f0157010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:43.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.494+0000 7f65f576b700 1 -- 172.21.15.156:0/1816803852 shutdown_connections 2023-11-28T11:08:43.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.494+0000 7f65f576b700 1 -- 172.21.15.156:0/1816803852 wait complete. 2023-11-28T11:08:43.508 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:43.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.630+0000 7f60cfd47700 1 Processor -- start 2023-11-28T11:08:43.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.631+0000 7f60cfd47700 1 -- start start 2023-11-28T11:08:43.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.631+0000 7f60cfd47700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f60c80cbdf0 0x7f60c80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.631+0000 7f60cfd47700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c80cc5f0 con 0x7f60c806ba90 2023-11-28T11:08:43.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.631+0000 7f60cfd47700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c815d470 con 0x7f60c815cef0 2023-11-28T11:08:43.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.631+0000 7f60cfd47700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c815d750 con 0x7f60c80cbdf0 2023-11-28T11:08:43.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.632+0000 7f60ce2e4700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f60c80cbdf0 0x7f60c80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.632+0000 7f60cd2e2700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f60c806ba90 0x7f60c80ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47446/0 (socket says 172.21.15.156:47446) 2023-11-28T11:08:43.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.632+0000 7f60cd2e2700 1 -- 172.21.15.156:0/2913634438 learned_addr learned my addr 172.21.15.156:0/2913634438 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:43.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.632+0000 7f60ce2e4700 1 -- 172.21.15.156:0/2913634438 >> v1:172.21.15.186:6789/0 conn(0x7f60c815cef0 legacy=0x7f60c806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.632+0000 7f60ce2e4700 1 -- 172.21.15.156:0/2913634438 >> v1:172.21.15.156:6789/0 conn(0x7f60c806ba90 legacy=0x7f60c80ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.632+0000 7f60ce2e4700 1 -- 172.21.15.156:0/2913634438 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c815da30 con 0x7f60c80cbdf0 2023-11-28T11:08:43.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.632+0000 7f60ccae1700 1 -- 172.21.15.156:0/2913634438 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936577049 0 0) 0x7f60c80cc5f0 con 0x7f60c806ba90 2023-11-28T11:08:43.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.632+0000 7f60ce2e4700 1 --2- 172.21.15.156:0/2913634438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c80cbdf0 0x7f60c80af400 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009d90 tx=0x7f60c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3655633f6a0ca9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.632+0000 7f60ccae1700 1 -- 172.21.15.156:0/2913634438 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c000e070 con 0x7f60c80cbdf0 2023-11-28T11:08:43.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.632+0000 7f60ccae1700 1 -- 172.21.15.156:0/2913634438 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c000bbd0 con 0x7f60c80cbdf0 2023-11-28T11:08:43.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.632+0000 7f60ccae1700 1 -- 172.21.15.156:0/2913634438 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0012430 con 0x7f60c80cbdf0 2023-11-28T11:08:43.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.633+0000 7f60cfd47700 1 -- 172.21.15.156:0/2913634438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c80cbdf0 msgr2=0x7f60c80af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.633+0000 7f60cfd47700 1 --2- 172.21.15.156:0/2913634438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c80cbdf0 0x7f60c80af400 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009d90 tx=0x7f60c000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:43.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.633+0000 7f60cfd47700 1 -- 172.21.15.156:0/2913634438 shutdown_connections 2023-11-28T11:08:43.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.633+0000 7f60cfd47700 1 --2- 172.21.15.156:0/2913634438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c80cbdf0 0x7f60c80af400 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.633+0000 7f60cfd47700 1 -- 172.21.15.156:0/2913634438 >> 172.21.15.156:0/2913634438 conn(0x7f60c80d0000 msgr2=0x7f60c80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:43.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.633+0000 7f60cfd47700 1 -- 172.21.15.156:0/2913634438 shutdown_connections 2023-11-28T11:08:43.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.633+0000 7f60cfd47700 1 -- 172.21.15.156:0/2913634438 wait complete. 2023-11-28T11:08:43.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.634+0000 7f60cfd47700 1 Processor -- start 2023-11-28T11:08:43.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.634+0000 7f60cfd47700 1 -- start start 2023-11-28T11:08:43.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cfd47700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60c806ba90 0x7f60c8153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cfd47700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c80cbdf0 0x7f60c8159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cfd47700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60c815cef0 0x7f60c81597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cfd47700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c8154920 con 0x7f60c80cbdf0 2023-11-28T11:08:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cfd47700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c80c7e80 con 0x7f60c815cef0 2023-11-28T11:08:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cfd47700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c80c7b80 con 0x7f60c806ba90 2023-11-28T11:08:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cd2e2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c80cbdf0 0x7f60c8159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cd2e2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c80cbdf0 0x7f60c8159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60806/0 (socket says 172.21.15.156:60806) 2023-11-28T11:08:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cd2e2700 1 -- 172.21.15.156:0/2660070794 learned_addr learned my addr 172.21.15.156:0/2660070794 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60ce2e4700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60c815cef0 0x7f60c81597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cdae3700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60c806ba90 0x7f60c8153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cd2e2700 1 -- 172.21.15.156:0/2660070794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60c806ba90 msgr2=0x7f60c8153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cd2e2700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60c806ba90 0x7f60c8153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cd2e2700 1 -- 172.21.15.156:0/2660070794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60c815cef0 msgr2=0x7f60c81597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cd2e2700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60c815cef0 0x7f60c81597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cd2e2700 1 -- 172.21.15.156:0/2660070794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c0009a70 con 0x7f60c80cbdf0 2023-11-28T11:08:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60cd2e2700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c80cbdf0 0x7f60c8159260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f60c400c550 tx=0x7f60c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.635+0000 7f60ba7fc700 1 -- 172.21.15.156:0/2660070794 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c4004c80 con 0x7f60c80cbdf0 2023-11-28T11:08:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.636+0000 7f60ba7fc700 1 -- 172.21.15.156:0/2660070794 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c4004e00 con 0x7f60c80cbdf0 2023-11-28T11:08:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.636+0000 7f60cfd47700 1 -- 172.21.15.156:0/2660070794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c815dff0 con 0x7f60c80cbdf0 2023-11-28T11:08:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.636+0000 7f60ba7fc700 1 -- 172.21.15.156:0/2660070794 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c4007310 con 0x7f60c80cbdf0 2023-11-28T11:08:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.636+0000 7f60cfd47700 1 -- 172.21.15.156:0/2660070794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c815dd10 con 0x7f60c80cbdf0 2023-11-28T11:08:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.637+0000 7f60cfd47700 1 -- 172.21.15.156:0/2660070794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c80cad60 con 0x7f60c80cbdf0 2023-11-28T11:08:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.637+0000 7f60ba7fc700 1 -- 172.21.15.156:0/2660070794 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f60c401a020 con 0x7f60c80cbdf0 2023-11-28T11:08:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.637+0000 7f60ba7fc700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60ac0682f0 0x7f60ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.637+0000 7f60ba7fc700 1 -- 172.21.15.156:0/2660070794 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f60c4085730 con 0x7f60c80cbdf0 2023-11-28T11:08:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.637+0000 7f60cdae3700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60ac0682f0 0x7f60ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.638+0000 7f60cdae3700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60ac0682f0 0x7f60ac06a7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f60c80b5e50 tx=0x7f60bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.642+0000 7f60ba7fc700 1 -- 172.21.15.156:0/2660070794 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f60c4051f90 con 0x7f60c80cbdf0 2023-11-28T11:08:43.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.828+0000 7f60cfd47700 1 -- 172.21.15.156:0/2660070794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f60c80c7ff0 con 0x7f60c80cbdf0 2023-11-28T11:08:43.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.828+0000 7f60ba7fc700 1 -- 172.21.15.156:0/2660070794 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f60c40555b0 con 0x7f60c80cbdf0 2023-11-28T11:08:43.830 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:08:43.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.830+0000 7f60cfd47700 1 -- 172.21.15.156:0/2660070794 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60ac0682f0 msgr2=0x7f60ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.830+0000 7f60cfd47700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60ac0682f0 0x7f60ac06a7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f60c80b5e50 tx=0x7f60bc009040 comp rx=0 tx=0).stop 2023-11-28T11:08:43.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.830+0000 7f60cfd47700 1 -- 172.21.15.156:0/2660070794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c80cbdf0 msgr2=0x7f60c8159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.830+0000 7f60cfd47700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c80cbdf0 0x7f60c8159260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f60c400c550 tx=0x7f60c400c860 comp rx=0 tx=0).stop 2023-11-28T11:08:43.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.830+0000 7f60cfd47700 1 -- 172.21.15.156:0/2660070794 shutdown_connections 2023-11-28T11:08:43.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.830+0000 7f60cfd47700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60c815cef0 0x7f60c81597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.830+0000 7f60cfd47700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60ac0682f0 0x7f60ac06a7a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.830+0000 7f60cfd47700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60c806ba90 0x7f60c8153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.830+0000 7f60cfd47700 1 --2- 172.21.15.156:0/2660070794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60c80cbdf0 0x7f60c8159260 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.830+0000 7f60cfd47700 1 -- 172.21.15.156:0/2660070794 >> 172.21.15.156:0/2660070794 conn(0x7f60c80d0000 msgr2=0x7f60c815c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:43.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.830+0000 7f60cfd47700 1 -- 172.21.15.156:0/2660070794 shutdown_connections 2023-11-28T11:08:43.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.830+0000 7f60cfd47700 1 -- 172.21.15.156:0/2660070794 wait complete. 2023-11-28T11:08:43.844 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:43.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.966+0000 7fc5bf7f6700 1 Processor -- start 2023-11-28T11:08:43.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.966+0000 7fc5bf7f6700 1 -- start start 2023-11-28T11:08:43.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5bf7f6700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc5b80cd8c0 0x7fc5b8140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5bf7f6700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b813ff00 con 0x7fc5b80cdc90 2023-11-28T11:08:43.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5bf7f6700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b81404c0 con 0x7fc5b806f850 2023-11-28T11:08:43.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5bf7f6700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b815aa80 con 0x7fc5b80cd8c0 2023-11-28T11:08:43.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5bd592700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc5b80cd8c0 0x7fc5b8140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5bcd91700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc5b80cdc90 0x7fc5b80b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47458/0 (socket says 172.21.15.156:47458) 2023-11-28T11:08:43.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5bcd91700 1 -- 172.21.15.156:0/126015038 learned_addr learned my addr 172.21.15.156:0/126015038 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:43.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5bd592700 1 -- 172.21.15.156:0/126015038 >> v1:172.21.15.186:6789/0 conn(0x7fc5b806f850 legacy=0x7fc5b806fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5bd592700 1 -- 172.21.15.156:0/126015038 >> v1:172.21.15.156:6789/0 conn(0x7fc5b80cdc90 legacy=0x7fc5b80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5bd592700 1 -- 172.21.15.156:0/126015038 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b80b3040 con 0x7fc5b80cd8c0 2023-11-28T11:08:43.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5affff700 1 -- 172.21.15.156:0/126015038 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071842135 0 0) 0x7fc5b813ff00 con 0x7fc5b80cdc90 2023-11-28T11:08:43.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5bd592700 1 --2- 172.21.15.156:0/126015038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b80cd8c0 0x7fc5b8140950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009820 tx=0x7fc5b4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae190b5324218c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.967+0000 7fc5affff700 1 -- 172.21.15.156:0/126015038 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4005e30 con 0x7fc5b80cd8c0 2023-11-28T11:08:43.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.968+0000 7fc5affff700 1 -- 172.21.15.156:0/126015038 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b4006030 con 0x7fc5b80cd8c0 2023-11-28T11:08:43.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.968+0000 7fc5affff700 1 -- 172.21.15.156:0/126015038 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b40063e0 con 0x7fc5b80cd8c0 2023-11-28T11:08:43.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.968+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/126015038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b80cd8c0 msgr2=0x7fc5b8140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.968+0000 7fc5bf7f6700 1 --2- 172.21.15.156:0/126015038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b80cd8c0 0x7fc5b8140950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009820 tx=0x7fc5b4009b30 comp rx=0 tx=0).stop 2023-11-28T11:08:43.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.968+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/126015038 shutdown_connections 2023-11-28T11:08:43.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.968+0000 7fc5bf7f6700 1 --2- 172.21.15.156:0/126015038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b80cd8c0 0x7fc5b8140950 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.968+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/126015038 >> 172.21.15.156:0/126015038 conn(0x7fc5b806aed0 msgr2=0x7fc5b806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:43.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.968+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/126015038 shutdown_connections 2023-11-28T11:08:43.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.968+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/126015038 wait complete. 2023-11-28T11:08:43.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.969+0000 7fc5bf7f6700 1 Processor -- start 2023-11-28T11:08:43.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.969+0000 7fc5bf7f6700 1 -- start start 2023-11-28T11:08:43.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.970+0000 7fc5bf7f6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5b806f850 0x7fc5b80b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.970+0000 7fc5bf7f6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5b80cd8c0 0x7fc5b814ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.970+0000 7fc5bf7f6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b80cdc90 0x7fc5b80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.970+0000 7fc5bf7f6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b80b6c80 con 0x7fc5b80cdc90 2023-11-28T11:08:43.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.970+0000 7fc5bf7f6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b8142ad0 con 0x7fc5b806f850 2023-11-28T11:08:43.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.970+0000 7fc5bf7f6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b80b6a30 con 0x7fc5b80cd8c0 2023-11-28T11:08:43.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.970+0000 7fc5bdd93700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b80cdc90 0x7fc5b80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.970+0000 7fc5bdd93700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b80cdc90 0x7fc5b80cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60820/0 (socket says 172.21.15.156:60820) 2023-11-28T11:08:43.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.970+0000 7fc5bdd93700 1 -- 172.21.15.156:0/1885917378 learned_addr learned my addr 172.21.15.156:0/1885917378 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:43.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.970+0000 7fc5bd592700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5b806f850 0x7fc5b80b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.970+0000 7fc5bcd91700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5b80cd8c0 0x7fc5b814ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.971+0000 7fc5bdd93700 1 -- 172.21.15.156:0/1885917378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5b80cd8c0 msgr2=0x7fc5b814ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.971+0000 7fc5bdd93700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5b80cd8c0 0x7fc5b814ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.971+0000 7fc5bdd93700 1 -- 172.21.15.156:0/1885917378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5b806f850 msgr2=0x7fc5b80b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.971+0000 7fc5bdd93700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5b806f850 0x7fc5b80b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.971+0000 7fc5bdd93700 1 -- 172.21.15.156:0/1885917378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b4009430 con 0x7fc5b80cdc90 2023-11-28T11:08:43.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.971+0000 7fc5bdd93700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b80cdc90 0x7fc5b80cea60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000cc50 tx=0x7fc5b000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.971+0000 7fc5adffb700 1 -- 172.21.15.156:0/1885917378 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0003c00 con 0x7fc5b80cdc90 2023-11-28T11:08:43.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.971+0000 7fc5adffb700 1 -- 172.21.15.156:0/1885917378 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b0003d60 con 0x7fc5b80cdc90 2023-11-28T11:08:43.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.971+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/1885917378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b80b35f0 con 0x7fc5b80cdc90 2023-11-28T11:08:43.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.971+0000 7fc5adffb700 1 -- 172.21.15.156:0/1885917378 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b000aab0 con 0x7fc5b80cdc90 2023-11-28T11:08:43.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.971+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/1885917378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b80b3310 con 0x7fc5b80cdc90 2023-11-28T11:08:43.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.972+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/1885917378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b8140150 con 0x7fc5b80cdc90 2023-11-28T11:08:43.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.972+0000 7fc5adffb700 1 -- 172.21.15.156:0/1885917378 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc5b0010050 con 0x7fc5b80cdc90 2023-11-28T11:08:43.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.973+0000 7fc5adffb700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc59c0682f0 0x7fc59c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.973+0000 7fc5adffb700 1 -- 172.21.15.156:0/1885917378 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5b0085ad0 con 0x7fc5b80cdc90 2023-11-28T11:08:43.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.973+0000 7fc5bd592700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc59c0682f0 0x7fc59c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.973+0000 7fc5bd592700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc59c0682f0 0x7fc59c06a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4013040 tx=0x7fc5b4008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:43.977+0000 7fc5adffb700 1 -- 172.21.15.156:0/1885917378 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc5b0052330 con 0x7fc5b80cdc90 2023-11-28T11:08:44.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.159+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/1885917378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5b8068690 con 0x7fc5b80cdc90 2023-11-28T11:08:44.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.160+0000 7fc5adffb700 1 -- 172.21.15.156:0/1885917378 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc5b0055950 con 0x7fc5b80cdc90 2023-11-28T11:08:44.161 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:08:44.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.162+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/1885917378 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc59c0682f0 msgr2=0x7fc59c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.162+0000 7fc5bf7f6700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc59c0682f0 0x7fc59c06a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4013040 tx=0x7fc5b4008070 comp rx=0 tx=0).stop 2023-11-28T11:08:44.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.162+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/1885917378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b80cdc90 msgr2=0x7fc5b80cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.162+0000 7fc5bf7f6700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b80cdc90 0x7fc5b80cea60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000cc50 tx=0x7fc5b000cf60 comp rx=0 tx=0).stop 2023-11-28T11:08:44.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.162+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/1885917378 shutdown_connections 2023-11-28T11:08:44.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.162+0000 7fc5bf7f6700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b80cdc90 0x7fc5b80cea60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.162+0000 7fc5bf7f6700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5b806f850 0x7fc5b80b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.162+0000 7fc5bf7f6700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc59c0682f0 0x7fc59c06a7a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.162+0000 7fc5bf7f6700 1 --2- 172.21.15.156:0/1885917378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5b80cd8c0 0x7fc5b814ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.162+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/1885917378 >> 172.21.15.156:0/1885917378 conn(0x7fc5b806aed0 msgr2=0x7fc5b813e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:44.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.162+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/1885917378 shutdown_connections 2023-11-28T11:08:44.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.162+0000 7fc5bf7f6700 1 -- 172.21.15.156:0/1885917378 wait complete. 2023-11-28T11:08:44.175 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:44.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.297+0000 7f0f41b3e700 1 Processor -- start 2023-11-28T11:08:44.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.297+0000 7f0f41b3e700 1 -- start start 2023-11-28T11:08:44.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.298+0000 7f0f41b3e700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0f3c0ce040 0x7f0f3c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.298+0000 7f0f41b3e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f3c06b420 con 0x7f0f3c0d0a00 2023-11-28T11:08:44.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.298+0000 7f0f41b3e700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f3c06b700 con 0x7f0f3c0b2930 2023-11-28T11:08:44.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.298+0000 7f0f41b3e700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f3c06ff60 con 0x7f0f3c0ce040 2023-11-28T11:08:44.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.298+0000 7f0f3affd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f0f3c0d0a00 0x7f0f3c06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47470/0 (socket says 172.21.15.156:47470) 2023-11-28T11:08:44.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.298+0000 7f0f3affd700 1 -- 172.21.15.156:0/1494607023 learned_addr learned my addr 172.21.15.156:0/1494607023 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:44.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.298+0000 7f0f3b7fe700 1 --2- 172.21.15.156:0/1494607023 >> v2:172.21.15.186:3300/0 conn(0x7f0f3c0ce040 0x7f0f3c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.298+0000 7f0f3a7fc700 1 -- 172.21.15.156:0/1494607023 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141304519 0 0) 0x7f0f3c06b420 con 0x7f0f3c0d0a00 2023-11-28T11:08:44.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.298+0000 7f0f3a7fc700 1 -- 172.21.15.156:0/1494607023 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f240035b0 con 0x7f0f3c0d0a00 2023-11-28T11:08:44.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.299+0000 7f0f3a7fc700 1 -- 172.21.15.156:0/1494607023 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1391685335 0 0) 0x7f0f240035b0 con 0x7f0f3c0d0a00 2023-11-28T11:08:44.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.299+0000 7f0f3b7fe700 1 -- 172.21.15.156:0/1494607023 >> v1:172.21.15.186:6789/0 conn(0x7f0f3c0b2930 legacy=0x7f0f3c06c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.299+0000 7f0f3b7fe700 1 -- 172.21.15.156:0/1494607023 >> v1:172.21.15.156:6789/0 conn(0x7f0f3c0d0a00 legacy=0x7f0f3c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.299+0000 7f0f3b7fe700 1 -- 172.21.15.156:0/1494607023 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c06b9e0 con 0x7f0f3c0ce040 2023-11-28T11:08:44.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.299+0000 7f0f3b7fe700 1 --2- 172.21.15.156:0/1494607023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f3c0ce040 0x7f0f3c15df10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009820 tx=0x7f0f30009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=584ed8a35fa7aaf1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.299+0000 7f0f3a7fc700 1 -- 172.21.15.156:0/1494607023 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c001670 con 0x7f0f3c0ce040 2023-11-28T11:08:44.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.299+0000 7f0f3a7fc700 1 -- 172.21.15.156:0/1494607023 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f30005c30 con 0x7f0f3c0ce040 2023-11-28T11:08:44.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.299+0000 7f0f3a7fc700 1 -- 172.21.15.156:0/1494607023 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30006030 con 0x7f0f3c0ce040 2023-11-28T11:08:44.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.300+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1494607023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f3c0ce040 msgr2=0x7f0f3c15df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.300+0000 7f0f41b3e700 1 --2- 172.21.15.156:0/1494607023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f3c0ce040 0x7f0f3c15df10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009820 tx=0x7f0f30009b30 comp rx=0 tx=0).stop 2023-11-28T11:08:44.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.300+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1494607023 shutdown_connections 2023-11-28T11:08:44.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.300+0000 7f0f41b3e700 1 --2- 172.21.15.156:0/1494607023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f3c0ce040 0x7f0f3c15df10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.300+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1494607023 >> 172.21.15.156:0/1494607023 conn(0x7f0f3c06aed0 msgr2=0x7f0f3c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:44.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.300+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1494607023 shutdown_connections 2023-11-28T11:08:44.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.300+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1494607023 wait complete. 2023-11-28T11:08:44.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.301+0000 7f0f41b3e700 1 Processor -- start 2023-11-28T11:08:44.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.301+0000 7f0f41b3e700 1 -- start start 2023-11-28T11:08:44.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f41b3e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f3c0b2930 0x7f0f3c0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f41b3e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c0ce040 0x7f0f3c151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f41b3e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f3c0d0a00 0x7f0f3c0b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f41b3e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c1f1f20 con 0x7f0f3c0ce040 2023-11-28T11:08:44.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f41b3e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c0c49c0 con 0x7f0f3c0d0a00 2023-11-28T11:08:44.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f41b3e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c0c4ca0 con 0x7f0f3c0b2930 2023-11-28T11:08:44.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f3affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c0ce040 0x7f0f3c151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f3affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c0ce040 0x7f0f3c151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60830/0 (socket says 172.21.15.156:60830) 2023-11-28T11:08:44.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f3affd700 1 -- 172.21.15.156:0/1267874198 learned_addr learned my addr 172.21.15.156:0/1267874198 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:44.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f3bfff700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f3c0d0a00 0x7f0f3c0b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f3b7fe700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f3c0b2930 0x7f0f3c0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f3affd700 1 -- 172.21.15.156:0/1267874198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f3c0b2930 msgr2=0x7f0f3c0c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f3affd700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f3c0b2930 0x7f0f3c0c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f3affd700 1 -- 172.21.15.156:0/1267874198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f3c0d0a00 msgr2=0x7f0f3c0b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f3affd700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f3c0d0a00 0x7f0f3c0b6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f3affd700 1 -- 172.21.15.156:0/1267874198 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f30009430 con 0x7f0f3c0ce040 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f3affd700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c0ce040 0x7f0f3c151ce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00c700 tx=0x7f0f2c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f23fff700 1 -- 172.21.15.156:0/1267874198 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c007560 con 0x7f0f3c0ce040 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f23fff700 1 -- 172.21.15.156:0/1267874198 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2c0076c0 con 0x7f0f3c0ce040 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1267874198 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3c06bfa0 con 0x7f0f3c0ce040 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f23fff700 1 -- 172.21.15.156:0/1267874198 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c004ab0 con 0x7f0f3c0ce040 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.302+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1267874198 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3c06bcc0 con 0x7f0f3c0ce040 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.303+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1267874198 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3c0b2db0 con 0x7f0f3c0ce040 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.304+0000 7f0f23fff700 1 -- 172.21.15.156:0/1267874198 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0f2c01a020 con 0x7f0f3c0ce040 2023-11-28T11:08:44.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.304+0000 7f0f23fff700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f280682f0 0x7f0f2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.304+0000 7f0f23fff700 1 -- 172.21.15.156:0/1267874198 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f2c085a60 con 0x7f0f3c0ce040 2023-11-28T11:08:44.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.304+0000 7f0f3b7fe700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f280682f0 0x7f0f2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.305+0000 7f0f3b7fe700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f280682f0 0x7f0f2806a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0f30015040 tx=0x7f0f30007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.308+0000 7f0f23fff700 1 -- 172.21.15.156:0/1267874198 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0f2c052260 con 0x7f0f3c0ce040 2023-11-28T11:08:44.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.488+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1267874198 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f3c068690 con 0x7f0f3c0ce040 2023-11-28T11:08:44.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.489+0000 7f0f23fff700 1 -- 172.21.15.156:0/1267874198 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f2c055880 con 0x7f0f3c0ce040 2023-11-28T11:08:44.491 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:08:44.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.491+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1267874198 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f280682f0 msgr2=0x7f0f2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.491+0000 7f0f41b3e700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f280682f0 0x7f0f2806a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0f30015040 tx=0x7f0f30007000 comp rx=0 tx=0).stop 2023-11-28T11:08:44.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.491+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1267874198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c0ce040 msgr2=0x7f0f3c151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.491+0000 7f0f41b3e700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c0ce040 0x7f0f3c151ce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00c700 tx=0x7f0f2c00cac0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.491+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1267874198 shutdown_connections 2023-11-28T11:08:44.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.491+0000 7f0f41b3e700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f3c0d0a00 0x7f0f3c0b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.491+0000 7f0f41b3e700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f280682f0 0x7f0f2806a7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.491+0000 7f0f41b3e700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f3c0b2930 0x7f0f3c0c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.491+0000 7f0f41b3e700 1 --2- 172.21.15.156:0/1267874198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c0ce040 0x7f0f3c151ce0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.491+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1267874198 >> 172.21.15.156:0/1267874198 conn(0x7f0f3c06aed0 msgr2=0x7f0f3c15d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:44.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.491+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1267874198 shutdown_connections 2023-11-28T11:08:44.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.491+0000 7f0f41b3e700 1 -- 172.21.15.156:0/1267874198 wait complete. 2023-11-28T11:08:44.505 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:44.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.625+0000 7f616ccf5700 1 Processor -- start 2023-11-28T11:08:44.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.625+0000 7f616ccf5700 1 -- start start 2023-11-28T11:08:44.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f616ccf5700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f61681658c0 0x7f6168167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f616ccf5700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616815d060 con 0x7f616806dee0 2023-11-28T11:08:44.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f616ccf5700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616815d340 con 0x7f6168071040 2023-11-28T11:08:44.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f616ccf5700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6168161550 con 0x7f61681658c0 2023-11-28T11:08:44.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f6166d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f61681658c0 0x7f6168167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f6165d9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f616806dee0 0x7f616806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47484/0 (socket says 172.21.15.156:47484) 2023-11-28T11:08:44.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f6165d9b700 1 -- 172.21.15.156:0/2313422085 learned_addr learned my addr 172.21.15.156:0/2313422085 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:44.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f6166d9d700 1 -- 172.21.15.156:0/2313422085 >> v1:172.21.15.186:6789/0 conn(0x7f6168071040 legacy=0x7f6168071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:44.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f6166d9d700 1 -- 172.21.15.156:0/2313422085 >> v1:172.21.15.156:6789/0 conn(0x7f616806dee0 legacy=0x7f616806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f6166d9d700 1 -- 172.21.15.156:0/2313422085 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616815d620 con 0x7f61681658c0 2023-11-28T11:08:44.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f616559a700 1 -- 172.21.15.156:0/2313422085 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010354693 0 0) 0x7f616815d060 con 0x7f616806dee0 2023-11-28T11:08:44.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f6166d9d700 1 --2- 172.21.15.156:0/2313422085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61681658c0 0x7f6168167ca0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6160009d90 tx=0x7f616000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5402df5c41224303 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f616559a700 1 -- 172.21.15.156:0/2313422085 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616000e070 con 0x7f61681658c0 2023-11-28T11:08:44.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f616559a700 1 -- 172.21.15.156:0/2313422085 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616000bbd0 con 0x7f61681658c0 2023-11-28T11:08:44.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.626+0000 7f616559a700 1 -- 172.21.15.156:0/2313422085 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160003a20 con 0x7f61681658c0 2023-11-28T11:08:44.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.627+0000 7f616ccf5700 1 -- 172.21.15.156:0/2313422085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61681658c0 msgr2=0x7f6168167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.627+0000 7f616ccf5700 1 --2- 172.21.15.156:0/2313422085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61681658c0 0x7f6168167ca0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6160009d90 tx=0x7f616000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:44.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.627+0000 7f616ccf5700 1 -- 172.21.15.156:0/2313422085 shutdown_connections 2023-11-28T11:08:44.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.627+0000 7f616ccf5700 1 --2- 172.21.15.156:0/2313422085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61681658c0 0x7f6168167ca0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.627+0000 7f616ccf5700 1 -- 172.21.15.156:0/2313422085 >> 172.21.15.156:0/2313422085 conn(0x7f61680d0000 msgr2=0x7f61680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:44.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.627+0000 7f616ccf5700 1 -- 172.21.15.156:0/2313422085 shutdown_connections 2023-11-28T11:08:44.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.627+0000 7f616ccf5700 1 -- 172.21.15.156:0/2313422085 wait complete. 2023-11-28T11:08:44.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.629+0000 7f616ccf5700 1 Processor -- start 2023-11-28T11:08:44.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.629+0000 7f616ccf5700 1 -- start start 2023-11-28T11:08:44.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f616ccf5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f616806dee0 0x7f6168151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f616ccf5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6168071040 0x7f6168153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f616ccf5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61681658c0 0x7f61681538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f616ccf5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6168151b40 con 0x7f61681658c0 2023-11-28T11:08:44.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f616ccf5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61680c6660 con 0x7f6168071040 2023-11-28T11:08:44.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f616ccf5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61680c5a50 con 0x7f616806dee0 2023-11-28T11:08:44.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f6166d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61681658c0 0x7f61681538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f6166d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61681658c0 0x7f61681538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60846/0 (socket says 172.21.15.156:60846) 2023-11-28T11:08:44.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f6166d9d700 1 -- 172.21.15.156:0/2954719777 learned_addr learned my addr 172.21.15.156:0/2954719777 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:44.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f6165d9b700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6168071040 0x7f6168153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f616659c700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f616806dee0 0x7f6168151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f6166d9d700 1 -- 172.21.15.156:0/2954719777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f616806dee0 msgr2=0x7f6168151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f6166d9d700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f616806dee0 0x7f6168151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f6166d9d700 1 -- 172.21.15.156:0/2954719777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6168071040 msgr2=0x7f6168153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f6166d9d700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6168071040 0x7f6168153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f6166d9d700 1 -- 172.21.15.156:0/2954719777 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6160009a70 con 0x7f61681658c0 2023-11-28T11:08:44.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f616659c700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f616806dee0 0x7f6168151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:44.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f6166d9d700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61681658c0 0x7f61681538c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6160004000 tx=0x7f6160004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f6156ffd700 1 -- 172.21.15.156:0/2954719777 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160012630 con 0x7f61681658c0 2023-11-28T11:08:44.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.630+0000 7f6156ffd700 1 -- 172.21.15.156:0/2954719777 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61600127b0 con 0x7f61681658c0 2023-11-28T11:08:44.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.631+0000 7f616ccf5700 1 -- 172.21.15.156:0/2954719777 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616815ccd0 con 0x7f61681658c0 2023-11-28T11:08:44.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.631+0000 7f6156ffd700 1 -- 172.21.15.156:0/2954719777 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160012b60 con 0x7f61681658c0 2023-11-28T11:08:44.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.631+0000 7f616ccf5700 1 -- 172.21.15.156:0/2954719777 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616815d900 con 0x7f61681658c0 2023-11-28T11:08:44.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.632+0000 7f616ccf5700 1 -- 172.21.15.156:0/2954719777 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616806a270 con 0x7f61681658c0 2023-11-28T11:08:44.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.632+0000 7f6156ffd700 1 -- 172.21.15.156:0/2954719777 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6160012cc0 con 0x7f61681658c0 2023-11-28T11:08:44.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.632+0000 7f6156ffd700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f61480682f0 0x7f614806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.632+0000 7f6156ffd700 1 -- 172.21.15.156:0/2954719777 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6160087f50 con 0x7f61681658c0 2023-11-28T11:08:44.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.632+0000 7f616659c700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f61480682f0 0x7f614806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.633+0000 7f616659c700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f61480682f0 0x7f614806a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6158009930 tx=0x7f6158008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.637+0000 7f6156ffd700 1 -- 172.21.15.156:0/2954719777 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f61600b95e0 con 0x7f61681658c0 2023-11-28T11:08:44.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.822+0000 7f616ccf5700 1 -- 172.21.15.156:0/2954719777 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61680c6810 con 0x7f61681658c0 2023-11-28T11:08:44.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.823+0000 7f6156ffd700 1 -- 172.21.15.156:0/2954719777 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f61680c6810 con 0x7f61681658c0 2023-11-28T11:08:44.824 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:44.824 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:18.354009+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:44.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.825+0000 7f616ccf5700 1 -- 172.21.15.156:0/2954719777 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f61480682f0 msgr2=0x7f614806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.825+0000 7f616ccf5700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f61480682f0 0x7f614806a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6158009930 tx=0x7f6158008040 comp rx=0 tx=0).stop 2023-11-28T11:08:44.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.825+0000 7f616ccf5700 1 -- 172.21.15.156:0/2954719777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61681658c0 msgr2=0x7f61681538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.825+0000 7f616ccf5700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61681658c0 0x7f61681538c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6160004000 tx=0x7f6160004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.825+0000 7f616ccf5700 1 -- 172.21.15.156:0/2954719777 shutdown_connections 2023-11-28T11:08:44.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.825+0000 7f616ccf5700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61681658c0 0x7f61681538c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.825+0000 7f616ccf5700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f61480682f0 0x7f614806a7a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.825+0000 7f616ccf5700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f616806dee0 0x7f6168151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.825+0000 7f616ccf5700 1 --2- 172.21.15.156:0/2954719777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6168071040 0x7f6168153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.825+0000 7f616ccf5700 1 -- 172.21.15.156:0/2954719777 >> 172.21.15.156:0/2954719777 conn(0x7f61680d0000 msgr2=0x7f6168156360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:44.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.825+0000 7f616ccf5700 1 -- 172.21.15.156:0/2954719777 shutdown_connections 2023-11-28T11:08:44.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.825+0000 7f616ccf5700 1 -- 172.21.15.156:0/2954719777 wait complete. 2023-11-28T11:08:44.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:15.021591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:16.332241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:44.842 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:44.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.965+0000 7fa585aa0700 1 Processor -- start 2023-11-28T11:08:44.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.965+0000 7fa585aa0700 1 -- start start 2023-11-28T11:08:44.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa585aa0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa58006ba90 0x7fa5800af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa585aa0700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa5800cbdf0 0x7fa5800ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa585aa0700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5800cc5f0 con 0x7fa58015cef0 2023-11-28T11:08:44.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa585aa0700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa58015d470 con 0x7fa58006ba90 2023-11-28T11:08:44.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa585aa0700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa58015d750 con 0x7fa5800cbdf0 2023-11-28T11:08:44.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa57effd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa58006ba90 0x7fa5800af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa57f7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa58015cef0 0x7fa58006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47496/0 (socket says 172.21.15.156:47496) 2023-11-28T11:08:44.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa57f7fe700 1 -- 172.21.15.156:0/1256279714 learned_addr learned my addr 172.21.15.156:0/1256279714 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:44.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa57ffff700 1 --2- 172.21.15.156:0/1256279714 >> v2:172.21.15.186:3300/0 conn(0x7fa5800cbdf0 0x7fa5800ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa57effd700 1 -- 172.21.15.156:0/1256279714 >> v2:172.21.15.186:3300/0 conn(0x7fa5800cbdf0 msgr2=0x7fa5800ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:44.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa57effd700 1 --2- 172.21.15.156:0/1256279714 >> v2:172.21.15.186:3300/0 conn(0x7fa5800cbdf0 0x7fa5800ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa57effd700 1 -- 172.21.15.156:0/1256279714 >> v1:172.21.15.156:6789/0 conn(0x7fa58015cef0 legacy=0x7fa58006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa57effd700 1 -- 172.21.15.156:0/1256279714 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58015da30 con 0x7fa58006ba90 2023-11-28T11:08:44.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa57e7fc700 1 -- 172.21.15.156:0/1256279714 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705195798 0 0) 0x7fa5800cc5f0 con 0x7fa58015cef0 2023-11-28T11:08:44.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.966+0000 7fa57effd700 1 --2- 172.21.15.156:0/1256279714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa58006ba90 0x7fa5800af400 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa580073b10 tx=0x7fa568009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3ab6b3ea79988 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.967+0000 7fa57e7fc700 1 -- 172.21.15.156:0/1256279714 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568011830 con 0x7fa58006ba90 2023-11-28T11:08:44.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.967+0000 7fa57e7fc700 1 -- 172.21.15.156:0/1256279714 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa568011990 con 0x7fa58006ba90 2023-11-28T11:08:44.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.967+0000 7fa57e7fc700 1 -- 172.21.15.156:0/1256279714 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568011d20 con 0x7fa58006ba90 2023-11-28T11:08:44.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.967+0000 7fa585aa0700 1 -- 172.21.15.156:0/1256279714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa58006ba90 msgr2=0x7fa5800af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.967+0000 7fa585aa0700 1 --2- 172.21.15.156:0/1256279714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa58006ba90 0x7fa5800af400 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa580073b10 tx=0x7fa568009b20 comp rx=0 tx=0).stop 2023-11-28T11:08:44.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.967+0000 7fa585aa0700 1 -- 172.21.15.156:0/1256279714 shutdown_connections 2023-11-28T11:08:44.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.967+0000 7fa585aa0700 1 --2- 172.21.15.156:0/1256279714 >> v2:172.21.15.186:3300/0 conn(0x7fa5800cbdf0 0x7fa5800ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.967+0000 7fa585aa0700 1 --2- 172.21.15.156:0/1256279714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa58006ba90 0x7fa5800af400 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.967+0000 7fa585aa0700 1 -- 172.21.15.156:0/1256279714 >> 172.21.15.156:0/1256279714 conn(0x7fa5800d0000 msgr2=0x7fa5800d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:44.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.967+0000 7fa585aa0700 1 -- 172.21.15.156:0/1256279714 shutdown_connections 2023-11-28T11:08:44.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.967+0000 7fa585aa0700 1 -- 172.21.15.156:0/1256279714 wait complete. 2023-11-28T11:08:44.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.969+0000 7fa585aa0700 1 Processor -- start 2023-11-28T11:08:44.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.969+0000 7fa585aa0700 1 -- start start 2023-11-28T11:08:44.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa585aa0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa58006ba90 0x7fa580156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa585aa0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5800cbdf0 0x7fa5800b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa585aa0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa58015cef0 0x7fa5800afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa585aa0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa580156ad0 con 0x7fa58006ba90 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa585aa0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800b4db0 con 0x7fa5800cbdf0 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa585aa0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800c7400 con 0x7fa58015cef0 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa57f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa58006ba90 0x7fa580156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa57f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa58006ba90 0x7fa580156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60856/0 (socket says 172.21.15.156:60856) 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa57f7fe700 1 -- 172.21.15.156:0/3763241014 learned_addr learned my addr 172.21.15.156:0/3763241014 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa57effd700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5800cbdf0 0x7fa5800b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa57ffff700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa58015cef0 0x7fa5800afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa57f7fe700 1 -- 172.21.15.156:0/3763241014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa58015cef0 msgr2=0x7fa5800afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa57f7fe700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa58015cef0 0x7fa5800afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa57f7fe700 1 -- 172.21.15.156:0/3763241014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5800cbdf0 msgr2=0x7fa5800b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa57f7fe700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5800cbdf0 0x7fa5800b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa57f7fe700 1 -- 172.21.15.156:0/3763241014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5680094a0 con 0x7fa58006ba90 2023-11-28T11:08:44.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa57ffff700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa58015cef0 0x7fa5800afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:44.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa57f7fe700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa58006ba90 0x7fa580156590 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa57000cc20 tx=0x7fa57000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa577fff700 1 -- 172.21.15.156:0/3763241014 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57000ac20 con 0x7fa58006ba90 2023-11-28T11:08:44.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa577fff700 1 -- 172.21.15.156:0/3763241014 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57000ad80 con 0x7fa58006ba90 2023-11-28T11:08:44.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.970+0000 7fa577fff700 1 -- 172.21.15.156:0/3763241014 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa570007310 con 0x7fa58006ba90 2023-11-28T11:08:44.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.971+0000 7fa585aa0700 1 -- 172.21.15.156:0/3763241014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58015dff0 con 0x7fa58006ba90 2023-11-28T11:08:44.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.971+0000 7fa585aa0700 1 -- 172.21.15.156:0/3763241014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58015dd10 con 0x7fa58006ba90 2023-11-28T11:08:44.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.971+0000 7fa585aa0700 1 -- 172.21.15.156:0/3763241014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5800cab90 con 0x7fa58006ba90 2023-11-28T11:08:44.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.972+0000 7fa577fff700 1 -- 172.21.15.156:0/3763241014 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa570010070 con 0x7fa58006ba90 2023-11-28T11:08:44.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.972+0000 7fa577fff700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa5600682f0 0x7fa56006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.972+0000 7fa57effd700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa5600682f0 0x7fa56006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.972+0000 7fa577fff700 1 -- 172.21.15.156:0/3763241014 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa570085670 con 0x7fa58006ba90 2023-11-28T11:08:44.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.973+0000 7fa57effd700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa5600682f0 0x7fa56006a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5680090b0 tx=0x7fa568008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:44.977+0000 7fa577fff700 1 -- 172.21.15.156:0/3763241014 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa570051ed0 con 0x7fa58006ba90 2023-11-28T11:08:45.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.158+0000 7fa585aa0700 1 -- 172.21.15.156:0/3763241014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5800ca430 con 0x7fa58006ba90 2023-11-28T11:08:45.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.158+0000 7fa577fff700 1 -- 172.21.15.156:0/3763241014 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa570018020 con 0x7fa58006ba90 2023-11-28T11:08:45.160 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:08:45.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.160+0000 7fa585aa0700 1 -- 172.21.15.156:0/3763241014 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa5600682f0 msgr2=0x7fa56006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.160+0000 7fa585aa0700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa5600682f0 0x7fa56006a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5680090b0 tx=0x7fa568008030 comp rx=0 tx=0).stop 2023-11-28T11:08:45.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.160+0000 7fa585aa0700 1 -- 172.21.15.156:0/3763241014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa58006ba90 msgr2=0x7fa580156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.160+0000 7fa585aa0700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa58006ba90 0x7fa580156590 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa57000cc20 tx=0x7fa57000a060 comp rx=0 tx=0).stop 2023-11-28T11:08:45.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.160+0000 7fa585aa0700 1 -- 172.21.15.156:0/3763241014 shutdown_connections 2023-11-28T11:08:45.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.160+0000 7fa585aa0700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa5600682f0 0x7fa56006a7a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.160+0000 7fa585aa0700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa58015cef0 0x7fa5800afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.160+0000 7fa585aa0700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa58006ba90 0x7fa580156590 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.160+0000 7fa585aa0700 1 --2- 172.21.15.156:0/3763241014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5800cbdf0 0x7fa5800b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.160+0000 7fa585aa0700 1 -- 172.21.15.156:0/3763241014 >> 172.21.15.156:0/3763241014 conn(0x7fa5800d0000 msgr2=0x7fa5800b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.161+0000 7fa585aa0700 1 -- 172.21.15.156:0/3763241014 shutdown_connections 2023-11-28T11:08:45.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.161+0000 7fa585aa0700 1 -- 172.21.15.156:0/3763241014 wait complete. 2023-11-28T11:08:45.174 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:45.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.298+0000 7f00e709e700 1 Processor -- start 2023-11-28T11:08:45.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.298+0000 7f00e709e700 1 -- start start 2023-11-28T11:08:45.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00e709e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00e0161340 0x7f00e00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00e709e700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f00e00d2320 0x7f00e00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00e709e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e00d1950 con 0x7f00e015c9b0 2023-11-28T11:08:45.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00e709e700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e00b5800 con 0x7f00e00d2320 2023-11-28T11:08:45.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00e709e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e00b5300 con 0x7f00e0161340 2023-11-28T11:08:45.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00e4e3a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f00e015c9b0 0x7f00e00b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47510/0 (socket says 172.21.15.156:47510) 2023-11-28T11:08:45.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00e4e3a700 1 -- 172.21.15.156:0/2909147014 learned_addr learned my addr 172.21.15.156:0/2909147014 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:45.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00dffff700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00e0161340 0x7f00e00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00e563b700 1 --2- 172.21.15.156:0/2909147014 >> v2:172.21.15.186:3300/0 conn(0x7f00e00d2320 0x7f00e00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411730784 0 0) 0x7f00e00d1950 con 0x7f00e015c9b0 2023-11-28T11:08:45.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d00035b0 con 0x7f00e015c9b0 2023-11-28T11:08:45.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2806691671 0 0) 0x7f00d00035b0 con 0x7f00e015c9b0 2023-11-28T11:08:45.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00e0161340 msgr2=0x7f00e00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00df7fe700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00e0161340 0x7f00e00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 >> v2:172.21.15.186:3300/0 conn(0x7f00e00d2320 msgr2=0x7f00e00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00df7fe700 1 --2- 172.21.15.156:0/2909147014 >> v2:172.21.15.186:3300/0 conn(0x7f00e00d2320 0x7f00e00cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e00b5520 con 0x7f00e015c9b0 2023-11-28T11:08:45.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f00d4001670 con 0x7f00e015c9b0 2023-11-28T11:08:45.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 >> v1:172.21.15.156:6789/0 conn(0x7f00e015c9b0 legacy=0x7f00e00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.299+0000 7f00df7fe700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00d0003a30 0x7f00d0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00df7fe700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00d0004850 0x7f00d0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00df7fe700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00d0005200 0x7f00d000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e00b5800 con 0x7f00d0003a30 2023-11-28T11:08:45.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e00b5300 con 0x7f00d0005200 2023-11-28T11:08:45.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d000bf80 con 0x7f00d0004850 2023-11-28T11:08:45.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00dffff700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00e0161340 0x7f00e00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:45.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00e563b700 1 --2- 172.21.15.156:0/2909147014 >> v2:172.21.15.186:3300/0 conn(0x7f00e00d2320 0x7f00e00cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00e4e3a700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00d0003a30 0x7f00d0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00dffff700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00d0004850 0x7f00d0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00e563b700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00d0005200 0x7f00d000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00e4e3a700 1 -- 172.21.15.156:0/2909147014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00d0004850 msgr2=0x7f00d0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00e4e3a700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00d0004850 0x7f00d0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00e4e3a700 1 -- 172.21.15.156:0/2909147014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00d0005200 msgr2=0x7f00d000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00e4e3a700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00d0005200 0x7f00d000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00e4e3a700 1 -- 172.21.15.156:0/2909147014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d00035b0 con 0x7f00d0003a30 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00e4e3a700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00d0003a30 0x7f00d0003e40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f00d400c240 tx=0x7f00d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7821292bb1db2fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d4007c10 con 0x7f00d0003a30 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d400cda0 con 0x7f00d0003a30 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.300+0000 7f00df7fe700 1 -- 172.21.15.156:0/2909147014 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d4005ba0 con 0x7f00d0003a30 2023-11-28T11:08:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.301+0000 7f00e709e700 1 -- 172.21.15.156:0/2909147014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00d0003a30 msgr2=0x7f00d0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.301+0000 7f00e709e700 1 --2- 172.21.15.156:0/2909147014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00d0003a30 0x7f00d0003e40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f00d400c240 tx=0x7f00d0005880 comp rx=0 tx=0).stop 2023-11-28T11:08:45.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.301+0000 7f00e563b700 1 -- 172.21.15.156:0/2909147014 reap_dead start 2023-11-28T11:08:45.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.301+0000 7f00e709e700 1 -- 172.21.15.156:0/2909147014 shutdown_connections 2023-11-28T11:08:45.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.301+0000 7f00e709e700 1 -- 172.21.15.156:0/2909147014 >> 172.21.15.156:0/2909147014 conn(0x7f00e0072600 msgr2=0x7f00e00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.301+0000 7f00e709e700 1 -- 172.21.15.156:0/2909147014 shutdown_connections 2023-11-28T11:08:45.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.301+0000 7f00e709e700 1 -- 172.21.15.156:0/2909147014 wait complete. 2023-11-28T11:08:45.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.302+0000 7f00e709e700 1 Processor -- start 2023-11-28T11:08:45.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.302+0000 7f00e709e700 1 -- start start 2023-11-28T11:08:45.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e709e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00d0003a30 0x7f00e0156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e709e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00e01527e0 0x7f00e0152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e709e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00e00c7160 0x7f00e00c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e709e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e00c7b10 con 0x7f00e00c7160 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e709e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e0151740 con 0x7f00d0003a30 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e709e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e0151a20 con 0x7f00e01527e0 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e563b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00e00c7160 0x7f00e00c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e563b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00e00c7160 0x7f00e00c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60880/0 (socket says 172.21.15.156:60880) 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e563b700 1 -- 172.21.15.156:0/3204279700 learned_addr learned my addr 172.21.15.156:0/3204279700 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e4e3a700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00d0003a30 0x7f00e0156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e563b700 1 -- 172.21.15.156:0/3204279700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00e01527e0 msgr2=0x7f00e0152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00dffff700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00e01527e0 0x7f00e0152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e563b700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00e01527e0 0x7f00e0152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e563b700 1 -- 172.21.15.156:0/3204279700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00d0003a30 msgr2=0x7f00e0156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e563b700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00d0003a30 0x7f00e0156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e563b700 1 -- 172.21.15.156:0/3204279700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e00d15e0 con 0x7f00e00c7160 2023-11-28T11:08:45.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e4e3a700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00d0003a30 0x7f00e0156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:45.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00e563b700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00e00c7160 0x7f00e00c75d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f00e008bfd0 tx=0x7f00d8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.303+0000 7f00dd7fa700 1 -- 172.21.15.156:0/3204279700 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d800e070 con 0x7f00e00c7160 2023-11-28T11:08:45.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.304+0000 7f00dd7fa700 1 -- 172.21.15.156:0/3204279700 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d8011cd0 con 0x7f00e00c7160 2023-11-28T11:08:45.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.304+0000 7f00e709e700 1 -- 172.21.15.156:0/3204279700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e00d1300 con 0x7f00e00c7160 2023-11-28T11:08:45.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.304+0000 7f00dd7fa700 1 -- 172.21.15.156:0/3204279700 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8011e60 con 0x7f00e00c7160 2023-11-28T11:08:45.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.304+0000 7f00e709e700 1 -- 172.21.15.156:0/3204279700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e015cd80 con 0x7f00e00c7160 2023-11-28T11:08:45.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.305+0000 7f00e709e700 1 -- 172.21.15.156:0/3204279700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e00745c0 con 0x7f00e00c7160 2023-11-28T11:08:45.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.305+0000 7f00dd7fa700 1 -- 172.21.15.156:0/3204279700 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f00d8003a20 con 0x7f00e00c7160 2023-11-28T11:08:45.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.305+0000 7f00dd7fa700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f00c40682f0 0x7f00c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.305+0000 7f00dd7fa700 1 -- 172.21.15.156:0/3204279700 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f00d8086540 con 0x7f00e00c7160 2023-11-28T11:08:45.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.305+0000 7f00e4e3a700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f00c40682f0 0x7f00c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.306+0000 7f00e4e3a700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f00c40682f0 0x7f00c406a7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f00d4015040 tx=0x7f00d4007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.309+0000 7f00dd7fa700 1 -- 172.21.15.156:0/3204279700 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f00d8052da0 con 0x7f00e00c7160 2023-11-28T11:08:45.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.489+0000 7f00e709e700 1 -- 172.21.15.156:0/3204279700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f00e00b90f0 con 0x7f00e00c7160 2023-11-28T11:08:45.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.489+0000 7f00dd7fa700 1 -- 172.21.15.156:0/3204279700 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f00d8003cb0 con 0x7f00e00c7160 2023-11-28T11:08:45.491 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:08:45.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.491+0000 7f00e709e700 1 -- 172.21.15.156:0/3204279700 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f00c40682f0 msgr2=0x7f00c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.491+0000 7f00e709e700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f00c40682f0 0x7f00c406a7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f00d4015040 tx=0x7f00d4007000 comp rx=0 tx=0).stop 2023-11-28T11:08:45.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.491+0000 7f00e709e700 1 -- 172.21.15.156:0/3204279700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00e00c7160 msgr2=0x7f00e00c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.491+0000 7f00e709e700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00e00c7160 0x7f00e00c75d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f00e008bfd0 tx=0x7f00d8009f90 comp rx=0 tx=0).stop 2023-11-28T11:08:45.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.492+0000 7f00e709e700 1 -- 172.21.15.156:0/3204279700 shutdown_connections 2023-11-28T11:08:45.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.492+0000 7f00e709e700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00e00c7160 0x7f00e00c75d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.492+0000 7f00e709e700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00d0003a30 0x7f00e0156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.492+0000 7f00e709e700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f00c40682f0 0x7f00c406a7a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.492+0000 7f00e709e700 1 --2- 172.21.15.156:0/3204279700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00e01527e0 0x7f00e0152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.492+0000 7f00e709e700 1 -- 172.21.15.156:0/3204279700 >> 172.21.15.156:0/3204279700 conn(0x7f00e0072600 msgr2=0x7f00e015ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.492+0000 7f00e709e700 1 -- 172.21.15.156:0/3204279700 shutdown_connections 2023-11-28T11:08:45.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.492+0000 7f00e709e700 1 -- 172.21.15.156:0/3204279700 wait complete. 2023-11-28T11:08:45.505 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:45.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.628+0000 7fe3e131d700 1 Processor -- start 2023-11-28T11:08:45.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.628+0000 7fe3e131d700 1 -- start start 2023-11-28T11:08:45.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3e131d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe3dc15c9b0 0x7fe3dc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3e131d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3dc0cd7c0 0x7fe3dc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3e131d700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3dc0d1950 con 0x7fe3dc161340 2023-11-28T11:08:45.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3e131d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3dc0b5800 con 0x7fe3dc15c9b0 2023-11-28T11:08:45.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3e131d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0ce140 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3daffd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe3dc15c9b0 0x7fe3dc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3da7fc700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fe3dc161340 0x7fe3dc0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:57892/0 (socket says 172.21.15.156:57892) 2023-11-28T11:08:45.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3da7fc700 1 -- 172.21.15.156:0/1721583060 learned_addr learned my addr 172.21.15.156:0/1721583060 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:45.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3db7fe700 1 --2- 172.21.15.156:0/1721583060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3dc0cd7c0 0x7fe3dc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3daffd700 1 -- 172.21.15.156:0/1721583060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3dc0cd7c0 msgr2=0x7fe3dc0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3daffd700 1 --2- 172.21.15.156:0/1721583060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3dc0cd7c0 0x7fe3dc0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3daffd700 1 -- 172.21.15.156:0/1721583060 >> v1:172.21.15.186:6789/0 conn(0x7fe3dc161340 legacy=0x7fe3dc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3daffd700 1 -- 172.21.15.156:0/1721583060 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc0d24f0 con 0x7fe3dc15c9b0 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.629+0000 7fe3d9ffb700 1 -- 172.21.15.156:0/1721583060 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157750992 0 0) 0x7fe3dc0d1950 con 0x7fe3dc161340 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.630+0000 7fe3daffd700 1 --2- 172.21.15.156:0/1721583060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3dc15c9b0 0x7fe3dc0b4bf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009840 tx=0x7fe3d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd8eb35ad142205 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.630+0000 7fe3d9ffb700 1 -- 172.21.15.156:0/1721583060 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0011830 con 0x7fe3dc15c9b0 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.630+0000 7fe3d9ffb700 1 -- 172.21.15.156:0/1721583060 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d0011990 con 0x7fe3dc15c9b0 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.630+0000 7fe3d9ffb700 1 -- 172.21.15.156:0/1721583060 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0011d20 con 0x7fe3dc15c9b0 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.630+0000 7fe3e131d700 1 -- 172.21.15.156:0/1721583060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3dc15c9b0 msgr2=0x7fe3dc0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.630+0000 7fe3e131d700 1 --2- 172.21.15.156:0/1721583060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3dc15c9b0 0x7fe3dc0b4bf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009840 tx=0x7fe3d0009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.630+0000 7fe3e131d700 1 -- 172.21.15.156:0/1721583060 shutdown_connections 2023-11-28T11:08:45.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.630+0000 7fe3e131d700 1 --2- 172.21.15.156:0/1721583060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3dc0cd7c0 0x7fe3dc0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.630+0000 7fe3e131d700 1 --2- 172.21.15.156:0/1721583060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3dc15c9b0 0x7fe3dc0b4bf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.630+0000 7fe3e131d700 1 -- 172.21.15.156:0/1721583060 >> 172.21.15.156:0/1721583060 conn(0x7fe3dc072600 msgr2=0x7fe3dc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.630+0000 7fe3e131d700 1 -- 172.21.15.156:0/1721583060 shutdown_connections 2023-11-28T11:08:45.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.630+0000 7fe3e131d700 1 -- 172.21.15.156:0/1721583060 wait complete. 2023-11-28T11:08:45.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.632+0000 7fe3e131d700 1 Processor -- start 2023-11-28T11:08:45.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.632+0000 7fe3e131d700 1 -- start start 2023-11-28T11:08:45.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.632+0000 7fe3e131d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0cd7c0 0x7fe3dc157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.632+0000 7fe3e131d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3dc15c9b0 0x7fe3dc1f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.632+0000 7fe3e131d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3dc161340 0x7fe3dc15e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.632+0000 7fe3e131d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc157f40 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.632+0000 7fe3e131d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc1f2ea0 con 0x7fe3dc161340 2023-11-28T11:08:45.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.632+0000 7fe3e131d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc1f3180 con 0x7fe3dc15c9b0 2023-11-28T11:08:45.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0cd7c0 0x7fe3dc157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0cd7c0 0x7fe3dc157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60890/0 (socket says 172.21.15.156:60890) 2023-11-28T11:08:45.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3daffd700 1 -- 172.21.15.156:0/3108641419 learned_addr learned my addr 172.21.15.156:0/3108641419 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:45.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3db7fe700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3dc161340 0x7fe3dc15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3da7fc700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3dc15c9b0 0x7fe3dc1f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3daffd700 1 -- 172.21.15.156:0/3108641419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3dc15c9b0 msgr2=0x7fe3dc1f2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3daffd700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3dc15c9b0 0x7fe3dc1f2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3daffd700 1 -- 172.21.15.156:0/3108641419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3dc161340 msgr2=0x7fe3dc15e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3daffd700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3dc161340 0x7fe3dc15e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3daffd700 1 -- 172.21.15.156:0/3108641419 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d00094a0 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3db7fe700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3dc161340 0x7fe3dc15e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:45.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3daffd700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0cd7c0 0x7fe3dc157a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009fd0 tx=0x7fe3d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3c37fe700 1 -- 172.21.15.156:0/3108641419 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00036a0 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3c37fe700 1 -- 172.21.15.156:0/3108641419 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d001e070 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3c37fe700 1 -- 172.21.15.156:0/3108641419 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0003a60 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3e131d700 1 -- 172.21.15.156:0/3108641419 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc0d1300 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.633+0000 7fe3e131d700 1 -- 172.21.15.156:0/3108641419 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc0b5520 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.634+0000 7fe3e131d700 1 -- 172.21.15.156:0/3108641419 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3dc0d15e0 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.635+0000 7fe3c37fe700 1 -- 172.21.15.156:0/3108641419 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe3d000f050 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.635+0000 7fe3c37fe700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe3c80682f0 0x7fe3c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.635+0000 7fe3c37fe700 1 -- 172.21.15.156:0/3108641419 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3d0086ce0 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.635+0000 7fe3da7fc700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe3c80682f0 0x7fe3c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.635+0000 7fe3da7fc700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe3c80682f0 0x7fe3c806a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0ca2f0 tx=0x7fe3cc007be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.639+0000 7fe3c37fe700 1 -- 172.21.15.156:0/3108641419 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe3d0053540 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.823+0000 7fe3e131d700 1 -- 172.21.15.156:0/3108641419 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3dc0b9180 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.823+0000 7fe3c37fe700 1 -- 172.21.15.156:0/3108641419 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3d0015020 con 0x7fe3dc0cd7c0 2023-11-28T11:08:45.824 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:08:45.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.825+0000 7fe3e131d700 1 -- 172.21.15.156:0/3108641419 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe3c80682f0 msgr2=0x7fe3c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.825+0000 7fe3e131d700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe3c80682f0 0x7fe3c806a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0ca2f0 tx=0x7fe3cc007be0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.825+0000 7fe3e131d700 1 -- 172.21.15.156:0/3108641419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0cd7c0 msgr2=0x7fe3dc157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.825+0000 7fe3e131d700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0cd7c0 0x7fe3dc157a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009fd0 tx=0x7fe3d0004750 comp rx=0 tx=0).stop 2023-11-28T11:08:45.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.825+0000 7fe3e131d700 1 -- 172.21.15.156:0/3108641419 shutdown_connections 2023-11-28T11:08:45.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.825+0000 7fe3e131d700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3dc161340 0x7fe3dc15e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.825+0000 7fe3e131d700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0cd7c0 0x7fe3dc157a00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.825+0000 7fe3e131d700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe3c80682f0 0x7fe3c806a7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.825+0000 7fe3e131d700 1 --2- 172.21.15.156:0/3108641419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3dc15c9b0 0x7fe3dc1f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.825+0000 7fe3e131d700 1 -- 172.21.15.156:0/3108641419 >> 172.21.15.156:0/3108641419 conn(0x7fe3dc072600 msgr2=0x7fe3dc15a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.825+0000 7fe3e131d700 1 -- 172.21.15.156:0/3108641419 shutdown_connections 2023-11-28T11:08:45.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.825+0000 7fe3e131d700 1 -- 172.21.15.156:0/3108641419 wait complete. 2023-11-28T11:08:45.838 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:45.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.960+0000 7f32f630f700 1 Processor -- start 2023-11-28T11:08:45.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.960+0000 7f32f630f700 1 -- start start 2023-11-28T11:08:45.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.961+0000 7f32f630f700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f32f0160740 0x7f32f00cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.961+0000 7f32f630f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f32f0160b10 0x7f32f00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.961+0000 7f32f630f700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f006bbb0 con 0x7f32f006b060 2023-11-28T11:08:45.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.961+0000 7f32f630f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f00ca720 con 0x7f32f0160b10 2023-11-28T11:08:45.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.961+0000 7f32f630f700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f00cc030 con 0x7f32f0160740 2023-11-28T11:08:45.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.961+0000 7f32ef7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f32f0160b10 0x7f32f00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.961+0000 7f32f48ac700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f32f006b060 0x7f32f006b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47548/0 (socket says 172.21.15.156:47548) 2023-11-28T11:08:45.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.961+0000 7f32f48ac700 1 -- 172.21.15.156:0/535841869 learned_addr learned my addr 172.21.15.156:0/535841869 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.961+0000 7f32effff700 1 --2- 172.21.15.156:0/535841869 >> v2:172.21.15.186:3300/0 conn(0x7f32f0160740 0x7f32f00cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32ef7fe700 1 -- 172.21.15.156:0/535841869 >> v2:172.21.15.186:3300/0 conn(0x7f32f0160740 msgr2=0x7f32f00cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32ef7fe700 1 --2- 172.21.15.156:0/535841869 >> v2:172.21.15.186:3300/0 conn(0x7f32f0160740 0x7f32f00cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32ef7fe700 1 -- 172.21.15.156:0/535841869 >> v1:172.21.15.156:6789/0 conn(0x7f32f006b060 legacy=0x7f32f006b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32ef7fe700 1 -- 172.21.15.156:0/535841869 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f00c9e10 con 0x7f32f0160b10 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32ef7fe700 1 --2- 172.21.15.156:0/535841869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f0160b10 0x7f32f00cd390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f32f00b23b0 tx=0x7f32e0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8af845c937b590d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32eeffd700 1 -- 172.21.15.156:0/535841869 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0011830 con 0x7f32f0160b10 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32eeffd700 1 -- 172.21.15.156:0/535841869 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e0011990 con 0x7f32f0160b10 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32eeffd700 1 -- 172.21.15.156:0/535841869 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0011d20 con 0x7f32f0160b10 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32f630f700 1 -- 172.21.15.156:0/535841869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f0160b10 msgr2=0x7f32f00cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32f630f700 1 --2- 172.21.15.156:0/535841869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f0160b10 0x7f32f00cd390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f32f00b23b0 tx=0x7f32e0009b20 comp rx=0 tx=0).stop 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32f630f700 1 -- 172.21.15.156:0/535841869 shutdown_connections 2023-11-28T11:08:45.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32f630f700 1 --2- 172.21.15.156:0/535841869 >> v2:172.21.15.186:3300/0 conn(0x7f32f0160740 0x7f32f00cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32f630f700 1 --2- 172.21.15.156:0/535841869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f0160b10 0x7f32f00cd390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32f630f700 1 -- 172.21.15.156:0/535841869 >> 172.21.15.156:0/535841869 conn(0x7f32f00b12b0 msgr2=0x7f32f00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32f630f700 1 -- 172.21.15.156:0/535841869 shutdown_connections 2023-11-28T11:08:45.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.962+0000 7f32f630f700 1 -- 172.21.15.156:0/535841869 wait complete. 2023-11-28T11:08:45.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.964+0000 7f32f630f700 1 Processor -- start 2023-11-28T11:08:45.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.964+0000 7f32f630f700 1 -- start start 2023-11-28T11:08:45.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32f630f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f006b060 0x7f32f00c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32f630f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0160740 0x7f32f015cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32f630f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f0160b10 0x7f32f015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32f630f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f00c6900 con 0x7f32f006b060 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32f630f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f00c4dd0 con 0x7f32f0160b10 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32f630f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f00c50b0 con 0x7f32f0160740 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32effff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f006b060 0x7f32f00c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32effff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f006b060 0x7f32f00c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60904/0 (socket says 172.21.15.156:60904) 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32effff700 1 -- 172.21.15.156:0/3875172525 learned_addr learned my addr 172.21.15.156:0/3875172525 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32f48ac700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f0160b10 0x7f32f015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32effff700 1 -- 172.21.15.156:0/3875172525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0160740 msgr2=0x7f32f015cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32effff700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0160740 0x7f32f015cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32effff700 1 -- 172.21.15.156:0/3875172525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f0160b10 msgr2=0x7f32f015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32effff700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f0160b10 0x7f32f015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32effff700 1 -- 172.21.15.156:0/3875172525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e00094a0 con 0x7f32f006b060 2023-11-28T11:08:45.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32f48ac700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f0160b10 0x7f32f015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:45.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32effff700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f006b060 0x7f32f00c5db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f32e400e810 tx=0x7f32e400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32ecff9700 1 -- 172.21.15.156:0/3875172525 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4013610 con 0x7f32f006b060 2023-11-28T11:08:45.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32ecff9700 1 -- 172.21.15.156:0/3875172525 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e4013770 con 0x7f32f006b060 2023-11-28T11:08:45.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32f630f700 1 -- 172.21.15.156:0/3875172525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f00ca0d0 con 0x7f32f006b060 2023-11-28T11:08:45.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32ecff9700 1 -- 172.21.15.156:0/3875172525 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4013b20 con 0x7f32f006b060 2023-11-28T11:08:45.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.965+0000 7f32f630f700 1 -- 172.21.15.156:0/3875172525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f006efc0 con 0x7f32f006b060 2023-11-28T11:08:45.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.966+0000 7f32f630f700 1 -- 172.21.15.156:0/3875172525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f00ca3b0 con 0x7f32f006b060 2023-11-28T11:08:45.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.967+0000 7f32ecff9700 1 -- 172.21.15.156:0/3875172525 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f32e4023070 con 0x7f32f006b060 2023-11-28T11:08:45.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.967+0000 7f32ecff9700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f32dc0682f0 0x7f32dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.967+0000 7f32ecff9700 1 -- 172.21.15.156:0/3875172525 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f32e40859a0 con 0x7f32f006b060 2023-11-28T11:08:45.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.967+0000 7f32ef7fe700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f32dc0682f0 0x7f32dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.968+0000 7f32ef7fe700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f32dc0682f0 0x7f32dc06a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f32f00c2e80 tx=0x7f32e0010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:45.971+0000 7f32ecff9700 1 -- 172.21.15.156:0/3875172525 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f32e4052180 con 0x7f32f006b060 2023-11-28T11:08:46.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.155+0000 7f32f630f700 1 -- 172.21.15.156:0/3875172525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f32f00c7130 con 0x7f32f006b060 2023-11-28T11:08:46.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.155+0000 7f32ecff9700 1 -- 172.21.15.156:0/3875172525 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f32e40557a0 con 0x7f32f006b060 2023-11-28T11:08:46.156 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:08:46.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.157+0000 7f32f630f700 1 -- 172.21.15.156:0/3875172525 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f32dc0682f0 msgr2=0x7f32dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.157+0000 7f32f630f700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f32dc0682f0 0x7f32dc06a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f32f00c2e80 tx=0x7f32e0010000 comp rx=0 tx=0).stop 2023-11-28T11:08:46.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.157+0000 7f32f630f700 1 -- 172.21.15.156:0/3875172525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f006b060 msgr2=0x7f32f00c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.157+0000 7f32f630f700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f006b060 0x7f32f00c5db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f32e400e810 tx=0x7f32e400ebd0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.157+0000 7f32f630f700 1 -- 172.21.15.156:0/3875172525 shutdown_connections 2023-11-28T11:08:46.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.157+0000 7f32f630f700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f0160b10 0x7f32f015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.157+0000 7f32f630f700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f006b060 0x7f32f00c5db0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.157+0000 7f32f630f700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f32dc0682f0 0x7f32dc06a7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.157+0000 7f32f630f700 1 --2- 172.21.15.156:0/3875172525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32f0160740 0x7f32f015cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.157+0000 7f32f630f700 1 -- 172.21.15.156:0/3875172525 >> 172.21.15.156:0/3875172525 conn(0x7f32f00b12b0 msgr2=0x7f32f015c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.157+0000 7f32f630f700 1 -- 172.21.15.156:0/3875172525 shutdown_connections 2023-11-28T11:08:46.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.157+0000 7f32f630f700 1 -- 172.21.15.156:0/3875172525 wait complete. 2023-11-28T11:08:46.170 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:46.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.290+0000 7f52211f5700 1 Processor -- start 2023-11-28T11:08:46.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.290+0000 7f52211f5700 1 -- start start 2023-11-28T11:08:46.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f52211f5700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f521c15cef0 0x7f521c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f52211f5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521c0cbdf0 0x7f521c06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f52211f5700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f521c06b8c0 0x7f521c071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f52211f5700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c06bd40 con 0x7f521c06b8c0 2023-11-28T11:08:46.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f52211f5700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c0cc5f0 con 0x7f521c15cef0 2023-11-28T11:08:46.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f52211f5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c15d470 con 0x7f521c0cbdf0 2023-11-28T11:08:46.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f521b59e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f521c06b8c0 0x7f521c071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f521b59e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f521c06b8c0 0x7f521c071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60916/0 (socket says 172.21.15.156:60916) 2023-11-28T11:08:46.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f521b59e700 1 -- 172.21.15.156:0/4081665527 learned_addr learned my addr 172.21.15.156:0/4081665527 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:46.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f521ad9d700 1 --2- 172.21.15.156:0/4081665527 >> v2:172.21.15.186:3300/0 conn(0x7f521c15cef0 0x7f521c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f521a59c700 1 --2- 172.21.15.156:0/4081665527 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521c0cbdf0 0x7f521c06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f521b59e700 1 -- 172.21.15.156:0/4081665527 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521c0cbdf0 msgr2=0x7f521c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:46.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f521b59e700 1 --2- 172.21.15.156:0/4081665527 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521c0cbdf0 0x7f521c06b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f521b59e700 1 -- 172.21.15.156:0/4081665527 >> v2:172.21.15.186:3300/0 conn(0x7f521c15cef0 msgr2=0x7f521c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:46.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f521b59e700 1 --2- 172.21.15.156:0/4081665527 >> v2:172.21.15.186:3300/0 conn(0x7f521c15cef0 0x7f521c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f521b59e700 1 -- 172.21.15.156:0/4081665527 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c15d750 con 0x7f521c06b8c0 2023-11-28T11:08:46.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f521ad9d700 1 --2- 172.21.15.156:0/4081665527 >> v2:172.21.15.186:3300/0 conn(0x7f521c15cef0 0x7f521c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:46.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.291+0000 7f521b59e700 1 --2- 172.21.15.156:0/4081665527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f521c06b8c0 0x7f521c071040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f521c0723f0 tx=0x7f521400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c4dde726775f8b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.292+0000 7f5219d9b700 1 -- 172.21.15.156:0/4081665527 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214010070 con 0x7f521c06b8c0 2023-11-28T11:08:46.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.292+0000 7f5219d9b700 1 -- 172.21.15.156:0/4081665527 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521400dd90 con 0x7f521c06b8c0 2023-11-28T11:08:46.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.292+0000 7f5219d9b700 1 -- 172.21.15.156:0/4081665527 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521400b310 con 0x7f521c06b8c0 2023-11-28T11:08:46.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.292+0000 7f52211f5700 1 -- 172.21.15.156:0/4081665527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f521c06b8c0 msgr2=0x7f521c071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.292+0000 7f52211f5700 1 --2- 172.21.15.156:0/4081665527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f521c06b8c0 0x7f521c071040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f521c0723f0 tx=0x7f521400d430 comp rx=0 tx=0).stop 2023-11-28T11:08:46.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.292+0000 7f52211f5700 1 -- 172.21.15.156:0/4081665527 shutdown_connections 2023-11-28T11:08:46.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.292+0000 7f52211f5700 1 --2- 172.21.15.156:0/4081665527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f521c06b8c0 0x7f521c071040 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.292+0000 7f52211f5700 1 --2- 172.21.15.156:0/4081665527 >> v2:172.21.15.186:3300/0 conn(0x7f521c15cef0 0x7f521c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.292+0000 7f52211f5700 1 --2- 172.21.15.156:0/4081665527 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521c0cbdf0 0x7f521c06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.292+0000 7f52211f5700 1 -- 172.21.15.156:0/4081665527 >> 172.21.15.156:0/4081665527 conn(0x7f521c0d0000 msgr2=0x7f521c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.292+0000 7f52211f5700 1 -- 172.21.15.156:0/4081665527 shutdown_connections 2023-11-28T11:08:46.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.292+0000 7f52211f5700 1 -- 172.21.15.156:0/4081665527 wait complete. 2023-11-28T11:08:46.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.294+0000 7f52211f5700 1 Processor -- start 2023-11-28T11:08:46.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.294+0000 7f52211f5700 1 -- start start 2023-11-28T11:08:46.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.294+0000 7f52211f5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f521c06b8c0 0x7f521c15b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f52211f5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521c0cbdf0 0x7f521c09b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f52211f5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521c15cef0 0x7f521c0c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f52211f5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c09b9b0 con 0x7f521c06b8c0 2023-11-28T11:08:46.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f52211f5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c09d9a0 con 0x7f521c15cef0 2023-11-28T11:08:46.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f52211f5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0acdf0 con 0x7f521c0cbdf0 2023-11-28T11:08:46.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f521ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f521c06b8c0 0x7f521c15b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f521ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f521c06b8c0 0x7f521c15b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60918/0 (socket says 172.21.15.156:60918) 2023-11-28T11:08:46.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f521ad9d700 1 -- 172.21.15.156:0/2764058978 learned_addr learned my addr 172.21.15.156:0/2764058978 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:46.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f521b59e700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521c15cef0 0x7f521c0c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f521a59c700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521c0cbdf0 0x7f521c09b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f521ad9d700 1 -- 172.21.15.156:0/2764058978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521c0cbdf0 msgr2=0x7f521c09b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f521ad9d700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521c0cbdf0 0x7f521c09b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f521ad9d700 1 -- 172.21.15.156:0/2764058978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521c15cef0 msgr2=0x7f521c0c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f521ad9d700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521c15cef0 0x7f521c0c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f521ad9d700 1 -- 172.21.15.156:0/2764058978 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521400bd40 con 0x7f521c06b8c0 2023-11-28T11:08:46.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f521ad9d700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f521c06b8c0 0x7f521c15b770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f521000c7f0 tx=0x7f521000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f52037fe700 1 -- 172.21.15.156:0/2764058978 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5210011800 con 0x7f521c06b8c0 2023-11-28T11:08:46.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f52037fe700 1 -- 172.21.15.156:0/2764058978 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5210011960 con 0x7f521c06b8c0 2023-11-28T11:08:46.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f52037fe700 1 -- 172.21.15.156:0/2764058978 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5210011d10 con 0x7f521c06b8c0 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f52211f5700 1 -- 172.21.15.156:0/2764058978 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c15dd10 con 0x7f521c06b8c0 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.295+0000 7f52211f5700 1 -- 172.21.15.156:0/2764058978 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c15da30 con 0x7f521c06b8c0 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.296+0000 7f52211f5700 1 -- 172.21.15.156:0/2764058978 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521c15dff0 con 0x7f521c06b8c0 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.297+0000 7f52037fe700 1 -- 172.21.15.156:0/2764058978 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5210021070 con 0x7f521c06b8c0 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.297+0000 7f52037fe700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52080682f0 0x7f520806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.297+0000 7f52037fe700 1 -- 172.21.15.156:0/2764058978 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5210085b90 con 0x7f521c06b8c0 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.297+0000 7f521a59c700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52080682f0 0x7f520806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.297+0000 7f521a59c700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52080682f0 0x7f520806a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f521c0cb0e0 tx=0x7f520c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.301+0000 7f52037fe700 1 -- 172.21.15.156:0/2764058978 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f52100523f0 con 0x7f521c06b8c0 2023-11-28T11:08:46.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.485+0000 7f52211f5700 1 -- 172.21.15.156:0/2764058978 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f521c0cb8c0 con 0x7f521c06b8c0 2023-11-28T11:08:46.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.486+0000 7f52037fe700 1 -- 172.21.15.156:0/2764058978 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f5210055a10 con 0x7f521c06b8c0 2023-11-28T11:08:46.490 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:46.491 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:18.354009+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:46.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.488+0000 7f52211f5700 1 -- 172.21.15.156:0/2764058978 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52080682f0 msgr2=0x7f520806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.488+0000 7f52211f5700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52080682f0 0x7f520806a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f521c0cb0e0 tx=0x7f520c008040 comp rx=0 tx=0).stop 2023-11-28T11:08:46.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.488+0000 7f52211f5700 1 -- 172.21.15.156:0/2764058978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f521c06b8c0 msgr2=0x7f521c15b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.488+0000 7f52211f5700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f521c06b8c0 0x7f521c15b770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f521000c7f0 tx=0x7f521000cbb0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.488+0000 7f52211f5700 1 -- 172.21.15.156:0/2764058978 shutdown_connections 2023-11-28T11:08:46.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.488+0000 7f52211f5700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521c15cef0 0x7f521c0c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.488+0000 7f52211f5700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f521c06b8c0 0x7f521c15b770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.488+0000 7f52211f5700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52080682f0 0x7f520806a7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.488+0000 7f52211f5700 1 --2- 172.21.15.156:0/2764058978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521c0cbdf0 0x7f521c09b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.488+0000 7f52211f5700 1 -- 172.21.15.156:0/2764058978 >> 172.21.15.156:0/2764058978 conn(0x7f521c0d0000 msgr2=0x7f521c0b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.488+0000 7f52211f5700 1 -- 172.21.15.156:0/2764058978 shutdown_connections 2023-11-28T11:08:46.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.488+0000 7f52211f5700 1 -- 172.21.15.156:0/2764058978 wait complete. 2023-11-28T11:08:46.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:15.021591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:16.332241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:46.503 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:46.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.627+0000 7f52bd784700 1 Processor -- start 2023-11-28T11:08:46.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.628+0000 7f52bd784700 1 -- start start 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.628+0000 7f52bd784700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52b8071040 0x7f52b8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.628+0000 7f52bd784700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b8071950 con 0x7f52b8160540 2023-11-28T11:08:46.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.628+0000 7f52bd784700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b815d060 con 0x7f52b806dee0 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.628+0000 7f52bd784700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b815d340 con 0x7f52b8071040 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.628+0000 7f52b77fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f52b8160540 0x7f52b81609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47576/0 (socket says 172.21.15.156:47576) 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b77fe700 1 -- 172.21.15.156:0/200982357 learned_addr learned my addr 172.21.15.156:0/200982357 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b6ffd700 1 --2- 172.21.15.156:0/200982357 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52b8071040 0x7f52b8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b5ffb700 1 -- 172.21.15.156:0/200982357 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585409570 0 0) 0x7f52b8071950 con 0x7f52b8160540 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b5ffb700 1 -- 172.21.15.156:0/200982357 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52a00035b0 con 0x7f52b8160540 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b5ffb700 1 -- 172.21.15.156:0/200982357 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804199884 0 0) 0x7f52b815d060 con 0x7f52b806dee0 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b5ffb700 1 -- 172.21.15.156:0/200982357 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b8071950 con 0x7f52b806dee0 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b5ffb700 1 -- 172.21.15.156:0/200982357 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4018540431 0 0) 0x7f52a00035b0 con 0x7f52b8160540 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b6ffd700 1 -- 172.21.15.156:0/200982357 >> v1:172.21.15.186:6789/0 conn(0x7f52b806dee0 legacy=0x7f52b806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b6ffd700 1 -- 172.21.15.156:0/200982357 >> v1:172.21.15.156:6789/0 conn(0x7f52b8160540 legacy=0x7f52b81609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b6ffd700 1 -- 172.21.15.156:0/200982357 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b815d620 con 0x7f52b8071040 2023-11-28T11:08:46.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b6ffd700 1 --2- 172.21.15.156:0/200982357 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52b8071040 0x7f52b8071410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009870 tx=0x7f52ac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1181aedc6c2f9c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b5ffb700 1 -- 172.21.15.156:0/200982357 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00019f0 con 0x7f52b8071040 2023-11-28T11:08:46.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b5ffb700 1 -- 172.21.15.156:0/200982357 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac004030 con 0x7f52b8071040 2023-11-28T11:08:46.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.629+0000 7f52b5ffb700 1 -- 172.21.15.156:0/200982357 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0043c0 con 0x7f52b8071040 2023-11-28T11:08:46.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.630+0000 7f52bd784700 1 -- 172.21.15.156:0/200982357 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52b8071040 msgr2=0x7f52b8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.630+0000 7f52bd784700 1 --2- 172.21.15.156:0/200982357 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52b8071040 0x7f52b8071410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009870 tx=0x7f52ac009b80 comp rx=0 tx=0).stop 2023-11-28T11:08:46.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.630+0000 7f52bd784700 1 -- 172.21.15.156:0/200982357 shutdown_connections 2023-11-28T11:08:46.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.630+0000 7f52bd784700 1 --2- 172.21.15.156:0/200982357 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52b8071040 0x7f52b8071410 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.630+0000 7f52bd784700 1 -- 172.21.15.156:0/200982357 >> 172.21.15.156:0/200982357 conn(0x7f52b80d0000 msgr2=0x7f52b80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.630+0000 7f52bd784700 1 -- 172.21.15.156:0/200982357 shutdown_connections 2023-11-28T11:08:46.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.630+0000 7f52bd784700 1 -- 172.21.15.156:0/200982357 wait complete. 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.631+0000 7f52bd784700 1 Processor -- start 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.631+0000 7f52bd784700 1 -- start start 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.632+0000 7f52bd784700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b806dee0 0x7f52b80b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.632+0000 7f52bd784700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52b8071040 0x7f52b815bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.632+0000 7f52bd784700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52b8160540 0x7f52b80b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.632+0000 7f52bd784700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b8151010 con 0x7f52b806dee0 2023-11-28T11:08:46.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.632+0000 7f52bd784700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b815a6d0 con 0x7f52b8071040 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.632+0000 7f52bd784700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b80b49c0 con 0x7f52b8160540 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.632+0000 7f52b6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b806dee0 0x7f52b80b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.632+0000 7f52b6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b806dee0 0x7f52b80b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60932/0 (socket says 172.21.15.156:60932) 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.632+0000 7f52b6ffd700 1 -- 172.21.15.156:0/81663833 learned_addr learned my addr 172.21.15.156:0/81663833 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.632+0000 7f52b67fc700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52b8071040 0x7f52b815bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.632+0000 7f52b77fe700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52b8160540 0x7f52b80b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.633+0000 7f52b6ffd700 1 -- 172.21.15.156:0/81663833 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52b8160540 msgr2=0x7f52b80b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.633+0000 7f52b6ffd700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52b8160540 0x7f52b80b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.633+0000 7f52b6ffd700 1 -- 172.21.15.156:0/81663833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52b8071040 msgr2=0x7f52b815bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.633+0000 7f52b6ffd700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52b8071040 0x7f52b815bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.633+0000 7f52b6ffd700 1 -- 172.21.15.156:0/81663833 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac0094a0 con 0x7f52b806dee0 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.633+0000 7f52b6ffd700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b806dee0 0x7f52b80b4350 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52ac000c00 tx=0x7f52ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.633+0000 7f529f7fe700 1 -- 172.21.15.156:0/81663833 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac003d00 con 0x7f52b806dee0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.633+0000 7f529f7fe700 1 -- 172.21.15.156:0/81663833 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac01f070 con 0x7f52b806dee0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.633+0000 7f529f7fe700 1 -- 172.21.15.156:0/81663833 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac011870 con 0x7f52b806dee0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.633+0000 7f52bd784700 1 -- 172.21.15.156:0/81663833 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b815ccd0 con 0x7f52b806dee0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.633+0000 7f52bd784700 1 -- 172.21.15.156:0/81663833 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b815d900 con 0x7f52b806dee0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.634+0000 7f52bd784700 1 -- 172.21.15.156:0/81663833 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b806a270 con 0x7f52b806dee0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.634+0000 7f529f7fe700 1 -- 172.21.15.156:0/81663833 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f52ac02a020 con 0x7f52b806dee0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.635+0000 7f529f7fe700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52a40682f0 0x7f52a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.635+0000 7f529f7fe700 1 -- 172.21.15.156:0/81663833 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52ac087040 con 0x7f52b806dee0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.635+0000 7f52b67fc700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52a40682f0 0x7f52a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.635+0000 7f52b67fc700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52a40682f0 0x7f52a406a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f52b80b1640 tx=0x7f52a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.639+0000 7f529f7fe700 1 -- 172.21.15.156:0/81663833 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f52ac0538a0 con 0x7f52b806dee0 2023-11-28T11:08:46.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.822+0000 7f52bd784700 1 -- 172.21.15.156:0/81663833 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52b80cd2d0 con 0x7f52b806dee0 2023-11-28T11:08:46.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.823+0000 7f529f7fe700 1 -- 172.21.15.156:0/81663833 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52ac056ec0 con 0x7f52b806dee0 2023-11-28T11:08:46.824 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:08:46.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.825+0000 7f52bd784700 1 -- 172.21.15.156:0/81663833 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52a40682f0 msgr2=0x7f52a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.825+0000 7f52bd784700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52a40682f0 0x7f52a406a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f52b80b1640 tx=0x7f52a8009040 comp rx=0 tx=0).stop 2023-11-28T11:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.825+0000 7f52bd784700 1 -- 172.21.15.156:0/81663833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b806dee0 msgr2=0x7f52b80b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.825+0000 7f52bd784700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b806dee0 0x7f52b80b4350 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52ac000c00 tx=0x7f52ac004750 comp rx=0 tx=0).stop 2023-11-28T11:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.825+0000 7f52bd784700 1 -- 172.21.15.156:0/81663833 shutdown_connections 2023-11-28T11:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.825+0000 7f52bd784700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52a40682f0 0x7f52a406a7a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.825+0000 7f52bd784700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52b8160540 0x7f52b80b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.825+0000 7f52bd784700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b806dee0 0x7f52b80b4350 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.825+0000 7f52bd784700 1 --2- 172.21.15.156:0/81663833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52b8071040 0x7f52b815bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.825+0000 7f52bd784700 1 -- 172.21.15.156:0/81663833 >> 172.21.15.156:0/81663833 conn(0x7f52b80d0000 msgr2=0x7f52b80cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.825+0000 7f52bd784700 1 -- 172.21.15.156:0/81663833 shutdown_connections 2023-11-28T11:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.825+0000 7f52bd784700 1 -- 172.21.15.156:0/81663833 wait complete. 2023-11-28T11:08:46.839 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:46.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.961+0000 7ff88291b700 1 Processor -- start 2023-11-28T11:08:46.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.961+0000 7ff88291b700 1 -- start start 2023-11-28T11:08:46.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff88291b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87c071040 0x7ff87c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff88291b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff87c06dee0 0x7ff87c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff88291b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff87c06cdc0 0x7ff87c06d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff88291b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87c071950 con 0x7ff87c06cdc0 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff88291b700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87c15d060 con 0x7ff87c06dee0 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff88291b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c15d340 con 0x7ff87c071040 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff880eb8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff87c06cdc0 0x7ff87c06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff880eb8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff87c06cdc0 0x7ff87c06d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60944/0 (socket says 172.21.15.156:60944) 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff880eb8700 1 -- 172.21.15.156:0/1026982674 learned_addr learned my addr 172.21.15.156:0/1026982674 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff87b7fe700 1 --2- 172.21.15.156:0/1026982674 >> v2:172.21.15.186:3300/0 conn(0x7ff87c06dee0 0x7ff87c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff87bfff700 1 --2- 172.21.15.156:0/1026982674 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87c071040 0x7ff87c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff880eb8700 1 -- 172.21.15.156:0/1026982674 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87c071040 msgr2=0x7ff87c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff880eb8700 1 --2- 172.21.15.156:0/1026982674 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87c071040 0x7ff87c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff880eb8700 1 -- 172.21.15.156:0/1026982674 >> v2:172.21.15.186:3300/0 conn(0x7ff87c06dee0 msgr2=0x7ff87c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:46.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff880eb8700 1 --2- 172.21.15.156:0/1026982674 >> v2:172.21.15.186:3300/0 conn(0x7ff87c06dee0 0x7ff87c06e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff880eb8700 1 -- 172.21.15.156:0/1026982674 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c15d620 con 0x7ff87c06cdc0 2023-11-28T11:08:46.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.962+0000 7ff880eb8700 1 --2- 172.21.15.156:0/1026982674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff87c06cdc0 0x7ff87c06d240 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff87c09e140 tx=0x7ff87400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b51cf1562d7bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.963+0000 7ff87affd700 1 -- 172.21.15.156:0/1026982674 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87400e070 con 0x7ff87c06cdc0 2023-11-28T11:08:46.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.963+0000 7ff87affd700 1 -- 172.21.15.156:0/1026982674 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87400b910 con 0x7ff87c06cdc0 2023-11-28T11:08:46.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.963+0000 7ff87affd700 1 -- 172.21.15.156:0/1026982674 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87400bcc0 con 0x7ff87c06cdc0 2023-11-28T11:08:46.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.963+0000 7ff88291b700 1 -- 172.21.15.156:0/1026982674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff87c06cdc0 msgr2=0x7ff87c06d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.963+0000 7ff88291b700 1 --2- 172.21.15.156:0/1026982674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff87c06cdc0 0x7ff87c06d240 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff87c09e140 tx=0x7ff87400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:46.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.963+0000 7ff88291b700 1 -- 172.21.15.156:0/1026982674 shutdown_connections 2023-11-28T11:08:46.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.963+0000 7ff88291b700 1 --2- 172.21.15.156:0/1026982674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff87c06cdc0 0x7ff87c06d240 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.963+0000 7ff88291b700 1 --2- 172.21.15.156:0/1026982674 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87c071040 0x7ff87c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.963+0000 7ff88291b700 1 --2- 172.21.15.156:0/1026982674 >> v2:172.21.15.186:3300/0 conn(0x7ff87c06dee0 0x7ff87c06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.963+0000 7ff88291b700 1 -- 172.21.15.156:0/1026982674 >> 172.21.15.156:0/1026982674 conn(0x7ff87c0d0000 msgr2=0x7ff87c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.963+0000 7ff88291b700 1 -- 172.21.15.156:0/1026982674 shutdown_connections 2023-11-28T11:08:46.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.963+0000 7ff88291b700 1 -- 172.21.15.156:0/1026982674 wait complete. 2023-11-28T11:08:46.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.964+0000 7ff88291b700 1 Processor -- start 2023-11-28T11:08:46.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.965+0000 7ff88291b700 1 -- start start 2023-11-28T11:08:46.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.965+0000 7ff88291b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff87c06cdc0 0x7ff87c144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.965+0000 7ff88291b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87c06dee0 0x7ff87c0c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.965+0000 7ff88291b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff87c071040 0x7ff87c15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.965+0000 7ff88291b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c1454e0 con 0x7ff87c06cdc0 2023-11-28T11:08:46.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.965+0000 7ff88291b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0c64b0 con 0x7ff87c071040 2023-11-28T11:08:46.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.965+0000 7ff88291b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0c5ae0 con 0x7ff87c06dee0 2023-11-28T11:08:46.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.965+0000 7ff87bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff87c06cdc0 0x7ff87c144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.965+0000 7ff87bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff87c06cdc0 0x7ff87c144fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60946/0 (socket says 172.21.15.156:60946) 2023-11-28T11:08:46.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.965+0000 7ff87bfff700 1 -- 172.21.15.156:0/3218735843 learned_addr learned my addr 172.21.15.156:0/3218735843 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:46.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff87b7fe700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87c06dee0 0x7ff87c0c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff880eb8700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff87c071040 0x7ff87c15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff87bfff700 1 -- 172.21.15.156:0/3218735843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87c06dee0 msgr2=0x7ff87c0c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff87bfff700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87c06dee0 0x7ff87c0c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff87bfff700 1 -- 172.21.15.156:0/3218735843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff87c071040 msgr2=0x7ff87c15bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff87bfff700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff87c071040 0x7ff87c15bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff87bfff700 1 -- 172.21.15.156:0/3218735843 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff874009b30 con 0x7ff87c06cdc0 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff87bfff700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff87c06cdc0 0x7ff87c144fa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff87000e8d0 tx=0x7ff87000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff878ff9700 1 -- 172.21.15.156:0/3218735843 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870009590 con 0x7ff87c06cdc0 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff878ff9700 1 -- 172.21.15.156:0/3218735843 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87000eed0 con 0x7ff87c06cdc0 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff88291b700 1 -- 172.21.15.156:0/3218735843 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c06e890 con 0x7ff87c06cdc0 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff878ff9700 1 -- 172.21.15.156:0/3218735843 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87000ca50 con 0x7ff87c06cdc0 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.966+0000 7ff88291b700 1 -- 172.21.15.156:0/3218735843 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87c15d900 con 0x7ff87c06cdc0 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.967+0000 7ff88291b700 1 -- 172.21.15.156:0/3218735843 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87c15ccd0 con 0x7ff87c06cdc0 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.967+0000 7ff878ff9700 1 -- 172.21.15.156:0/3218735843 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff870019030 con 0x7ff87c06cdc0 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.968+0000 7ff878ff9700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff8680682f0 0x7ff86806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.968+0000 7ff878ff9700 1 -- 172.21.15.156:0/3218735843 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff870085ab0 con 0x7ff87c06cdc0 2023-11-28T11:08:46.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.968+0000 7ff87b7fe700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff8680682f0 0x7ff86806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.968+0000 7ff87b7fe700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff8680682f0 0x7ff86806a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff87c1f6190 tx=0x7ff86c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:46.972+0000 7ff878ff9700 1 -- 172.21.15.156:0/3218735843 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff870052310 con 0x7ff87c06cdc0 2023-11-28T11:08:47.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.155+0000 7ff88291b700 1 -- 172.21.15.156:0/3218735843 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff87c0720a0 con 0x7ff87c06cdc0 2023-11-28T11:08:47.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.155+0000 7ff878ff9700 1 -- 172.21.15.156:0/3218735843 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff870055930 con 0x7ff87c06cdc0 2023-11-28T11:08:47.157 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:08:47.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.157+0000 7ff88291b700 1 -- 172.21.15.156:0/3218735843 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff8680682f0 msgr2=0x7ff86806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.157+0000 7ff88291b700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff8680682f0 0x7ff86806a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff87c1f6190 tx=0x7ff86c008040 comp rx=0 tx=0).stop 2023-11-28T11:08:47.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.157+0000 7ff88291b700 1 -- 172.21.15.156:0/3218735843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff87c06cdc0 msgr2=0x7ff87c144fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.157+0000 7ff88291b700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff87c06cdc0 0x7ff87c144fa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff87000e8d0 tx=0x7ff87000ec90 comp rx=0 tx=0).stop 2023-11-28T11:08:47.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.158+0000 7ff88291b700 1 -- 172.21.15.156:0/3218735843 shutdown_connections 2023-11-28T11:08:47.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.158+0000 7ff88291b700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff87c071040 0x7ff87c15bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.158+0000 7ff88291b700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff87c06cdc0 0x7ff87c144fa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.158+0000 7ff88291b700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff8680682f0 0x7ff86806a7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.158+0000 7ff88291b700 1 --2- 172.21.15.156:0/3218735843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87c06dee0 0x7ff87c0c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.158+0000 7ff88291b700 1 -- 172.21.15.156:0/3218735843 >> 172.21.15.156:0/3218735843 conn(0x7ff87c0d0000 msgr2=0x7ff87c15e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.158+0000 7ff88291b700 1 -- 172.21.15.156:0/3218735843 shutdown_connections 2023-11-28T11:08:47.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.158+0000 7ff88291b700 1 -- 172.21.15.156:0/3218735843 wait complete. 2023-11-28T11:08:47.171 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:47.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.294+0000 7f67fdd55700 1 Processor -- start 2023-11-28T11:08:47.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.294+0000 7f67fdd55700 1 -- start start 2023-11-28T11:08:47.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.294+0000 7f67fdd55700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f67f806dee0 0x7f67f806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.294+0000 7f67fdd55700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f67f8160540 0x7f67f8160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.294+0000 7f67fdd55700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f815d060 con 0x7f67f8071040 2023-11-28T11:08:47.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.294+0000 7f67fdd55700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f815d340 con 0x7f67f806dee0 2023-11-28T11:08:47.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.294+0000 7f67fdd55700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f815d620 con 0x7f67f8160540 2023-11-28T11:08:47.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.294+0000 7f67f6ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f67f806dee0 0x7f67f806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f77fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f67f8071040 0x7f67f8071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47600/0 (socket says 172.21.15.156:47600) 2023-11-28T11:08:47.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f6ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f67f806dee0 0x7f67f806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60956/0 (socket says 172.21.15.156:60956) 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f6ffd700 1 -- 172.21.15.156:0/200949503 learned_addr learned my addr 172.21.15.156:0/200949503 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f7fff700 1 --2- 172.21.15.156:0/200949503 >> v2:172.21.15.186:3300/0 conn(0x7f67f8160540 0x7f67f8160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f6ffd700 1 -- 172.21.15.156:0/200949503 >> v2:172.21.15.186:3300/0 conn(0x7f67f8160540 msgr2=0x7f67f8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f6ffd700 1 --2- 172.21.15.156:0/200949503 >> v2:172.21.15.186:3300/0 conn(0x7f67f8160540 0x7f67f8160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f6ffd700 1 -- 172.21.15.156:0/200949503 >> v1:172.21.15.156:6789/0 conn(0x7f67f8071040 legacy=0x7f67f8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f6ffd700 1 -- 172.21.15.156:0/200949503 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f815d900 con 0x7f67f806dee0 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f67fc700 1 -- 172.21.15.156:0/200949503 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229720907 0 0) 0x7f67f815d060 con 0x7f67f8071040 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f6ffd700 1 --2- 172.21.15.156:0/200949503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f806dee0 0x7f67f806e330 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f67f80c3a10 tx=0x7f67e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78befa16212f921b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f67fc700 1 -- 172.21.15.156:0/200949503 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800b870 con 0x7f67f806dee0 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f67fc700 1 -- 172.21.15.156:0/200949503 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e800b9d0 con 0x7f67f806dee0 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.295+0000 7f67f67fc700 1 -- 172.21.15.156:0/200949503 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800bd60 con 0x7f67f806dee0 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.296+0000 7f67fdd55700 1 -- 172.21.15.156:0/200949503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f806dee0 msgr2=0x7f67f806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.296+0000 7f67fdd55700 1 --2- 172.21.15.156:0/200949503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f806dee0 0x7f67f806e330 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f67f80c3a10 tx=0x7f67e8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.296+0000 7f67fdd55700 1 -- 172.21.15.156:0/200949503 shutdown_connections 2023-11-28T11:08:47.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.296+0000 7f67fdd55700 1 --2- 172.21.15.156:0/200949503 >> v2:172.21.15.186:3300/0 conn(0x7f67f8160540 0x7f67f8160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.296+0000 7f67fdd55700 1 --2- 172.21.15.156:0/200949503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f806dee0 0x7f67f806e330 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.296+0000 7f67fdd55700 1 -- 172.21.15.156:0/200949503 >> 172.21.15.156:0/200949503 conn(0x7f67f80d0000 msgr2=0x7f67f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.296+0000 7f67fdd55700 1 -- 172.21.15.156:0/200949503 shutdown_connections 2023-11-28T11:08:47.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.296+0000 7f67fdd55700 1 -- 172.21.15.156:0/200949503 wait complete. 2023-11-28T11:08:47.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.297+0000 7f67fdd55700 1 Processor -- start 2023-11-28T11:08:47.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.297+0000 7f67fdd55700 1 -- start start 2023-11-28T11:08:47.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67fdd55700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67f806dee0 0x7f67f815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67fdd55700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f8071040 0x7f67f81f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67fdd55700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f8160540 0x7f67f814b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67fdd55700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f81f63b0 con 0x7f67f8071040 2023-11-28T11:08:47.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67fdd55700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f81f5dc0 con 0x7f67f8160540 2023-11-28T11:08:47.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67fdd55700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f8148cd0 con 0x7f67f806dee0 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67f6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f8071040 0x7f67f81f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67f6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f8071040 0x7f67f81f67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60962/0 (socket says 172.21.15.156:60962) 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67f6ffd700 1 -- 172.21.15.156:0/2965185914 learned_addr learned my addr 172.21.15.156:0/2965185914 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67f7fff700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f8160540 0x7f67f814b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67f77fe700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67f806dee0 0x7f67f815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67f6ffd700 1 -- 172.21.15.156:0/2965185914 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67f806dee0 msgr2=0x7f67f815c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67f6ffd700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67f806dee0 0x7f67f815c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67f6ffd700 1 -- 172.21.15.156:0/2965185914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f8160540 msgr2=0x7f67f814b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67f6ffd700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f8160540 0x7f67f814b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67f6ffd700 1 -- 172.21.15.156:0/2965185914 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e8009550 con 0x7f67f8071040 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67f7fff700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f8160540 0x7f67f814b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.298+0000 7f67f6ffd700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f8071040 0x7f67f81f67d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f67e8000c00 tx=0x7f67e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.299+0000 7f67effff700 1 -- 172.21.15.156:0/2965185914 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8004520 con 0x7f67f8071040 2023-11-28T11:08:47.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.299+0000 7f67effff700 1 -- 172.21.15.156:0/2965185914 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e801e070 con 0x7f67f8071040 2023-11-28T11:08:47.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.299+0000 7f67fdd55700 1 -- 172.21.15.156:0/2965185914 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f815ccd0 con 0x7f67f8071040 2023-11-28T11:08:47.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.299+0000 7f67effff700 1 -- 172.21.15.156:0/2965185914 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e80038e0 con 0x7f67f8071040 2023-11-28T11:08:47.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.299+0000 7f67fdd55700 1 -- 172.21.15.156:0/2965185914 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f806e870 con 0x7f67f8071040 2023-11-28T11:08:47.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.300+0000 7f67fdd55700 1 -- 172.21.15.156:0/2965185914 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f806a270 con 0x7f67f8071040 2023-11-28T11:08:47.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.300+0000 7f67effff700 1 -- 172.21.15.156:0/2965185914 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f67e800f050 con 0x7f67f8071040 2023-11-28T11:08:47.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.300+0000 7f67effff700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67d80682f0 0x7f67d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.301+0000 7f67f77fe700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67d80682f0 0x7f67d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.301+0000 7f67effff700 1 -- 172.21.15.156:0/2965185914 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67e8020070 con 0x7f67f8071040 2023-11-28T11:08:47.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.301+0000 7f67f77fe700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67d80682f0 0x7f67d806a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f67e0000f60 tx=0x7f67e0007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.305+0000 7f67effff700 1 -- 172.21.15.156:0/2965185914 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f67e80538e0 con 0x7f67f8071040 2023-11-28T11:08:47.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.484+0000 7f67fdd55700 1 -- 172.21.15.156:0/2965185914 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67f81f64f0 con 0x7f67f8071040 2023-11-28T11:08:47.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.485+0000 7f67effff700 1 -- 172.21.15.156:0/2965185914 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67e8015020 con 0x7f67f8071040 2023-11-28T11:08:47.486 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:08:47.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.487+0000 7f67fdd55700 1 -- 172.21.15.156:0/2965185914 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67d80682f0 msgr2=0x7f67d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.487+0000 7f67fdd55700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67d80682f0 0x7f67d806a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f67e0000f60 tx=0x7f67e0007c90 comp rx=0 tx=0).stop 2023-11-28T11:08:47.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.487+0000 7f67fdd55700 1 -- 172.21.15.156:0/2965185914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f8071040 msgr2=0x7f67f81f67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.487+0000 7f67fdd55700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f8071040 0x7f67f81f67d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f67e8000c00 tx=0x7f67e8004750 comp rx=0 tx=0).stop 2023-11-28T11:08:47.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.487+0000 7f67fdd55700 1 -- 172.21.15.156:0/2965185914 shutdown_connections 2023-11-28T11:08:47.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.487+0000 7f67fdd55700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f8160540 0x7f67f814b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.487+0000 7f67fdd55700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67d80682f0 0x7f67d806a7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.487+0000 7f67fdd55700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67f806dee0 0x7f67f815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.487+0000 7f67fdd55700 1 --2- 172.21.15.156:0/2965185914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f8071040 0x7f67f81f67d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.487+0000 7f67fdd55700 1 -- 172.21.15.156:0/2965185914 >> 172.21.15.156:0/2965185914 conn(0x7f67f80d0000 msgr2=0x7f67f80aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.487+0000 7f67fdd55700 1 -- 172.21.15.156:0/2965185914 shutdown_connections 2023-11-28T11:08:47.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.487+0000 7f67fdd55700 1 -- 172.21.15.156:0/2965185914 wait complete. 2023-11-28T11:08:47.501 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:47.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.622+0000 7f0c6841f700 1 Processor -- start 2023-11-28T11:08:47.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.622+0000 7f0c6841f700 1 -- start start 2023-11-28T11:08:47.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c6841f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0c600cd8c0 0x7f0c60140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c6841f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c600cdc90 0x7f0c600b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c6841f700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0c600b2e70 0x7f0c6006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c6841f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6013ff00 con 0x7f0c600cd8c0 2023-11-28T11:08:47.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c6841f700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c601404c0 con 0x7f0c600b2e70 2023-11-28T11:08:47.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c6841f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6015aa80 con 0x7f0c600cdc90 2023-11-28T11:08:47.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c661bb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0c600cd8c0 0x7f0c60140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c661bb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0c600cd8c0 0x7f0c60140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60968/0 (socket says 172.21.15.156:60968) 2023-11-28T11:08:47.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c661bb700 1 -- 172.21.15.156:0/62785976 learned_addr learned my addr 172.21.15.156:0/62785976 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:47.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c659ba700 1 --2- 172.21.15.156:0/62785976 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c600cdc90 0x7f0c600b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c669bc700 1 --2- 172.21.15.156:0/62785976 >> v2:172.21.15.186:3300/0 conn(0x7f0c600b2e70 0x7f0c6006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c661bb700 1 -- 172.21.15.156:0/62785976 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c600cdc90 msgr2=0x7f0c600b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:47.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c661bb700 1 --2- 172.21.15.156:0/62785976 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c600cdc90 0x7f0c600b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c661bb700 1 -- 172.21.15.156:0/62785976 >> v2:172.21.15.186:3300/0 conn(0x7f0c600b2e70 msgr2=0x7f0c6006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:47.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c661bb700 1 --2- 172.21.15.156:0/62785976 >> v2:172.21.15.186:3300/0 conn(0x7f0c600b2e70 0x7f0c6006f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c661bb700 1 -- 172.21.15.156:0/62785976 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c600b3310 con 0x7f0c600cd8c0 2023-11-28T11:08:47.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c669bc700 1 --2- 172.21.15.156:0/62785976 >> v2:172.21.15.186:3300/0 conn(0x7f0c600b2e70 0x7f0c6006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:47.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.623+0000 7f0c661bb700 1 --2- 172.21.15.156:0/62785976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c600cd8c0 0x7f0c60140950 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0c601444c0 tx=0x7f0c50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117ceec27c4adf50 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.624+0000 7f0c651b9700 1 -- 172.21.15.156:0/62785976 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50005e30 con 0x7f0c600cd8c0 2023-11-28T11:08:47.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.624+0000 7f0c651b9700 1 -- 172.21.15.156:0/62785976 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c50004030 con 0x7f0c600cd8c0 2023-11-28T11:08:47.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.624+0000 7f0c651b9700 1 -- 172.21.15.156:0/62785976 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c500043e0 con 0x7f0c600cd8c0 2023-11-28T11:08:47.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.624+0000 7f0c6841f700 1 -- 172.21.15.156:0/62785976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c600cd8c0 msgr2=0x7f0c60140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.624+0000 7f0c6841f700 1 --2- 172.21.15.156:0/62785976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c600cd8c0 0x7f0c60140950 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0c601444c0 tx=0x7f0c50009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.624+0000 7f0c6841f700 1 -- 172.21.15.156:0/62785976 shutdown_connections 2023-11-28T11:08:47.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.624+0000 7f0c6841f700 1 --2- 172.21.15.156:0/62785976 >> v2:172.21.15.186:3300/0 conn(0x7f0c600b2e70 0x7f0c6006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.624+0000 7f0c6841f700 1 --2- 172.21.15.156:0/62785976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c600cd8c0 0x7f0c60140950 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.624+0000 7f0c6841f700 1 --2- 172.21.15.156:0/62785976 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c600cdc90 0x7f0c600b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.624+0000 7f0c6841f700 1 -- 172.21.15.156:0/62785976 >> 172.21.15.156:0/62785976 conn(0x7f0c6006aed0 msgr2=0x7f0c6006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.624+0000 7f0c6841f700 1 -- 172.21.15.156:0/62785976 shutdown_connections 2023-11-28T11:08:47.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.624+0000 7f0c6841f700 1 -- 172.21.15.156:0/62785976 wait complete. 2023-11-28T11:08:47.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.625+0000 7f0c6841f700 1 Processor -- start 2023-11-28T11:08:47.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.626+0000 7f0c6841f700 1 -- start start 2023-11-28T11:08:47.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.626+0000 7f0c6841f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c600b2e70 0x7f0c601f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.626+0000 7f0c6841f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c600cd8c0 0x7f0c6013ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.626+0000 7f0c6841f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c600cdc90 0x7f0c6013d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.626+0000 7f0c6841f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c601f51f0 con 0x7f0c600b2e70 2023-11-28T11:08:47.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.626+0000 7f0c6841f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6015bb80 con 0x7f0c600cd8c0 2023-11-28T11:08:47.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.626+0000 7f0c6841f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6015be60 con 0x7f0c600cdc90 2023-11-28T11:08:47.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.626+0000 7f0c661bb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c600b2e70 0x7f0c601f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.626+0000 7f0c661bb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c600b2e70 0x7f0c601f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60974/0 (socket says 172.21.15.156:60974) 2023-11-28T11:08:47.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.626+0000 7f0c661bb700 1 -- 172.21.15.156:0/4100295460 learned_addr learned my addr 172.21.15.156:0/4100295460 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:47.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c669bc700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c600cdc90 0x7f0c6013d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c661bb700 1 -- 172.21.15.156:0/4100295460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c600cdc90 msgr2=0x7f0c6013d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c661bb700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c600cdc90 0x7f0c6013d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c661bb700 1 -- 172.21.15.156:0/4100295460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c600cd8c0 msgr2=0x7f0c6013ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c659ba700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c600cd8c0 0x7f0c6013ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c661bb700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c600cd8c0 0x7f0c6013ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c661bb700 1 -- 172.21.15.156:0/4100295460 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c50009500 con 0x7f0c600b2e70 2023-11-28T11:08:47.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c661bb700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c600b2e70 0x7f0c601f4cb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009130 tx=0x7f0c50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c56ffd700 1 -- 172.21.15.156:0/4100295460 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50003dd0 con 0x7f0c600b2e70 2023-11-28T11:08:47.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c56ffd700 1 -- 172.21.15.156:0/4100295460 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c50011420 con 0x7f0c600b2e70 2023-11-28T11:08:47.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c6841f700 1 -- 172.21.15.156:0/4100295460 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c60140150 con 0x7f0c600b2e70 2023-11-28T11:08:47.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c56ffd700 1 -- 172.21.15.156:0/4100295460 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c500117d0 con 0x7f0c600b2e70 2023-11-28T11:08:47.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.627+0000 7f0c6841f700 1 -- 172.21.15.156:0/4100295460 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c600b35f0 con 0x7f0c600b2e70 2023-11-28T11:08:47.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.628+0000 7f0c6841f700 1 -- 172.21.15.156:0/4100295460 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c600ce100 con 0x7f0c600b2e70 2023-11-28T11:08:47.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.628+0000 7f0c56ffd700 1 -- 172.21.15.156:0/4100295460 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c5002c070 con 0x7f0c600b2e70 2023-11-28T11:08:47.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.629+0000 7f0c56ffd700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c440682f0 0x7f0c4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.629+0000 7f0c56ffd700 1 -- 172.21.15.156:0/4100295460 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0c50086d70 con 0x7f0c600b2e70 2023-11-28T11:08:47.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.629+0000 7f0c659ba700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c440682f0 0x7f0c4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.629+0000 7f0c659ba700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c440682f0 0x7f0c4406a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c600cb460 tx=0x7f0c5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.633+0000 7f0c56ffd700 1 -- 172.21.15.156:0/4100295460 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0c500535d0 con 0x7f0c600b2e70 2023-11-28T11:08:47.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.814+0000 7f0c6841f700 1 -- 172.21.15.156:0/4100295460 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0c60068690 con 0x7f0c600b2e70 2023-11-28T11:08:47.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.815+0000 7f0c56ffd700 1 -- 172.21.15.156:0/4100295460 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0c50016020 con 0x7f0c600b2e70 2023-11-28T11:08:47.816 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:08:47.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.816+0000 7f0c6841f700 1 -- 172.21.15.156:0/4100295460 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c440682f0 msgr2=0x7f0c4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.816+0000 7f0c6841f700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c440682f0 0x7f0c4406a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c600cb460 tx=0x7f0c5c008040 comp rx=0 tx=0).stop 2023-11-28T11:08:47.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.816+0000 7f0c6841f700 1 -- 172.21.15.156:0/4100295460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c600b2e70 msgr2=0x7f0c601f4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.816+0000 7f0c6841f700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c600b2e70 0x7f0c601f4cb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009130 tx=0x7f0c50004750 comp rx=0 tx=0).stop 2023-11-28T11:08:47.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.816+0000 7f0c6841f700 1 -- 172.21.15.156:0/4100295460 shutdown_connections 2023-11-28T11:08:47.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.816+0000 7f0c6841f700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c440682f0 0x7f0c4406a7a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.816+0000 7f0c6841f700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c600cdc90 0x7f0c6013d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.816+0000 7f0c6841f700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c600b2e70 0x7f0c601f4cb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.816+0000 7f0c6841f700 1 --2- 172.21.15.156:0/4100295460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c600cd8c0 0x7f0c6013ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.817+0000 7f0c6841f700 1 -- 172.21.15.156:0/4100295460 >> 172.21.15.156:0/4100295460 conn(0x7f0c6006aed0 msgr2=0x7f0c6013ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.817+0000 7f0c6841f700 1 -- 172.21.15.156:0/4100295460 shutdown_connections 2023-11-28T11:08:47.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.817+0000 7f0c6841f700 1 -- 172.21.15.156:0/4100295460 wait complete. 2023-11-28T11:08:47.831 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:47.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.950+0000 7f1d0f6ed700 1 Processor -- start 2023-11-28T11:08:47.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.950+0000 7f1d0f6ed700 1 -- start start 2023-11-28T11:08:47.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.951+0000 7f1d0f6ed700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1d0815cef0 0x7f1d080af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.951+0000 7f1d0f6ed700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d080cbdf0 0x7f1d0806b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.951+0000 7f1d0f6ed700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d080cc5f0 con 0x7f1d0806b8c0 2023-11-28T11:08:47.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.951+0000 7f1d0f6ed700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0806bd40 con 0x7f1d0815cef0 2023-11-28T11:08:47.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.951+0000 7f1d0f6ed700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0815d470 con 0x7f1d080cbdf0 2023-11-28T11:08:47.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.951+0000 7f1d0d489700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1d0815cef0 0x7f1d080af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.951+0000 7f1d0d489700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1d0815cef0 0x7f1d080af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60982/0 (socket says 172.21.15.156:60982) 2023-11-28T11:08:47.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.951+0000 7f1d0d489700 1 -- 172.21.15.156:0/3649085648 learned_addr learned my addr 172.21.15.156:0/3649085648 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:47.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1d0cc88700 1 --2- 172.21.15.156:0/3649085648 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d080cbdf0 0x7f1d0806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1d0d489700 1 -- 172.21.15.156:0/3649085648 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d080cbdf0 msgr2=0x7f1d0806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:47.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1d0d489700 1 --2- 172.21.15.156:0/3649085648 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d080cbdf0 0x7f1d0806b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1d0d489700 1 -- 172.21.15.156:0/3649085648 >> v1:172.21.15.186:6789/0 conn(0x7f1d0806b8c0 legacy=0x7f1d08071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:47.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1d0d489700 1 -- 172.21.15.156:0/3649085648 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0815d750 con 0x7f1d0815cef0 2023-11-28T11:08:47.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1d0d489700 1 --2- 172.21.15.156:0/3649085648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d0815cef0 0x7f1d080af400 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80098f0 tx=0x7f1cf8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475efb4a3c0ede7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1cfffff700 1 -- 172.21.15.156:0/3649085648 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8004230 con 0x7f1d0815cef0 2023-11-28T11:08:47.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1cfffff700 1 -- 172.21.15.156:0/3649085648 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf8004390 con 0x7f1d0815cef0 2023-11-28T11:08:47.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1cfffff700 1 -- 172.21.15.156:0/3649085648 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80036a0 con 0x7f1d0815cef0 2023-11-28T11:08:47.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/3649085648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d0815cef0 msgr2=0x7f1d080af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1d0f6ed700 1 --2- 172.21.15.156:0/3649085648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d0815cef0 0x7f1d080af400 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80098f0 tx=0x7f1cf8009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:47.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/3649085648 shutdown_connections 2023-11-28T11:08:47.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1d0f6ed700 1 --2- 172.21.15.156:0/3649085648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d0815cef0 0x7f1d080af400 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.952+0000 7f1d0f6ed700 1 --2- 172.21.15.156:0/3649085648 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d080cbdf0 0x7f1d0806b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.953+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/3649085648 >> 172.21.15.156:0/3649085648 conn(0x7f1d080d0000 msgr2=0x7f1d080d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.953+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/3649085648 shutdown_connections 2023-11-28T11:08:47.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.953+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/3649085648 wait complete. 2023-11-28T11:08:47.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.954+0000 7f1d0f6ed700 1 Processor -- start 2023-11-28T11:08:47.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.954+0000 7f1d0f6ed700 1 -- start start 2023-11-28T11:08:47.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.954+0000 7f1d0f6ed700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d0806b8c0 0x7f1d080c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.954+0000 7f1d0f6ed700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d080cbdf0 0x7f1d080ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.954+0000 7f1d0f6ed700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d0815cef0 0x7f1d080ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.954+0000 7f1d0f6ed700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080c65f0 con 0x7f1d0806b8c0 2023-11-28T11:08:47.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.954+0000 7f1d0f6ed700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d081e3f90 con 0x7f1d0815cef0 2023-11-28T11:08:47.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.954+0000 7f1d0f6ed700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d081e4270 con 0x7f1d080cbdf0 2023-11-28T11:08:47.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1d0d489700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d0806b8c0 0x7f1d080c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1d0d489700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d0806b8c0 0x7f1d080c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60988/0 (socket says 172.21.15.156:60988) 2023-11-28T11:08:47.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1d0d489700 1 -- 172.21.15.156:0/1089838590 learned_addr learned my addr 172.21.15.156:0/1089838590 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:47.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1d0d489700 1 -- 172.21.15.156:0/1089838590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d080cbdf0 msgr2=0x7f1d080ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1d0d489700 1 --2- 172.21.15.156:0/1089838590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d080cbdf0 0x7f1d080ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1d0d489700 1 -- 172.21.15.156:0/1089838590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d0815cef0 msgr2=0x7f1d080ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1d0d489700 1 --2- 172.21.15.156:0/1089838590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d0815cef0 0x7f1d080ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1d0d489700 1 -- 172.21.15.156:0/1089838590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf8009550 con 0x7f1d0806b8c0 2023-11-28T11:08:47.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1d0d489700 1 --2- 172.21.15.156:0/1089838590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d0806b8c0 0x7f1d080c60b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009130 tx=0x7f1cf8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1cfdffb700 1 -- 172.21.15.156:0/1089838590 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8003e70 con 0x7f1d0806b8c0 2023-11-28T11:08:47.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1cfdffb700 1 -- 172.21.15.156:0/1089838590 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf8020070 con 0x7f1d0806b8c0 2023-11-28T11:08:47.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/1089838590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0815dd10 con 0x7f1d0806b8c0 2023-11-28T11:08:47.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1cfdffb700 1 -- 172.21.15.156:0/1089838590 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8013870 con 0x7f1d0806b8c0 2023-11-28T11:08:47.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.955+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/1089838590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0815da30 con 0x7f1d0806b8c0 2023-11-28T11:08:47.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.956+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/1089838590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0815dff0 con 0x7f1d0806b8c0 2023-11-28T11:08:47.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.957+0000 7f1cfdffb700 1 -- 172.21.15.156:0/1089838590 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1cf802b020 con 0x7f1d0806b8c0 2023-11-28T11:08:47.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.957+0000 7f1cfdffb700 1 --2- 172.21.15.156:0/1089838590 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1cec0682f0 0x7f1cec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.957+0000 7f1cfdffb700 1 -- 172.21.15.156:0/1089838590 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1cf8089b00 con 0x7f1d0806b8c0 2023-11-28T11:08:47.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.957+0000 7f1d0cc88700 1 --2- 172.21.15.156:0/1089838590 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1cec0682f0 0x7f1cec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.958+0000 7f1d0cc88700 1 --2- 172.21.15.156:0/1089838590 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1cec0682f0 0x7f1cec06a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1d08154e60 tx=0x7f1d04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:47.961+0000 7f1cfdffb700 1 -- 172.21.15.156:0/1089838590 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1cf8056360 con 0x7f1d0806b8c0 2023-11-28T11:08:48.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.146+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/1089838590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d0815dff0 con 0x7f1d0806b8c0 2023-11-28T11:08:48.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.147+0000 7f1cfdffb700 1 -- 172.21.15.156:0/1089838590 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f1cf8059980 con 0x7f1d0806b8c0 2023-11-28T11:08:48.148 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:48.149 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:18.354009+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:48.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.149+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/1089838590 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1cec0682f0 msgr2=0x7f1cec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.149+0000 7f1d0f6ed700 1 --2- 172.21.15.156:0/1089838590 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1cec0682f0 0x7f1cec06a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1d08154e60 tx=0x7f1d04008040 comp rx=0 tx=0).stop 2023-11-28T11:08:48.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.149+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/1089838590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d0806b8c0 msgr2=0x7f1d080c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.149+0000 7f1d0f6ed700 1 --2- 172.21.15.156:0/1089838590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d0806b8c0 0x7f1d080c60b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009130 tx=0x7f1cf8004750 comp rx=0 tx=0).stop 2023-11-28T11:08:48.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.149+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/1089838590 shutdown_connections 2023-11-28T11:08:48.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.149+0000 7f1d0f6ed700 1 --2- 172.21.15.156:0/1089838590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d0815cef0 0x7f1d080ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.149+0000 7f1d0f6ed700 1 --2- 172.21.15.156:0/1089838590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d0806b8c0 0x7f1d080c60b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.149+0000 7f1d0f6ed700 1 --2- 172.21.15.156:0/1089838590 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1cec0682f0 0x7f1cec06a7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.149+0000 7f1d0f6ed700 1 --2- 172.21.15.156:0/1089838590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d080cbdf0 0x7f1d080ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.149+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/1089838590 >> 172.21.15.156:0/1089838590 conn(0x7f1d080d0000 msgr2=0x7f1d0806d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:48.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.149+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/1089838590 shutdown_connections 2023-11-28T11:08:48.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.149+0000 7f1d0f6ed700 1 -- 172.21.15.156:0/1089838590 wait complete. 2023-11-28T11:08:48.162 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:15.021591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:16.332241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:48.163 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:48.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.285+0000 7f537ecb7700 1 Processor -- start 2023-11-28T11:08:48.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.285+0000 7f537ecb7700 1 -- start start 2023-11-28T11:08:48.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.286+0000 7f537ecb7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5378160b10 0x7f53780cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.286+0000 7f537ecb7700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f537806b060 0x7f537806b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.286+0000 7f537ecb7700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53780c9e10 con 0x7f5378160740 2023-11-28T11:08:48.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f537ecb7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53780ca720 con 0x7f5378160b10 2023-11-28T11:08:48.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f537ecb7700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537806efc0 con 0x7f537806b060 2023-11-28T11:08:48.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f5377fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5378160b10 0x7f53780cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f5377fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5378160b10 0x7f53780cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32768/0 (socket says 172.21.15.156:32768) 2023-11-28T11:08:48.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f5377fff700 1 -- 172.21.15.156:0/3485552859 learned_addr learned my addr 172.21.15.156:0/3485552859 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:48.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f537d254700 1 --2- 172.21.15.156:0/3485552859 >> v2:172.21.15.186:3300/0 conn(0x7f537806b060 0x7f537806b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f5377fff700 1 -- 172.21.15.156:0/3485552859 >> v2:172.21.15.186:3300/0 conn(0x7f537806b060 msgr2=0x7f537806b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:48.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f5377fff700 1 --2- 172.21.15.156:0/3485552859 >> v2:172.21.15.186:3300/0 conn(0x7f537806b060 0x7f537806b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f5377fff700 1 -- 172.21.15.156:0/3485552859 >> v1:172.21.15.186:6789/0 conn(0x7f5378160740 legacy=0x7f53780cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:48.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f5377fff700 1 -- 172.21.15.156:0/3485552859 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53780ca0d0 con 0x7f5378160b10 2023-11-28T11:08:48.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f5377fff700 1 --2- 172.21.15.156:0/3485552859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5378160b10 0x7f53780cd390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f537806c470 tx=0x7f5368009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de158675360a72c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f53777fe700 1 -- 172.21.15.156:0/3485552859 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368004230 con 0x7f5378160b10 2023-11-28T11:08:48.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f53777fe700 1 -- 172.21.15.156:0/3485552859 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5368004390 con 0x7f5378160b10 2023-11-28T11:08:48.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.287+0000 7f53777fe700 1 -- 172.21.15.156:0/3485552859 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53680036a0 con 0x7f5378160b10 2023-11-28T11:08:48.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.288+0000 7f537ecb7700 1 -- 172.21.15.156:0/3485552859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5378160b10 msgr2=0x7f53780cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.288+0000 7f537ecb7700 1 --2- 172.21.15.156:0/3485552859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5378160b10 0x7f53780cd390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f537806c470 tx=0x7f5368009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.288+0000 7f537ecb7700 1 -- 172.21.15.156:0/3485552859 shutdown_connections 2023-11-28T11:08:48.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.288+0000 7f537ecb7700 1 --2- 172.21.15.156:0/3485552859 >> v2:172.21.15.186:3300/0 conn(0x7f537806b060 0x7f537806b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.288+0000 7f537ecb7700 1 --2- 172.21.15.156:0/3485552859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5378160b10 0x7f53780cd390 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.288+0000 7f537ecb7700 1 -- 172.21.15.156:0/3485552859 >> 172.21.15.156:0/3485552859 conn(0x7f53780b12b0 msgr2=0x7f53780b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:48.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.288+0000 7f537ecb7700 1 -- 172.21.15.156:0/3485552859 shutdown_connections 2023-11-28T11:08:48.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.288+0000 7f537ecb7700 1 -- 172.21.15.156:0/3485552859 wait complete. 2023-11-28T11:08:48.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.289+0000 7f537ecb7700 1 Processor -- start 2023-11-28T11:08:48.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.289+0000 7f537ecb7700 1 -- start start 2023-11-28T11:08:48.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ecb7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537806b060 0x7f53781600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ecb7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5378160740 0x7f53780ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ecb7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5378160b10 0x7f537815eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ecb7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537815e660 con 0x7f537806b060 2023-11-28T11:08:48.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ecb7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537815f740 con 0x7f5378160740 2023-11-28T11:08:48.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ecb7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537815e310 con 0x7f5378160b10 2023-11-28T11:08:48.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ca53700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537806b060 0x7f53781600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ca53700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537806b060 0x7f53781600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32770/0 (socket says 172.21.15.156:32770) 2023-11-28T11:08:48.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ca53700 1 -- 172.21.15.156:0/41322731 learned_addr learned my addr 172.21.15.156:0/41322731 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:48.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f5377fff700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5378160740 0x7f53780ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537d254700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5378160b10 0x7f537815eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ca53700 1 -- 172.21.15.156:0/41322731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5378160b10 msgr2=0x7f537815eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ca53700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5378160b10 0x7f537815eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ca53700 1 -- 172.21.15.156:0/41322731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5378160740 msgr2=0x7f53780ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ca53700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5378160740 0x7f53780ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ca53700 1 -- 172.21.15.156:0/41322731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368009550 con 0x7f537806b060 2023-11-28T11:08:48.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.290+0000 7f537ca53700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537806b060 0x7f53781600d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f536000c4a0 tx=0x7f536000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.291+0000 7f53757fa700 1 -- 172.21.15.156:0/41322731 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53600054e0 con 0x7f537806b060 2023-11-28T11:08:48.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.291+0000 7f53757fa700 1 -- 172.21.15.156:0/41322731 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5360003680 con 0x7f537806b060 2023-11-28T11:08:48.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.291+0000 7f537ecb7700 1 -- 172.21.15.156:0/41322731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53780cd8d0 con 0x7f537806b060 2023-11-28T11:08:48.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.291+0000 7f53757fa700 1 -- 172.21.15.156:0/41322731 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5360003a30 con 0x7f537806b060 2023-11-28T11:08:48.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.291+0000 7f537ecb7700 1 -- 172.21.15.156:0/41322731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53780ca3b0 con 0x7f537806b060 2023-11-28T11:08:48.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.292+0000 7f537ecb7700 1 -- 172.21.15.156:0/41322731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5378160f60 con 0x7f537806b060 2023-11-28T11:08:48.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.292+0000 7f53757fa700 1 -- 172.21.15.156:0/41322731 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5360019030 con 0x7f537806b060 2023-11-28T11:08:48.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.292+0000 7f53757fa700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f53580682f0 0x7f535806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.293+0000 7f53757fa700 1 -- 172.21.15.156:0/41322731 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5360085080 con 0x7f537806b060 2023-11-28T11:08:48.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.293+0000 7f5377fff700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f53580682f0 0x7f535806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.293+0000 7f5377fff700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f53580682f0 0x7f535806a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5368009130 tx=0x7f5368008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.297+0000 7f53757fa700 1 -- 172.21.15.156:0/41322731 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f53600518e0 con 0x7f537806b060 2023-11-28T11:08:48.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.478+0000 7f537ecb7700 1 -- 172.21.15.156:0/41322731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f537815e940 con 0x7f537806b060 2023-11-28T11:08:48.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.478+0000 7f53757fa700 1 -- 172.21.15.156:0/41322731 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5360017020 con 0x7f537806b060 2023-11-28T11:08:48.480 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:08:48.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.481+0000 7f537ecb7700 1 -- 172.21.15.156:0/41322731 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f53580682f0 msgr2=0x7f535806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.481+0000 7f537ecb7700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f53580682f0 0x7f535806a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5368009130 tx=0x7f5368008030 comp rx=0 tx=0).stop 2023-11-28T11:08:48.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.481+0000 7f537ecb7700 1 -- 172.21.15.156:0/41322731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537806b060 msgr2=0x7f53781600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.481+0000 7f537ecb7700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537806b060 0x7f53781600d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f536000c4a0 tx=0x7f536000c860 comp rx=0 tx=0).stop 2023-11-28T11:08:48.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.481+0000 7f537ecb7700 1 -- 172.21.15.156:0/41322731 shutdown_connections 2023-11-28T11:08:48.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.481+0000 7f537ecb7700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f53580682f0 0x7f535806a7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.481+0000 7f537ecb7700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5378160b10 0x7f537815eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.481+0000 7f537ecb7700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537806b060 0x7f53781600d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.481+0000 7f537ecb7700 1 --2- 172.21.15.156:0/41322731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5378160740 0x7f53780ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.481+0000 7f537ecb7700 1 -- 172.21.15.156:0/41322731 >> 172.21.15.156:0/41322731 conn(0x7f53780b12b0 msgr2=0x7f537806a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:48.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.481+0000 7f537ecb7700 1 -- 172.21.15.156:0/41322731 shutdown_connections 2023-11-28T11:08:48.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.481+0000 7f537ecb7700 1 -- 172.21.15.156:0/41322731 wait complete. 2023-11-28T11:08:48.494 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:48.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.617+0000 7f02d7851700 1 Processor -- start 2023-11-28T11:08:48.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.617+0000 7f02d7851700 1 -- start start 2023-11-28T11:08:48.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.618+0000 7f02d7851700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d01658c0 0x7f02d0167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.618+0000 7f02d7851700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d015d060 con 0x7f02d0071040 2023-11-28T11:08:48.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.618+0000 7f02d7851700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d015d340 con 0x7f02d006dee0 2023-11-28T11:08:48.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.618+0000 7f02d7851700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d0161550 con 0x7f02d01658c0 2023-11-28T11:08:48.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.618+0000 7f02d55ed700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f02d0071040 0x7f02d0071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47656/0 (socket says 172.21.15.156:47656) 2023-11-28T11:08:48.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.618+0000 7f02d55ed700 1 -- 172.21.15.156:0/4214836518 learned_addr learned my addr 172.21.15.156:0/4214836518 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:48.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.618+0000 7f02d5dee700 1 --2- 172.21.15.156:0/4214836518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d01658c0 0x7f02d0167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459416295 0 0) 0x7f02d015d060 con 0x7f02d0071040 2023-11-28T11:08:48.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02bc0035b0 con 0x7f02d0071040 2023-11-28T11:08:48.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918604757 0 0) 0x7f02d015d340 con 0x7f02d006dee0 2023-11-28T11:08:48.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d015d060 con 0x7f02d006dee0 2023-11-28T11:08:48.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2330451984 0 0) 0x7f02bc0035b0 con 0x7f02d0071040 2023-11-28T11:08:48.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d01658c0 msgr2=0x7f02d0167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:48.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 --2- 172.21.15.156:0/4214836518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d01658c0 0x7f02d0167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 >> v1:172.21.15.186:6789/0 conn(0x7f02d006dee0 legacy=0x7f02d006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d015d620 con 0x7f02d0071040 2023-11-28T11:08:48.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f02c0001670 con 0x7f02d0071040 2023-11-28T11:08:48.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 >> v1:172.21.15.156:6789/0 conn(0x7f02d0071040 legacy=0x7f02d0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 --2- 172.21.15.156:0/4214836518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d01658c0 0x7f02bc003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 --2- 172.21.15.156:0/4214836518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02bc004170 0x7f02bc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 --2- 172.21.15.156:0/4214836518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02bc004d90 0x7f02bc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d0161550 con 0x7f02bc004170 2023-11-28T11:08:48.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc00bc50 con 0x7f02bc004d90 2023-11-28T11:08:48.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc00be10 con 0x7f02d01658c0 2023-11-28T11:08:48.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02d4dec700 1 --2- 172.21.15.156:0/4214836518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02bc004170 0x7f02bc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02d55ed700 1 --2- 172.21.15.156:0/4214836518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d01658c0 0x7f02bc003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02d5dee700 1 --2- 172.21.15.156:0/4214836518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02bc004d90 0x7f02bc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02d4dec700 1 -- 172.21.15.156:0/4214836518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d01658c0 msgr2=0x7f02bc003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02d4dec700 1 --2- 172.21.15.156:0/4214836518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d01658c0 0x7f02bc003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02d4dec700 1 -- 172.21.15.156:0/4214836518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02bc004d90 msgr2=0x7f02bc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02d4dec700 1 --2- 172.21.15.156:0/4214836518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02bc004d90 0x7f02bc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02d4dec700 1 -- 172.21.15.156:0/4214836518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc0035b0 con 0x7f02bc004170 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.619+0000 7f02d4dec700 1 --2- 172.21.15.156:0/4214836518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02bc004170 0x7f02bc004850 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f02cc007fd0 tx=0x7f02bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c8c3876a17930f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.620+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc001d80 con 0x7f02bc004170 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.620+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc0052d0 con 0x7f02bc004170 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.620+0000 7f02c7fff700 1 -- 172.21.15.156:0/4214836518 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc001470 con 0x7f02bc004170 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.620+0000 7f02d7851700 1 -- 172.21.15.156:0/4214836518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02bc004170 msgr2=0x7f02bc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.620+0000 7f02d7851700 1 --2- 172.21.15.156:0/4214836518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02bc004170 0x7f02bc004850 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f02cc007fd0 tx=0x7f02bc005880 comp rx=0 tx=0).stop 2023-11-28T11:08:48.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.620+0000 7f02d5dee700 1 -- 172.21.15.156:0/4214836518 reap_dead start 2023-11-28T11:08:48.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.620+0000 7f02d7851700 1 -- 172.21.15.156:0/4214836518 shutdown_connections 2023-11-28T11:08:48.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.620+0000 7f02d7851700 1 -- 172.21.15.156:0/4214836518 >> 172.21.15.156:0/4214836518 conn(0x7f02d00d0000 msgr2=0x7f02d00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:48.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.620+0000 7f02d7851700 1 -- 172.21.15.156:0/4214836518 shutdown_connections 2023-11-28T11:08:48.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.620+0000 7f02d7851700 1 -- 172.21.15.156:0/4214836518 wait complete. 2023-11-28T11:08:48.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.621+0000 7f02d7851700 1 Processor -- start 2023-11-28T11:08:48.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.622+0000 7f02d7851700 1 -- start start 2023-11-28T11:08:48.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.622+0000 7f02d7851700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02bc004170 0x7f02d0159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.622+0000 7f02d7851700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d00c64b0 0x7f02d00c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.622+0000 7f02d7851700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d0145e40 0x7f02d0145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.622+0000 7f02d7851700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d00afa00 con 0x7f02d0145e40 2023-11-28T11:08:48.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.622+0000 7f02d7851700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d00afc10 con 0x7f02d00c64b0 2023-11-28T11:08:48.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.622+0000 7f02d7851700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d00afef0 con 0x7f02bc004170 2023-11-28T11:08:48.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.622+0000 7f02d5dee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d0145e40 0x7f02d0145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.622+0000 7f02d5dee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d0145e40 0x7f02d0145180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32794/0 (socket says 172.21.15.156:32794) 2023-11-28T11:08:48.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.622+0000 7f02d5dee700 1 -- 172.21.15.156:0/1822573951 learned_addr learned my addr 172.21.15.156:0/1822573951 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:48.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.622+0000 7f02d4dec700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d00c64b0 0x7f02d00c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.622+0000 7f02d55ed700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02bc004170 0x7f02d0159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02d5dee700 1 -- 172.21.15.156:0/1822573951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02bc004170 msgr2=0x7f02d0159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02d5dee700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02bc004170 0x7f02d0159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02d5dee700 1 -- 172.21.15.156:0/1822573951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d00c64b0 msgr2=0x7f02d00c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02d5dee700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d00c64b0 0x7f02d00c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02d5dee700 1 -- 172.21.15.156:0/1822573951 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d015ccd0 con 0x7f02d0145e40 2023-11-28T11:08:48.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02d4dec700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d00c64b0 0x7f02d00c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:48.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02d55ed700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02bc004170 0x7f02d0159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:48.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02d5dee700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d0145e40 0x7f02d0145180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f02d00c3a30 tx=0x7f02c8011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02c5ffb700 1 -- 172.21.15.156:0/1822573951 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c800e040 con 0x7f02d0145e40 2023-11-28T11:08:48.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02c5ffb700 1 -- 172.21.15.156:0/1822573951 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c800f040 con 0x7f02d0145e40 2023-11-28T11:08:48.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02d7851700 1 -- 172.21.15.156:0/1822573951 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d015d900 con 0x7f02d0145e40 2023-11-28T11:08:48.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02c5ffb700 1 -- 172.21.15.156:0/1822573951 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c800fa00 con 0x7f02d0145e40 2023-11-28T11:08:48.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.623+0000 7f02d7851700 1 -- 172.21.15.156:0/1822573951 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d006a270 con 0x7f02d0145e40 2023-11-28T11:08:48.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.624+0000 7f02d7851700 1 -- 172.21.15.156:0/1822573951 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d00afd50 con 0x7f02d0145e40 2023-11-28T11:08:48.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.624+0000 7f02c5ffb700 1 -- 172.21.15.156:0/1822573951 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f02c800fb60 con 0x7f02d0145e40 2023-11-28T11:08:48.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.625+0000 7f02c5ffb700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f02b40682f0 0x7f02b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.625+0000 7f02c5ffb700 1 -- 172.21.15.156:0/1822573951 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f02c8086330 con 0x7f02d0145e40 2023-11-28T11:08:48.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.625+0000 7f02d55ed700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f02b40682f0 0x7f02b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.625+0000 7f02d55ed700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f02b40682f0 0x7f02b406a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f02d0148f90 tx=0x7f02c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.629+0000 7f02c5ffb700 1 -- 172.21.15.156:0/1822573951 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f02d00afd50 con 0x7f02d0145e40 2023-11-28T11:08:48.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.809+0000 7f02d7851700 1 -- 172.21.15.156:0/1822573951 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f02d00afd50 con 0x7f02d0145e40 2023-11-28T11:08:48.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.810+0000 7f02c5ffb700 1 -- 172.21.15.156:0/1822573951 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f02d00afd50 con 0x7f02d0145e40 2023-11-28T11:08:48.811 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:08:48.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.812+0000 7f02d7851700 1 -- 172.21.15.156:0/1822573951 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f02b40682f0 msgr2=0x7f02b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.812+0000 7f02d7851700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f02b40682f0 0x7f02b406a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f02d0148f90 tx=0x7f02c0009040 comp rx=0 tx=0).stop 2023-11-28T11:08:48.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.812+0000 7f02d7851700 1 -- 172.21.15.156:0/1822573951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d0145e40 msgr2=0x7f02d0145180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.812+0000 7f02d7851700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d0145e40 0x7f02d0145180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f02d00c3a30 tx=0x7f02c8011f80 comp rx=0 tx=0).stop 2023-11-28T11:08:48.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.812+0000 7f02d7851700 1 -- 172.21.15.156:0/1822573951 shutdown_connections 2023-11-28T11:08:48.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.812+0000 7f02d7851700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d0145e40 0x7f02d0145180 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.812+0000 7f02d7851700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f02b40682f0 0x7f02b406a7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.812+0000 7f02d7851700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02bc004170 0x7f02d0159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.812+0000 7f02d7851700 1 --2- 172.21.15.156:0/1822573951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d00c64b0 0x7f02d00c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.812+0000 7f02d7851700 1 -- 172.21.15.156:0/1822573951 >> 172.21.15.156:0/1822573951 conn(0x7f02d00d0000 msgr2=0x7f02d015f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:48.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.812+0000 7f02d7851700 1 -- 172.21.15.156:0/1822573951 shutdown_connections 2023-11-28T11:08:48.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.812+0000 7f02d7851700 1 -- 172.21.15.156:0/1822573951 wait complete. 2023-11-28T11:08:48.826 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:48.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.947+0000 7fe475d54700 1 Processor -- start 2023-11-28T11:08:48.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.947+0000 7fe475d54700 1 -- start start 2023-11-28T11:08:48.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.948+0000 7fe475d54700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe47006dee0 0x7fe47006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.948+0000 7fe475d54700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47015d060 con 0x7fe470160540 2023-11-28T11:08:48.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.948+0000 7fe475d54700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47015d340 con 0x7fe470071040 2023-11-28T11:08:48.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.948+0000 7fe475d54700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47015d620 con 0x7fe47006dee0 2023-11-28T11:08:48.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.948+0000 7fe46ffff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe470160540 0x7fe470160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47680/0 (socket says 172.21.15.156:47680) 2023-11-28T11:08:48.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.948+0000 7fe46ffff700 1 -- 172.21.15.156:0/220882741 learned_addr learned my addr 172.21.15.156:0/220882741 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:48.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46effd700 1 --2- 172.21.15.156:0/220882741 >> v2:172.21.15.186:3300/0 conn(0x7fe47006dee0 0x7fe47006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814265883 0 0) 0x7fe47015d060 con 0x7fe470160540 2023-11-28T11:08:48.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe45c0035b0 con 0x7fe470160540 2023-11-28T11:08:48.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093874233 0 0) 0x7fe47015d340 con 0x7fe470071040 2023-11-28T11:08:48.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe47015d060 con 0x7fe470071040 2023-11-28T11:08:48.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3690191062 0 0) 0x7fe45c0035b0 con 0x7fe470160540 2023-11-28T11:08:48.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 >> v2:172.21.15.186:3300/0 conn(0x7fe47006dee0 msgr2=0x7fe47006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:48.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 --2- 172.21.15.156:0/220882741 >> v2:172.21.15.186:3300/0 conn(0x7fe47006dee0 0x7fe47006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 >> v1:172.21.15.186:6789/0 conn(0x7fe470071040 legacy=0x7fe470071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47015d900 con 0x7fe470160540 2023-11-28T11:08:48.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fe4680019f0 con 0x7fe470160540 2023-11-28T11:08:48.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 >> v1:172.21.15.156:6789/0 conn(0x7fe470160540 legacy=0x7fe470160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 --2- 172.21.15.156:0/220882741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe45c003b90 0x7fe45c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 --2- 172.21.15.156:0/220882741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe45c004850 0x7fe45c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 --2- 172.21.15.156:0/220882741 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe45c005270 0x7fe45c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47015d620 con 0x7fe45c004850 2023-11-28T11:08:48.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c00c1b0 con 0x7fe45c003b90 2023-11-28T11:08:48.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c00c370 con 0x7fe45c005270 2023-11-28T11:08:48.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46effd700 1 --2- 172.21.15.156:0/220882741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe45c004850 0x7fe45c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46f7fe700 1 --2- 172.21.15.156:0/220882741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe45c003b90 0x7fe45c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46ffff700 1 --2- 172.21.15.156:0/220882741 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe45c005270 0x7fe45c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46effd700 1 -- 172.21.15.156:0/220882741 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe45c005270 msgr2=0x7fe45c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:48.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46effd700 1 --2- 172.21.15.156:0/220882741 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe45c005270 0x7fe45c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46effd700 1 -- 172.21.15.156:0/220882741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe45c003b90 msgr2=0x7fe45c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:48.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46effd700 1 --2- 172.21.15.156:0/220882741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe45c003b90 0x7fe45c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46effd700 1 -- 172.21.15.156:0/220882741 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c0035b0 con 0x7fe45c004850 2023-11-28T11:08:48.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46ffff700 1 --2- 172.21.15.156:0/220882741 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe45c005270 0x7fe45c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:48.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.949+0000 7fe46ffff700 1 -- 172.21.15.156:0/220882741 reap_dead start 2023-11-28T11:08:48.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.950+0000 7fe46effd700 1 --2- 172.21.15.156:0/220882741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe45c004850 0x7fe45c004cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe460009b60 tx=0x7fe45c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67a573039fe826c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.950+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46000e6c0 con 0x7fe45c004850 2023-11-28T11:08:48.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.950+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46000e820 con 0x7fe45c004850 2023-11-28T11:08:48.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.950+0000 7fe46e7fc700 1 -- 172.21.15.156:0/220882741 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46000eb90 con 0x7fe45c004850 2023-11-28T11:08:48.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.950+0000 7fe475d54700 1 -- 172.21.15.156:0/220882741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe45c004850 msgr2=0x7fe45c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.950+0000 7fe475d54700 1 --2- 172.21.15.156:0/220882741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe45c004850 0x7fe45c004cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe460009b60 tx=0x7fe45c005880 comp rx=0 tx=0).stop 2023-11-28T11:08:48.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.950+0000 7fe475d54700 1 -- 172.21.15.156:0/220882741 shutdown_connections 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.950+0000 7fe475d54700 1 --2- 172.21.15.156:0/220882741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe45c004850 0x7fe45c004cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.950+0000 7fe475d54700 1 -- 172.21.15.156:0/220882741 >> 172.21.15.156:0/220882741 conn(0x7fe4700d0000 msgr2=0x7fe4700d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.950+0000 7fe475d54700 1 -- 172.21.15.156:0/220882741 shutdown_connections 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.950+0000 7fe475d54700 1 -- 172.21.15.156:0/220882741 wait complete. 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.951+0000 7fe475d54700 1 Processor -- start 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.952+0000 7fe475d54700 1 -- start start 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.952+0000 7fe475d54700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe45c004850 0x7fe4700adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.952+0000 7fe475d54700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4700afa00 0x7fe470150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.952+0000 7fe475d54700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4700c4bf0 0x7fe4700c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.952+0000 7fe475d54700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700c55a0 con 0x7fe4700afa00 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.952+0000 7fe475d54700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700b0d60 con 0x7fe45c004850 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.952+0000 7fe475d54700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe470150a70 con 0x7fe4700c4bf0 2023-11-28T11:08:48.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe46effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4700afa00 0x7fe470150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe46effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4700afa00 0x7fe470150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32812/0 (socket says 172.21.15.156:32812) 2023-11-28T11:08:48.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe46effd700 1 -- 172.21.15.156:0/2397227352 learned_addr learned my addr 172.21.15.156:0/2397227352 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:48.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe46f7fe700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe45c004850 0x7fe4700adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe46ffff700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4700c4bf0 0x7fe4700c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe46effd700 1 -- 172.21.15.156:0/2397227352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4700c4bf0 msgr2=0x7fe4700c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe46effd700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4700c4bf0 0x7fe4700c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe46effd700 1 -- 172.21.15.156:0/2397227352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe45c004850 msgr2=0x7fe4700adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:48.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe46effd700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe45c004850 0x7fe4700adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:48.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe46effd700 1 -- 172.21.15.156:0/2397227352 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47006a270 con 0x7fe4700afa00 2023-11-28T11:08:48.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe46f7fe700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe45c004850 0x7fe4700adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:48.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe46effd700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4700afa00 0x7fe470150530 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe46000e4b0 tx=0x7fe4600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe467fff700 1 -- 172.21.15.156:0/2397227352 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46000e6c0 con 0x7fe4700afa00 2023-11-28T11:08:48.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe467fff700 1 -- 172.21.15.156:0/2397227352 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe460003ad0 con 0x7fe4700afa00 2023-11-28T11:08:48.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe475d54700 1 -- 172.21.15.156:0/2397227352 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47015ccd0 con 0x7fe4700afa00 2023-11-28T11:08:48.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe467fff700 1 -- 172.21.15.156:0/2397227352 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460003e80 con 0x7fe4700afa00 2023-11-28T11:08:48.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.953+0000 7fe475d54700 1 -- 172.21.15.156:0/2397227352 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4700b0430 con 0x7fe4700afa00 2023-11-28T11:08:48.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.954+0000 7fe475d54700 1 -- 172.21.15.156:0/2397227352 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47006a400 con 0x7fe4700afa00 2023-11-28T11:08:48.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.955+0000 7fe467fff700 1 -- 172.21.15.156:0/2397227352 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe460019030 con 0x7fe4700afa00 2023-11-28T11:08:48.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.955+0000 7fe467fff700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe4500682f0 0x7fe45006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:48.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.955+0000 7fe467fff700 1 -- 172.21.15.156:0/2397227352 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4600879c0 con 0x7fe4700afa00 2023-11-28T11:08:48.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.955+0000 7fe46f7fe700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe4500682f0 0x7fe45006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:48.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.956+0000 7fe46f7fe700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe4500682f0 0x7fe45006a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe458007fd0 tx=0x7fe458009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:48.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:48.959+0000 7fe467fff700 1 -- 172.21.15.156:0/2397227352 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe4600541b0 con 0x7fe4700afa00 2023-11-28T11:08:49.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.143+0000 7fe475d54700 1 -- 172.21.15.156:0/2397227352 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe47006a400 con 0x7fe4700afa00 2023-11-28T11:08:49.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.143+0000 7fe467fff700 1 -- 172.21.15.156:0/2397227352 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe47006a400 con 0x7fe4700afa00 2023-11-28T11:08:49.145 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:08:49.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.145+0000 7fe475d54700 1 -- 172.21.15.156:0/2397227352 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe4500682f0 msgr2=0x7fe45006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.145+0000 7fe475d54700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe4500682f0 0x7fe45006a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe458007fd0 tx=0x7fe458009040 comp rx=0 tx=0).stop 2023-11-28T11:08:49.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.145+0000 7fe475d54700 1 -- 172.21.15.156:0/2397227352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4700afa00 msgr2=0x7fe470150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.145+0000 7fe475d54700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4700afa00 0x7fe470150530 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe46000e4b0 tx=0x7fe4600047c0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.145+0000 7fe475d54700 1 -- 172.21.15.156:0/2397227352 shutdown_connections 2023-11-28T11:08:49.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.145+0000 7fe475d54700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe4500682f0 0x7fe45006a7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.145+0000 7fe475d54700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4700c4bf0 0x7fe4700c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.145+0000 7fe475d54700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe45c004850 0x7fe4700adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.145+0000 7fe475d54700 1 --2- 172.21.15.156:0/2397227352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4700afa00 0x7fe470150530 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.145+0000 7fe475d54700 1 -- 172.21.15.156:0/2397227352 >> 172.21.15.156:0/2397227352 conn(0x7fe4700d0000 msgr2=0x7fe47006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:49.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.146+0000 7fe475d54700 1 -- 172.21.15.156:0/2397227352 shutdown_connections 2023-11-28T11:08:49.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.146+0000 7fe475d54700 1 -- 172.21.15.156:0/2397227352 wait complete. 2023-11-28T11:08:49.159 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:49.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.281+0000 7fa5060a0700 1 Processor -- start 2023-11-28T11:08:49.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.281+0000 7fa5060a0700 1 -- start start 2023-11-28T11:08:49.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.281+0000 7fa5060a0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa50006dee0 0x7fa50006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.281+0000 7fa5060a0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa500160540 0x7fa5001609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.281+0000 7fa5060a0700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50015d060 con 0x7fa500071040 2023-11-28T11:08:49.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.281+0000 7fa5060a0700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50015d340 con 0x7fa50006dee0 2023-11-28T11:08:49.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.281+0000 7fa5060a0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50015d620 con 0x7fa500160540 2023-11-28T11:08:49.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4feffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa50006dee0 0x7fa50006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4ff7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa500071040 0x7fa500071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47696/0 (socket says 172.21.15.156:47696) 2023-11-28T11:08:49.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4ff7fe700 1 -- 172.21.15.156:0/352888807 learned_addr learned my addr 172.21.15.156:0/352888807 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:49.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4fffff700 1 --2- 172.21.15.156:0/352888807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa500160540 0x7fa5001609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4feffd700 1 -- 172.21.15.156:0/352888807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa500160540 msgr2=0x7fa5001609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:49.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4feffd700 1 --2- 172.21.15.156:0/352888807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa500160540 0x7fa5001609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4feffd700 1 -- 172.21.15.156:0/352888807 >> v1:172.21.15.156:6789/0 conn(0x7fa500071040 legacy=0x7fa500071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4fe7fc700 1 -- 172.21.15.156:0/352888807 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812403005 0 0) 0x7fa50015d060 con 0x7fa500071040 2023-11-28T11:08:49.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4feffd700 1 -- 172.21.15.156:0/352888807 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50015d900 con 0x7fa50006dee0 2023-11-28T11:08:49.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4feffd700 1 --2- 172.21.15.156:0/352888807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa50006dee0 0x7fa50006e330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009840 tx=0x7fa4e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51638836f57adefb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4fe7fc700 1 -- 172.21.15.156:0/352888807 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8011830 con 0x7fa50006dee0 2023-11-28T11:08:49.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4fe7fc700 1 -- 172.21.15.156:0/352888807 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e8011990 con 0x7fa50006dee0 2023-11-28T11:08:49.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.282+0000 7fa4fe7fc700 1 -- 172.21.15.156:0/352888807 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8011d20 con 0x7fa50006dee0 2023-11-28T11:08:49.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.283+0000 7fa5060a0700 1 -- 172.21.15.156:0/352888807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa50006dee0 msgr2=0x7fa50006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.283+0000 7fa5060a0700 1 --2- 172.21.15.156:0/352888807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa50006dee0 0x7fa50006e330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009840 tx=0x7fa4e8009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:49.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.283+0000 7fa5060a0700 1 -- 172.21.15.156:0/352888807 shutdown_connections 2023-11-28T11:08:49.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.283+0000 7fa5060a0700 1 --2- 172.21.15.156:0/352888807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa500160540 0x7fa5001609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.283+0000 7fa5060a0700 1 --2- 172.21.15.156:0/352888807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa50006dee0 0x7fa50006e330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.283+0000 7fa5060a0700 1 -- 172.21.15.156:0/352888807 >> 172.21.15.156:0/352888807 conn(0x7fa5000d0000 msgr2=0x7fa5000d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:49.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.283+0000 7fa5060a0700 1 -- 172.21.15.156:0/352888807 shutdown_connections 2023-11-28T11:08:49.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.283+0000 7fa5060a0700 1 -- 172.21.15.156:0/352888807 wait complete. 2023-11-28T11:08:49.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.284+0000 7fa5060a0700 1 Processor -- start 2023-11-28T11:08:49.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.284+0000 7fa5060a0700 1 -- start start 2023-11-28T11:08:49.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa5060a0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa50006dee0 0x7fa500152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa5060a0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa500071040 0x7fa5000b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa5060a0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa500160540 0x7fa5000b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa5060a0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000b2e30 con 0x7fa500071040 2023-11-28T11:08:49.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa5060a0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa500153430 con 0x7fa500160540 2023-11-28T11:08:49.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa5060a0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa500153920 con 0x7fa50006dee0 2023-11-28T11:08:49.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa4feffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa500071040 0x7fa5000b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa4feffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa500071040 0x7fa5000b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32826/0 (socket says 172.21.15.156:32826) 2023-11-28T11:08:49.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa4feffd700 1 -- 172.21.15.156:0/1055260141 learned_addr learned my addr 172.21.15.156:0/1055260141 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:49.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa4ff7fe700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa50006dee0 0x7fa500152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa4fffff700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa500160540 0x7fa5000b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa4feffd700 1 -- 172.21.15.156:0/1055260141 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa50006dee0 msgr2=0x7fa500152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa4feffd700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa50006dee0 0x7fa500152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa4feffd700 1 -- 172.21.15.156:0/1055260141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa500160540 msgr2=0x7fa5000b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa4feffd700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa500160540 0x7fa5000b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa4feffd700 1 -- 172.21.15.156:0/1055260141 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e80094a0 con 0x7fa500071040 2023-11-28T11:08:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.285+0000 7fa4fffff700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa500160540 0x7fa5000b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.286+0000 7fa4feffd700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa500071040 0x7fa5000b0a50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80090b0 tx=0x7fa4e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.286+0000 7fa4f7fff700 1 -- 172.21.15.156:0/1055260141 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e80045d0 con 0x7fa500071040 2023-11-28T11:08:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.286+0000 7fa4f7fff700 1 -- 172.21.15.156:0/1055260141 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e801e070 con 0x7fa500071040 2023-11-28T11:08:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.286+0000 7fa5060a0700 1 -- 172.21.15.156:0/1055260141 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50015ccd0 con 0x7fa500071040 2023-11-28T11:08:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.286+0000 7fa4f7fff700 1 -- 172.21.15.156:0/1055260141 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e80038e0 con 0x7fa500071040 2023-11-28T11:08:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.286+0000 7fa5060a0700 1 -- 172.21.15.156:0/1055260141 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50006e870 con 0x7fa500071040 2023-11-28T11:08:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.287+0000 7fa5060a0700 1 -- 172.21.15.156:0/1055260141 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50006a270 con 0x7fa500071040 2023-11-28T11:08:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.287+0000 7fa4f7fff700 1 -- 172.21.15.156:0/1055260141 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa4e800f050 con 0x7fa500071040 2023-11-28T11:08:49.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.288+0000 7fa4f7fff700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4e00682f0 0x7fa4e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.288+0000 7fa4ff7fe700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4e00682f0 0x7fa4e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.288+0000 7fa4f7fff700 1 -- 172.21.15.156:0/1055260141 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa4e8086cb0 con 0x7fa500071040 2023-11-28T11:08:49.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.288+0000 7fa4ff7fe700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4e00682f0 0x7fa4e006a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0000f60 tx=0x7fa4f0007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.292+0000 7fa4f7fff700 1 -- 172.21.15.156:0/1055260141 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa4e8053510 con 0x7fa500071040 2023-11-28T11:08:49.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.478+0000 7fa5060a0700 1 -- 172.21.15.156:0/1055260141 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5000720a0 con 0x7fa500071040 2023-11-28T11:08:49.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.479+0000 7fa4f7fff700 1 -- 172.21.15.156:0/1055260141 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4e80150a0 con 0x7fa500071040 2023-11-28T11:08:49.480 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:08:49.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.480+0000 7fa5060a0700 1 -- 172.21.15.156:0/1055260141 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4e00682f0 msgr2=0x7fa4e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.480+0000 7fa5060a0700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4e00682f0 0x7fa4e006a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0000f60 tx=0x7fa4f0007e70 comp rx=0 tx=0).stop 2023-11-28T11:08:49.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.480+0000 7fa5060a0700 1 -- 172.21.15.156:0/1055260141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa500071040 msgr2=0x7fa5000b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.480+0000 7fa5060a0700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa500071040 0x7fa5000b0a50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80090b0 tx=0x7fa4e8004750 comp rx=0 tx=0).stop 2023-11-28T11:08:49.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.480+0000 7fa5060a0700 1 -- 172.21.15.156:0/1055260141 shutdown_connections 2023-11-28T11:08:49.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.480+0000 7fa5060a0700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa500160540 0x7fa5000b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.481+0000 7fa5060a0700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa4e00682f0 0x7fa4e006a7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.481+0000 7fa5060a0700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa50006dee0 0x7fa500152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.481+0000 7fa5060a0700 1 --2- 172.21.15.156:0/1055260141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa500071040 0x7fa5000b0a50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.481+0000 7fa5060a0700 1 -- 172.21.15.156:0/1055260141 >> 172.21.15.156:0/1055260141 conn(0x7fa5000d0000 msgr2=0x7fa50015e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:49.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.481+0000 7fa5060a0700 1 -- 172.21.15.156:0/1055260141 shutdown_connections 2023-11-28T11:08:49.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.481+0000 7fa5060a0700 1 -- 172.21.15.156:0/1055260141 wait complete. 2023-11-28T11:08:49.509 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:08:49.509 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T11:08:49.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.630+0000 7f45eaf1a700 1 Processor -- start 2023-11-28T11:08:49.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.630+0000 7f45eaf1a700 1 -- start start 2023-11-28T11:08:49.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.630+0000 7f45eaf1a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f45e40cd8c0 0x7f45e4140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.630+0000 7f45eaf1a700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f45e406f850 0x7f45e406fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.630+0000 7f45eaf1a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e413ff00 con 0x7f45e40cdc90 2023-11-28T11:08:49.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.630+0000 7f45eaf1a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e41404c0 con 0x7f45e40cd8c0 2023-11-28T11:08:49.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.630+0000 7f45eaf1a700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e415aa80 con 0x7f45e406f850 2023-11-28T11:08:49.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e8cb6700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f45e40cd8c0 0x7f45e4140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e3fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f45e40cdc90 0x7f45e40b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47712/0 (socket says 172.21.15.156:47712) 2023-11-28T11:08:49.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e3fff700 1 -- 172.21.15.156:0/1371959166 learned_addr learned my addr 172.21.15.156:0/1371959166 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:49.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e94b7700 1 --2- 172.21.15.156:0/1371959166 >> v2:172.21.15.186:3300/0 conn(0x7f45e406f850 0x7f45e406fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e8cb6700 1 -- 172.21.15.156:0/1371959166 >> v2:172.21.15.186:3300/0 conn(0x7f45e406f850 msgr2=0x7f45e406fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:49.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e8cb6700 1 --2- 172.21.15.156:0/1371959166 >> v2:172.21.15.186:3300/0 conn(0x7f45e406f850 0x7f45e406fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e8cb6700 1 -- 172.21.15.156:0/1371959166 >> v1:172.21.15.156:6789/0 conn(0x7f45e40cdc90 legacy=0x7f45e40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e8cb6700 1 -- 172.21.15.156:0/1371959166 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e40b3040 con 0x7f45e40cd8c0 2023-11-28T11:08:49.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e37fe700 1 -- 172.21.15.156:0/1371959166 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007300204 0 0) 0x7f45e413ff00 con 0x7f45e40cdc90 2023-11-28T11:08:49.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e8cb6700 1 --2- 172.21.15.156:0/1371959166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e40cd8c0 0x7f45e4140950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f45e406ea90 tx=0x7f45d8009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee39f94c877be2f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e37fe700 1 -- 172.21.15.156:0/1371959166 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8005e30 con 0x7f45e40cd8c0 2023-11-28T11:08:49.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e37fe700 1 -- 172.21.15.156:0/1371959166 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d8006030 con 0x7f45e40cd8c0 2023-11-28T11:08:49.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.631+0000 7f45e37fe700 1 -- 172.21.15.156:0/1371959166 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d80063c0 con 0x7f45e40cd8c0 2023-11-28T11:08:49.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.632+0000 7f45eaf1a700 1 -- 172.21.15.156:0/1371959166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e40cd8c0 msgr2=0x7f45e4140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.632+0000 7f45eaf1a700 1 --2- 172.21.15.156:0/1371959166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e40cd8c0 0x7f45e4140950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f45e406ea90 tx=0x7f45d8009b00 comp rx=0 tx=0).stop 2023-11-28T11:08:49.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.632+0000 7f45eaf1a700 1 -- 172.21.15.156:0/1371959166 shutdown_connections 2023-11-28T11:08:49.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.632+0000 7f45eaf1a700 1 --2- 172.21.15.156:0/1371959166 >> v2:172.21.15.186:3300/0 conn(0x7f45e406f850 0x7f45e406fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.632+0000 7f45eaf1a700 1 --2- 172.21.15.156:0/1371959166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e40cd8c0 0x7f45e4140950 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.632+0000 7f45eaf1a700 1 -- 172.21.15.156:0/1371959166 >> 172.21.15.156:0/1371959166 conn(0x7f45e406aed0 msgr2=0x7f45e406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:49.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.632+0000 7f45eaf1a700 1 -- 172.21.15.156:0/1371959166 shutdown_connections 2023-11-28T11:08:49.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.632+0000 7f45eaf1a700 1 -- 172.21.15.156:0/1371959166 wait complete. 2023-11-28T11:08:49.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.633+0000 7f45eaf1a700 1 Processor -- start 2023-11-28T11:08:49.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.633+0000 7f45eaf1a700 1 -- start start 2023-11-28T11:08:49.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45eaf1a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e406f850 0x7f45e40b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45eaf1a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45e40cd8c0 0x7f45e4141b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45eaf1a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45e40cdc90 0x7f45e413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45eaf1a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e406b760 con 0x7f45e406f850 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45eaf1a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e4155620 con 0x7f45e40cd8c0 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45eaf1a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40b1310 con 0x7f45e40cdc90 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45e8cb6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e406f850 0x7f45e40b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45e8cb6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e406f850 0x7f45e40b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32838/0 (socket says 172.21.15.156:32838) 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45e8cb6700 1 -- 172.21.15.156:0/231511309 learned_addr learned my addr 172.21.15.156:0/231511309 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45e3fff700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45e40cd8c0 0x7f45e4141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45e94b7700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45e40cdc90 0x7f45e413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45e8cb6700 1 -- 172.21.15.156:0/231511309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45e40cdc90 msgr2=0x7f45e413ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45e8cb6700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45e40cdc90 0x7f45e413ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45e8cb6700 1 -- 172.21.15.156:0/231511309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45e40cd8c0 msgr2=0x7f45e4141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45e8cb6700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45e40cd8c0 0x7f45e4141b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.634+0000 7f45e8cb6700 1 -- 172.21.15.156:0/231511309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d8009430 con 0x7f45e406f850 2023-11-28T11:08:49.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.635+0000 7f45e8cb6700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e406f850 0x7f45e40b0a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009fd0 tx=0x7f45d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.635+0000 7f45e17fa700 1 -- 172.21.15.156:0/231511309 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8006870 con 0x7f45e406f850 2023-11-28T11:08:49.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.635+0000 7f45e17fa700 1 -- 172.21.15.156:0/231511309 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d8020070 con 0x7f45e406f850 2023-11-28T11:08:49.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.635+0000 7f45eaf1a700 1 -- 172.21.15.156:0/231511309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e40b35f0 con 0x7f45e406f850 2023-11-28T11:08:49.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.635+0000 7f45e17fa700 1 -- 172.21.15.156:0/231511309 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8004040 con 0x7f45e406f850 2023-11-28T11:08:49.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.635+0000 7f45eaf1a700 1 -- 172.21.15.156:0/231511309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e40b3310 con 0x7f45e406f850 2023-11-28T11:08:49.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.636+0000 7f45eaf1a700 1 -- 172.21.15.156:0/231511309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e4140150 con 0x7f45e406f850 2023-11-28T11:08:49.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.636+0000 7f45e17fa700 1 -- 172.21.15.156:0/231511309 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f45d802b020 con 0x7f45e406f850 2023-11-28T11:08:49.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.636+0000 7f45e17fa700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f45c80682f0 0x7f45c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.637+0000 7f45e17fa700 1 -- 172.21.15.156:0/231511309 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f45d80872b0 con 0x7f45e406f850 2023-11-28T11:08:49.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.637+0000 7f45e3fff700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f45c80682f0 0x7f45c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.637+0000 7f45e3fff700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f45c80682f0 0x7f45c806a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f45e41e3910 tx=0x7f45d0007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.641+0000 7f45e17fa700 1 -- 172.21.15.156:0/231511309 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f45d8053b10 con 0x7f45e406f850 2023-11-28T11:08:49.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:49.897+0000 7f45eaf1a700 1 -- 172.21.15.156:0/231511309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f45e4068690 con 0x7f45e406f850 2023-11-28T11:08:50.505 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:08:50.508 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:08:50.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.831+0000 7f45e17fa700 1 -- 172.21.15.156:0/231511309 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v49) v1 ==== 145+0+0 (secure 0 0 0) 0x7f45d8057130 con 0x7f45e406f850 2023-11-28T11:08:50.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.834+0000 7f45eaf1a700 1 -- 172.21.15.156:0/231511309 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f45c80682f0 msgr2=0x7f45c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.834+0000 7f45eaf1a700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f45c80682f0 0x7f45c806a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f45e41e3910 tx=0x7f45d0007e70 comp rx=0 tx=0).stop 2023-11-28T11:08:50.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.834+0000 7f45eaf1a700 1 -- 172.21.15.156:0/231511309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e406f850 msgr2=0x7f45e40b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.834+0000 7f45eaf1a700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e406f850 0x7f45e40b0a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009fd0 tx=0x7f45d8004750 comp rx=0 tx=0).stop 2023-11-28T11:08:50.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.834+0000 7f45eaf1a700 1 -- 172.21.15.156:0/231511309 shutdown_connections 2023-11-28T11:08:50.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.834+0000 7f45eaf1a700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f45c80682f0 0x7f45c806a7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.834+0000 7f45eaf1a700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45e40cdc90 0x7f45e413ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.834+0000 7f45eaf1a700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e406f850 0x7f45e40b0a10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.834+0000 7f45eaf1a700 1 --2- 172.21.15.156:0/231511309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45e40cd8c0 0x7f45e4141b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.834+0000 7f45eaf1a700 1 -- 172.21.15.156:0/231511309 >> 172.21.15.156:0/231511309 conn(0x7f45e406aed0 msgr2=0x7f45e40cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:50.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.834+0000 7f45eaf1a700 1 -- 172.21.15.156:0/231511309 shutdown_connections 2023-11-28T11:08:50.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.834+0000 7f45eaf1a700 1 -- 172.21.15.156:0/231511309 wait complete. 2023-11-28T11:08:50.850 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:08:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.978+0000 7f03ea909700 1 Processor -- start 2023-11-28T11:08:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.978+0000 7f03ea909700 1 -- start start 2023-11-28T11:08:50.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03ea909700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f03e406ba90 0x7f03e40af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03ea909700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e40cc5f0 con 0x7f03e415cef0 2023-11-28T11:08:50.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03ea909700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e415d470 con 0x7f03e40cbdf0 2023-11-28T11:08:50.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03ea909700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e415d750 con 0x7f03e406ba90 2023-11-28T11:08:50.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03e37fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f03e406ba90 0x7f03e40af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03e3fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f03e415cef0 0x7f03e406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47724/0 (socket says 172.21.15.156:47724) 2023-11-28T11:08:50.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03e37fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f03e406ba90 0x7f03e40af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32848/0 (socket says 172.21.15.156:32848) 2023-11-28T11:08:50.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03e3fff700 1 -- 172.21.15.156:0/3913220271 learned_addr learned my addr 172.21.15.156:0/3913220271 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:50.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03e37fe700 1 -- 172.21.15.156:0/3913220271 >> v1:172.21.15.186:6789/0 conn(0x7f03e40cbdf0 legacy=0x7f03e40ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03e37fe700 1 -- 172.21.15.156:0/3913220271 >> v1:172.21.15.156:6789/0 conn(0x7f03e415cef0 legacy=0x7f03e406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03e37fe700 1 -- 172.21.15.156:0/3913220271 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e415da30 con 0x7f03e406ba90 2023-11-28T11:08:50.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03e2ffd700 1 -- 172.21.15.156:0/3913220271 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775643167 0 0) 0x7f03e40cc5f0 con 0x7f03e415cef0 2023-11-28T11:08:50.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03e37fe700 1 --2- 172.21.15.156:0/3913220271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03e406ba90 0x7f03e40af400 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009840 tx=0x7f03d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b6fb7133901c43 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03e2ffd700 1 -- 172.21.15.156:0/3913220271 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8011830 con 0x7f03e406ba90 2023-11-28T11:08:50.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03e2ffd700 1 -- 172.21.15.156:0/3913220271 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d8011990 con 0x7f03e406ba90 2023-11-28T11:08:50.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.979+0000 7f03e2ffd700 1 -- 172.21.15.156:0/3913220271 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8011d20 con 0x7f03e406ba90 2023-11-28T11:08:50.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.980+0000 7f03ea909700 1 -- 172.21.15.156:0/3913220271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03e406ba90 msgr2=0x7f03e40af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.980+0000 7f03ea909700 1 --2- 172.21.15.156:0/3913220271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03e406ba90 0x7f03e40af400 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009840 tx=0x7f03d8009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:50.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.980+0000 7f03ea909700 1 -- 172.21.15.156:0/3913220271 shutdown_connections 2023-11-28T11:08:50.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.980+0000 7f03ea909700 1 --2- 172.21.15.156:0/3913220271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03e406ba90 0x7f03e40af400 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.980+0000 7f03ea909700 1 -- 172.21.15.156:0/3913220271 >> 172.21.15.156:0/3913220271 conn(0x7f03e40d0000 msgr2=0x7f03e40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:50.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.980+0000 7f03ea909700 1 -- 172.21.15.156:0/3913220271 shutdown_connections 2023-11-28T11:08:50.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.980+0000 7f03ea909700 1 -- 172.21.15.156:0/3913220271 wait complete. 2023-11-28T11:08:50.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.981+0000 7f03ea909700 1 Processor -- start 2023-11-28T11:08:50.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.981+0000 7f03ea909700 1 -- start start 2023-11-28T11:08:50.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.982+0000 7f03ea909700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03e406ba90 0x7f03e40c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.982+0000 7f03ea909700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03e40cbdf0 0x7f03e41f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.982+0000 7f03ea909700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03e415cef0 0x7f03e41f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.982+0000 7f03ea909700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e41f26a0 con 0x7f03e415cef0 2023-11-28T11:08:50.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.982+0000 7f03ea909700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e41f6720 con 0x7f03e406ba90 2023-11-28T11:08:50.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.982+0000 7f03ea909700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e41f6160 con 0x7f03e40cbdf0 2023-11-28T11:08:50.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.982+0000 7f03e8ea6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03e415cef0 0x7f03e41f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.982+0000 7f03e8ea6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03e415cef0 0x7f03e41f5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32856/0 (socket says 172.21.15.156:32856) 2023-11-28T11:08:50.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.982+0000 7f03e8ea6700 1 -- 172.21.15.156:0/2481327826 learned_addr learned my addr 172.21.15.156:0/2481327826 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:50.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.982+0000 7f03e3fff700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03e406ba90 0x7f03e40c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.982+0000 7f03e37fe700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03e40cbdf0 0x7f03e41f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.982+0000 7f03e8ea6700 1 -- 172.21.15.156:0/2481327826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03e40cbdf0 msgr2=0x7f03e41f2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.983+0000 7f03e8ea6700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03e40cbdf0 0x7f03e41f2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.983+0000 7f03e8ea6700 1 -- 172.21.15.156:0/2481327826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03e406ba90 msgr2=0x7f03e40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.983+0000 7f03e8ea6700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03e406ba90 0x7f03e40c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.983+0000 7f03e8ea6700 1 -- 172.21.15.156:0/2481327826 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d80094a0 con 0x7f03e415cef0 2023-11-28T11:08:50.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.983+0000 7f03e3fff700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03e406ba90 0x7f03e40c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:50.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.983+0000 7f03e37fe700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03e40cbdf0 0x7f03e41f2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:50.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.983+0000 7f03e8ea6700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03e415cef0 0x7f03e41f5810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00cc50 tx=0x7f03dc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.983+0000 7f03e0ff9700 1 -- 172.21.15.156:0/2481327826 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00ad80 con 0x7f03e415cef0 2023-11-28T11:08:50.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.983+0000 7f03e0ff9700 1 -- 172.21.15.156:0/2481327826 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc00ce50 con 0x7f03e415cef0 2023-11-28T11:08:50.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.983+0000 7f03ea909700 1 -- 172.21.15.156:0/2481327826 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e415dff0 con 0x7f03e415cef0 2023-11-28T11:08:50.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.983+0000 7f03e0ff9700 1 -- 172.21.15.156:0/2481327826 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc003c40 con 0x7f03e415cef0 2023-11-28T11:08:50.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.983+0000 7f03ea909700 1 -- 172.21.15.156:0/2481327826 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e415dd10 con 0x7f03e415cef0 2023-11-28T11:08:50.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.984+0000 7f03ea909700 1 -- 172.21.15.156:0/2481327826 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e40cad60 con 0x7f03e415cef0 2023-11-28T11:08:50.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.984+0000 7f03e0ff9700 1 -- 172.21.15.156:0/2481327826 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f03dc010050 con 0x7f03e415cef0 2023-11-28T11:08:50.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.985+0000 7f03e0ff9700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f03d00682f0 0x7f03d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.985+0000 7f03e0ff9700 1 -- 172.21.15.156:0/2481327826 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03dc085d10 con 0x7f03e415cef0 2023-11-28T11:08:50.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.985+0000 7f03e3fff700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f03d00682f0 0x7f03d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.985+0000 7f03e3fff700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f03d00682f0 0x7f03d006a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f03d4000f60 tx=0x7f03d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:50.989+0000 7f03e0ff9700 1 -- 172.21.15.156:0/2481327826 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f03dc052480 con 0x7f03e415cef0 2023-11-28T11:08:51.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.172+0000 7f03ea909700 1 -- 172.21.15.156:0/2481327826 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f03e41f27e0 con 0x7f03e415cef0 2023-11-28T11:08:51.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.176+0000 7f03e0ff9700 1 -- 172.21.15.156:0/2481327826 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f03dc055aa0 con 0x7f03e415cef0 2023-11-28T11:08:51.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.178+0000 7f03ea909700 1 -- 172.21.15.156:0/2481327826 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f03d00682f0 msgr2=0x7f03d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.178+0000 7f03ea909700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f03d00682f0 0x7f03d006a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f03d4000f60 tx=0x7f03d4009040 comp rx=0 tx=0).stop 2023-11-28T11:08:51.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.178+0000 7f03ea909700 1 -- 172.21.15.156:0/2481327826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03e415cef0 msgr2=0x7f03e41f5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.178+0000 7f03ea909700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03e415cef0 0x7f03e41f5810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00cc50 tx=0x7f03dc00a340 comp rx=0 tx=0).stop 2023-11-28T11:08:51.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.178+0000 7f03ea909700 1 -- 172.21.15.156:0/2481327826 shutdown_connections 2023-11-28T11:08:51.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.178+0000 7f03ea909700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03e415cef0 0x7f03e41f5810 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.178+0000 7f03ea909700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03e406ba90 0x7f03e40c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.178+0000 7f03ea909700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f03d00682f0 0x7f03d006a7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.178+0000 7f03ea909700 1 --2- 172.21.15.156:0/2481327826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03e40cbdf0 0x7f03e41f2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.178+0000 7f03ea909700 1 -- 172.21.15.156:0/2481327826 >> 172.21.15.156:0/2481327826 conn(0x7f03e40d0000 msgr2=0x7f03e40b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:51.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.178+0000 7f03ea909700 1 -- 172.21.15.156:0/2481327826 shutdown_connections 2023-11-28T11:08:51.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.178+0000 7f03ea909700 1 -- 172.21.15.156:0/2481327826 wait complete. 2023-11-28T11:08:51.194 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:08:51.194 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:08:51.238 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:08:51.289 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:08:51.289 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:08:51.289 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:51.307 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:08:51.307 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:51.365 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:08:51.365 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:51.422 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:51.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.582+0000 7fa77f244700 1 Processor -- start 2023-11-28T11:08:51.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.582+0000 7fa77f244700 1 -- start start 2023-11-28T11:08:51.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.583+0000 7fa77f244700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa778140950 0x7fa7780b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.583+0000 7fa77f244700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa7780b2e70 0x7fa77806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.583+0000 7fa77f244700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77813ff00 con 0x7fa7780cd8c0 2023-11-28T11:08:51.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.583+0000 7fa77f244700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7781404c0 con 0x7fa7780b2e70 2023-11-28T11:08:51.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.583+0000 7fa77f244700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa778140dc0 con 0x7fa778140950 2023-11-28T11:08:51.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.583+0000 7fa77d7e1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa7780b2e70 0x7fa77806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.583+0000 7fa77cfe0700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa7780cd8c0 0x7fa7780cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47738/0 (socket says 172.21.15.156:47738) 2023-11-28T11:08:51.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.583+0000 7fa77cfe0700 1 -- 172.21.15.156:0/104003160 learned_addr learned my addr 172.21.15.156:0/104003160 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:51.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.584+0000 7fa777fff700 1 --2- 172.21.15.156:0/104003160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa778140950 0x7fa7780b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.584+0000 7fa77d7e1700 1 -- 172.21.15.156:0/104003160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa778140950 msgr2=0x7fa7780b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:51.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.584+0000 7fa77d7e1700 1 --2- 172.21.15.156:0/104003160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa778140950 0x7fa7780b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.584+0000 7fa77d7e1700 1 -- 172.21.15.156:0/104003160 >> v1:172.21.15.156:6789/0 conn(0x7fa7780cd8c0 legacy=0x7fa7780cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.584+0000 7fa77d7e1700 1 -- 172.21.15.156:0/104003160 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77815aa80 con 0x7fa7780b2e70 2023-11-28T11:08:51.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.584+0000 7fa7777fe700 1 -- 172.21.15.156:0/104003160 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574694218 0 0) 0x7fa77813ff00 con 0x7fa7780cd8c0 2023-11-28T11:08:51.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.584+0000 7fa777fff700 1 --2- 172.21.15.156:0/104003160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa778140950 0x7fa7780b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:51.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.584+0000 7fa77d7e1700 1 --2- 172.21.15.156:0/104003160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780b2e70 0x7fa77806f850 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa770009d90 tx=0x7fa77000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4724dd3d6ddfbbda server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.584+0000 7fa7777fe700 1 -- 172.21.15.156:0/104003160 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77000e070 con 0x7fa7780b2e70 2023-11-28T11:08:51.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.584+0000 7fa7777fe700 1 -- 172.21.15.156:0/104003160 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77000bbd0 con 0x7fa7780b2e70 2023-11-28T11:08:51.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.584+0000 7fa7777fe700 1 -- 172.21.15.156:0/104003160 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa770012430 con 0x7fa7780b2e70 2023-11-28T11:08:51.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.585+0000 7fa77f244700 1 -- 172.21.15.156:0/104003160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780b2e70 msgr2=0x7fa77806f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.585+0000 7fa77f244700 1 --2- 172.21.15.156:0/104003160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780b2e70 0x7fa77806f850 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa770009d90 tx=0x7fa77000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:51.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.585+0000 7fa77f244700 1 -- 172.21.15.156:0/104003160 shutdown_connections 2023-11-28T11:08:51.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.585+0000 7fa77f244700 1 --2- 172.21.15.156:0/104003160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780b2e70 0x7fa77806f850 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.585+0000 7fa77f244700 1 --2- 172.21.15.156:0/104003160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa778140950 0x7fa7780b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.585+0000 7fa77f244700 1 -- 172.21.15.156:0/104003160 >> 172.21.15.156:0/104003160 conn(0x7fa77806aed0 msgr2=0x7fa77806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:51.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.585+0000 7fa77f244700 1 -- 172.21.15.156:0/104003160 shutdown_connections 2023-11-28T11:08:51.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.585+0000 7fa77f244700 1 -- 172.21.15.156:0/104003160 wait complete. 2023-11-28T11:08:51.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.586+0000 7fa77f244700 1 Processor -- start 2023-11-28T11:08:51.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.586+0000 7fa77f244700 1 -- start start 2023-11-28T11:08:51.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77f244700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780b2e70 0x7fa77815d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77f244700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780cd8c0 0x7fa7780b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77f244700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa778140950 0x7fa7780b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77f244700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77813fd70 con 0x7fa778140950 2023-11-28T11:08:51.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77f244700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77813f0f0 con 0x7fa7780cd8c0 2023-11-28T11:08:51.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77f244700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77813f3d0 con 0x7fa7780b2e70 2023-11-28T11:08:51.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77d7e1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa778140950 0x7fa7780b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77d7e1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa778140950 0x7fa7780b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32868/0 (socket says 172.21.15.156:32868) 2023-11-28T11:08:51.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77d7e1700 1 -- 172.21.15.156:0/2811832917 learned_addr learned my addr 172.21.15.156:0/2811832917 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:51.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa777fff700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780cd8c0 0x7fa7780b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77cfe0700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780b2e70 0x7fa77815d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77d7e1700 1 -- 172.21.15.156:0/2811832917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780b2e70 msgr2=0x7fa77815d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77d7e1700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780b2e70 0x7fa77815d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77d7e1700 1 -- 172.21.15.156:0/2811832917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780cd8c0 msgr2=0x7fa7780b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77d7e1700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780cd8c0 0x7fa7780b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77d7e1700 1 -- 172.21.15.156:0/2811832917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa770009a70 con 0x7fa778140950 2023-11-28T11:08:51.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa777fff700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780cd8c0 0x7fa7780b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:51.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77cfe0700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780b2e70 0x7fa77815d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:51.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.587+0000 7fa77d7e1700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa778140950 0x7fa7780b5b30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa770013000 tx=0x7fa770004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.588+0000 7fa7757fa700 1 -- 172.21.15.156:0/2811832917 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa770003ea0 con 0x7fa778140950 2023-11-28T11:08:51.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.588+0000 7fa7757fa700 1 -- 172.21.15.156:0/2811832917 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa770020070 con 0x7fa778140950 2023-11-28T11:08:51.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.588+0000 7fa77f244700 1 -- 172.21.15.156:0/2811832917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7780b35f0 con 0x7fa778140950 2023-11-28T11:08:51.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.588+0000 7fa7757fa700 1 -- 172.21.15.156:0/2811832917 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa770004280 con 0x7fa778140950 2023-11-28T11:08:51.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.588+0000 7fa77f244700 1 -- 172.21.15.156:0/2811832917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7780b3310 con 0x7fa778140950 2023-11-28T11:08:51.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.588+0000 7fa77f244700 1 -- 172.21.15.156:0/2811832917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa778140150 con 0x7fa778140950 2023-11-28T11:08:51.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.589+0000 7fa7757fa700 1 -- 172.21.15.156:0/2811832917 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa7700043e0 con 0x7fa778140950 2023-11-28T11:08:51.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.589+0000 7fa7757fa700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa75c0682f0 0x7fa75c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.590+0000 7fa77cfe0700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa75c0682f0 0x7fa75c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.590+0000 7fa7757fa700 1 -- 172.21.15.156:0/2811832917 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa770022070 con 0x7fa778140950 2023-11-28T11:08:51.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.590+0000 7fa77cfe0700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa75c0682f0 0x7fa75c06a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0010e0 tx=0x7fa76c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.594+0000 7fa7757fa700 1 -- 172.21.15.156:0/2811832917 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa770054a80 con 0x7fa778140950 2023-11-28T11:08:51.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.777+0000 7fa77f244700 1 -- 172.21.15.156:0/2811832917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa778068690 con 0x7fa778140950 2023-11-28T11:08:51.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.778+0000 7fa7757fa700 1 -- 172.21.15.156:0/2811832917 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17955 (secure 0 0 0) 0x7fa7700263e0 con 0x7fa778140950 2023-11-28T11:08:51.779 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:51.779 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:18.354009+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:51.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.780+0000 7fa77f244700 1 -- 172.21.15.156:0/2811832917 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa75c0682f0 msgr2=0x7fa75c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.780+0000 7fa77f244700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa75c0682f0 0x7fa75c06a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0010e0 tx=0x7fa76c009040 comp rx=0 tx=0).stop 2023-11-28T11:08:51.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.780+0000 7fa77f244700 1 -- 172.21.15.156:0/2811832917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa778140950 msgr2=0x7fa7780b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.780+0000 7fa77f244700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa778140950 0x7fa7780b5b30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa770013000 tx=0x7fa770004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.780+0000 7fa77f244700 1 -- 172.21.15.156:0/2811832917 shutdown_connections 2023-11-28T11:08:51.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.780+0000 7fa77f244700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa778140950 0x7fa7780b5b30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.780+0000 7fa77f244700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa75c0682f0 0x7fa75c06a7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.780+0000 7fa77f244700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780b2e70 0x7fa77815d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.780+0000 7fa77f244700 1 --2- 172.21.15.156:0/2811832917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780cd8c0 0x7fa7780b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.780+0000 7fa77f244700 1 -- 172.21.15.156:0/2811832917 >> 172.21.15.156:0/2811832917 conn(0x7fa77806aed0 msgr2=0x7fa7780b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:51.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.780+0000 7fa77f244700 1 -- 172.21.15.156:0/2811832917 shutdown_connections 2023-11-28T11:08:51.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.780+0000 7fa77f244700 1 -- 172.21.15.156:0/2811832917 wait complete. 2023-11-28T11:08:51.793 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:15.021591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:16.332241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:51.794 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:51.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.916+0000 7fdcbda8c700 1 Processor -- start 2023-11-28T11:08:51.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.916+0000 7fdcbda8c700 1 -- start start 2023-11-28T11:08:51.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.917+0000 7fdcbda8c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdcb8071040 0x7fdcb8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.917+0000 7fdcbda8c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdcb8160540 0x7fdcb8160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.917+0000 7fdcbda8c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb8071950 con 0x7fdcb806dee0 2023-11-28T11:08:51.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.917+0000 7fdcbda8c700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb815d060 con 0x7fdcb8160540 2023-11-28T11:08:51.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.917+0000 7fdcbda8c700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb815d340 con 0x7fdcb8071040 2023-11-28T11:08:51.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.917+0000 7fdcb7fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdcb8160540 0x7fdcb8160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.917+0000 7fdcb6ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fdcb806dee0 0x7fdcb806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47752/0 (socket says 172.21.15.156:47752) 2023-11-28T11:08:51.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.917+0000 7fdcb6ffd700 1 -- 172.21.15.156:0/1409592978 learned_addr learned my addr 172.21.15.156:0/1409592978 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:51.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcb77fe700 1 --2- 172.21.15.156:0/1409592978 >> v2:172.21.15.186:3300/0 conn(0x7fdcb8071040 0x7fdcb8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcb7fff700 1 -- 172.21.15.156:0/1409592978 >> v2:172.21.15.186:3300/0 conn(0x7fdcb8071040 msgr2=0x7fdcb8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:51.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcb7fff700 1 --2- 172.21.15.156:0/1409592978 >> v2:172.21.15.186:3300/0 conn(0x7fdcb8071040 0x7fdcb8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcb7fff700 1 -- 172.21.15.156:0/1409592978 >> v1:172.21.15.156:6789/0 conn(0x7fdcb806dee0 legacy=0x7fdcb806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcb7fff700 1 -- 172.21.15.156:0/1409592978 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb815d620 con 0x7fdcb8160540 2023-11-28T11:08:51.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcb67fc700 1 -- 172.21.15.156:0/1409592978 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219221369 0 0) 0x7fdcb8071950 con 0x7fdcb806dee0 2023-11-28T11:08:51.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcb77fe700 1 --2- 172.21.15.156:0/1409592978 >> v2:172.21.15.186:3300/0 conn(0x7fdcb8071040 0x7fdcb8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:51.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcb7fff700 1 --2- 172.21.15.156:0/1409592978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb8160540 0x7fdcb8160980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80d0f50 tx=0x7fdcb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea449e0ad672eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcb67fc700 1 -- 172.21.15.156:0/1409592978 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb000e070 con 0x7fdcb8160540 2023-11-28T11:08:51.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcb67fc700 1 -- 172.21.15.156:0/1409592978 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb000bbd0 con 0x7fdcb8160540 2023-11-28T11:08:51.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcb67fc700 1 -- 172.21.15.156:0/1409592978 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0012430 con 0x7fdcb8160540 2023-11-28T11:08:51.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1409592978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb8160540 msgr2=0x7fdcb8160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcbda8c700 1 --2- 172.21.15.156:0/1409592978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb8160540 0x7fdcb8160980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80d0f50 tx=0x7fdcb000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:51.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1409592978 shutdown_connections 2023-11-28T11:08:51.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcbda8c700 1 --2- 172.21.15.156:0/1409592978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb8160540 0x7fdcb8160980 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcbda8c700 1 --2- 172.21.15.156:0/1409592978 >> v2:172.21.15.186:3300/0 conn(0x7fdcb8071040 0x7fdcb8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1409592978 >> 172.21.15.156:0/1409592978 conn(0x7fdcb80d0000 msgr2=0x7fdcb80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:51.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1409592978 shutdown_connections 2023-11-28T11:08:51.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.918+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1409592978 wait complete. 2023-11-28T11:08:51.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.920+0000 7fdcbda8c700 1 Processor -- start 2023-11-28T11:08:51.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.920+0000 7fdcbda8c700 1 -- start start 2023-11-28T11:08:51.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcbda8c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb806dee0 0x7fdcb8158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcbda8c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcb8071040 0x7fdcb80c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcbda8c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcb8160540 0x7fdcb80cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcbda8c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c6750 con 0x7fdcb806dee0 2023-11-28T11:08:51.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcbda8c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80b40c0 con 0x7fdcb8160540 2023-11-28T11:08:51.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcbda8c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80b34b0 con 0x7fdcb8071040 2023-11-28T11:08:51.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcb77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb806dee0 0x7fdcb8158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcb77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb806dee0 0x7fdcb8158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32878/0 (socket says 172.21.15.156:32878) 2023-11-28T11:08:51.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcb77fe700 1 -- 172.21.15.156:0/1657673665 learned_addr learned my addr 172.21.15.156:0/1657673665 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:51.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcb7fff700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcb8160540 0x7fdcb80cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcb6ffd700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcb8071040 0x7fdcb80c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcb77fe700 1 -- 172.21.15.156:0/1657673665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcb8071040 msgr2=0x7fdcb80c54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcb77fe700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcb8071040 0x7fdcb80c54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcb77fe700 1 -- 172.21.15.156:0/1657673665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcb8160540 msgr2=0x7fdcb80cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcb77fe700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcb8160540 0x7fdcb80cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcb77fe700 1 -- 172.21.15.156:0/1657673665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb0009a70 con 0x7fdcb806dee0 2023-11-28T11:08:51.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcb6ffd700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcb8071040 0x7fdcb80c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:51.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcb77fe700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb806dee0 0x7fdcb8158620 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00c7f0 tx=0x7fdcac00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdc9ffff700 1 -- 172.21.15.156:0/1657673665 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac011800 con 0x7fdcb806dee0 2023-11-28T11:08:51.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdc9ffff700 1 -- 172.21.15.156:0/1657673665 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcac011960 con 0x7fdcb806dee0 2023-11-28T11:08:51.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdc9ffff700 1 -- 172.21.15.156:0/1657673665 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac011d10 con 0x7fdcb806dee0 2023-11-28T11:08:51.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1657673665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb815ccd0 con 0x7fdcb806dee0 2023-11-28T11:08:51.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.921+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1657673665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb815d900 con 0x7fdcb806dee0 2023-11-28T11:08:51.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.922+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1657673665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb806a270 con 0x7fdcb806dee0 2023-11-28T11:08:51.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.923+0000 7fdc9ffff700 1 -- 172.21.15.156:0/1657673665 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdcac021070 con 0x7fdcb806dee0 2023-11-28T11:08:51.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.923+0000 7fdc9ffff700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdca40682f0 0x7fdca406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.924+0000 7fdc9ffff700 1 -- 172.21.15.156:0/1657673665 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdcac086050 con 0x7fdcb806dee0 2023-11-28T11:08:51.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.924+0000 7fdcb6ffd700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdca40682f0 0x7fdca406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.924+0000 7fdcb6ffd700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdca40682f0 0x7fdca406a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdca8007fd0 tx=0x7fdca8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:51.927+0000 7fdc9ffff700 1 -- 172.21.15.156:0/1657673665 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdcac0527c0 con 0x7fdcb806dee0 2023-11-28T11:08:52.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.110+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1657673665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcb806b5c0 con 0x7fdcb806dee0 2023-11-28T11:08:52.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.110+0000 7fdc9ffff700 1 -- 172.21.15.156:0/1657673665 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcac01f0a0 con 0x7fdcb806dee0 2023-11-28T11:08:52.112 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:08:52.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.112+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1657673665 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdca40682f0 msgr2=0x7fdca406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.112+0000 7fdcbda8c700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdca40682f0 0x7fdca406a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdca8007fd0 tx=0x7fdca8009040 comp rx=0 tx=0).stop 2023-11-28T11:08:52.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.112+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1657673665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb806dee0 msgr2=0x7fdcb8158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.112+0000 7fdcbda8c700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb806dee0 0x7fdcb8158620 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00c7f0 tx=0x7fdcac00cbb0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.112+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1657673665 shutdown_connections 2023-11-28T11:08:52.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.112+0000 7fdcbda8c700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcb8160540 0x7fdcb80cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.112+0000 7fdcbda8c700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcb806dee0 0x7fdcb8158620 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.112+0000 7fdcbda8c700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdca40682f0 0x7fdca406a7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.112+0000 7fdcbda8c700 1 --2- 172.21.15.156:0/1657673665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcb8071040 0x7fdcb80c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.112+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1657673665 >> 172.21.15.156:0/1657673665 conn(0x7fdcb80d0000 msgr2=0x7fdcb80afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.112+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1657673665 shutdown_connections 2023-11-28T11:08:52.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.112+0000 7fdcbda8c700 1 -- 172.21.15.156:0/1657673665 wait complete. 2023-11-28T11:08:52.126 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:52.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.248+0000 7f04a6d7a700 1 Processor -- start 2023-11-28T11:08:52.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.248+0000 7f04a6d7a700 1 -- start start 2023-11-28T11:08:52.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.248+0000 7f04a6d7a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f04a0071040 0x7f04a0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.248+0000 7f04a6d7a700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f04a006dee0 0x7f04a006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f04a6d7a700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a0071950 con 0x7f04a006cdc0 2023-11-28T11:08:52.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f04a6d7a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a0161550 con 0x7f04a0071040 2023-11-28T11:08:52.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f04a6d7a700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a015d060 con 0x7f04a006dee0 2023-11-28T11:08:52.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f04a4b16700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f04a0071040 0x7f04a0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f04a4b16700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f04a0071040 0x7f04a0071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32886/0 (socket says 172.21.15.156:32886) 2023-11-28T11:08:52.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f04a4b16700 1 -- 172.21.15.156:0/519713209 learned_addr learned my addr 172.21.15.156:0/519713209 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:52.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f049ffff700 1 --2- 172.21.15.156:0/519713209 >> v2:172.21.15.186:3300/0 conn(0x7f04a006dee0 0x7f04a006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f04a4b16700 1 -- 172.21.15.156:0/519713209 >> v2:172.21.15.186:3300/0 conn(0x7f04a006dee0 msgr2=0x7f04a006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f04a4b16700 1 --2- 172.21.15.156:0/519713209 >> v2:172.21.15.186:3300/0 conn(0x7f04a006dee0 0x7f04a006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f04a4b16700 1 -- 172.21.15.156:0/519713209 >> v1:172.21.15.186:6789/0 conn(0x7f04a006cdc0 legacy=0x7f04a006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f04a4b16700 1 -- 172.21.15.156:0/519713209 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a015d340 con 0x7f04a0071040 2023-11-28T11:08:52.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f04a4b16700 1 --2- 172.21.15.156:0/519713209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04a0071040 0x7f04a0071410 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f04a006fb00 tx=0x7f0494009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840f449efc8985f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f049f7fe700 1 -- 172.21.15.156:0/519713209 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494004230 con 0x7f04a0071040 2023-11-28T11:08:52.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f049f7fe700 1 -- 172.21.15.156:0/519713209 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0494004390 con 0x7f04a0071040 2023-11-28T11:08:52.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.249+0000 7f049f7fe700 1 -- 172.21.15.156:0/519713209 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04940036a0 con 0x7f04a0071040 2023-11-28T11:08:52.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.250+0000 7f04a6d7a700 1 -- 172.21.15.156:0/519713209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04a0071040 msgr2=0x7f04a0071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.250+0000 7f04a6d7a700 1 --2- 172.21.15.156:0/519713209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04a0071040 0x7f04a0071410 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f04a006fb00 tx=0x7f0494009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.250+0000 7f04a6d7a700 1 -- 172.21.15.156:0/519713209 shutdown_connections 2023-11-28T11:08:52.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.250+0000 7f04a6d7a700 1 --2- 172.21.15.156:0/519713209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04a0071040 0x7f04a0071410 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.250+0000 7f04a6d7a700 1 --2- 172.21.15.156:0/519713209 >> v2:172.21.15.186:3300/0 conn(0x7f04a006dee0 0x7f04a006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.250+0000 7f04a6d7a700 1 -- 172.21.15.156:0/519713209 >> 172.21.15.156:0/519713209 conn(0x7f04a00d0000 msgr2=0x7f04a00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.250+0000 7f04a6d7a700 1 -- 172.21.15.156:0/519713209 shutdown_connections 2023-11-28T11:08:52.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.250+0000 7f04a6d7a700 1 -- 172.21.15.156:0/519713209 wait complete. 2023-11-28T11:08:52.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.251+0000 7f04a6d7a700 1 Processor -- start 2023-11-28T11:08:52.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.251+0000 7f04a6d7a700 1 -- start start 2023-11-28T11:08:52.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f04a6d7a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04a006cdc0 0x7f04a00b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f04a6d7a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04a006dee0 0x7f04a014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f04a6d7a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04a0071040 0x7f04a015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f04a6d7a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a00c3c20 con 0x7f04a006dee0 2023-11-28T11:08:52.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f04a6d7a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a00c3f00 con 0x7f04a0071040 2023-11-28T11:08:52.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f04a6d7a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a00c41e0 con 0x7f04a006cdc0 2023-11-28T11:08:52.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f049ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04a006dee0 0x7f04a014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f049ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04a006dee0 0x7f04a014e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32894/0 (socket says 172.21.15.156:32894) 2023-11-28T11:08:52.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f049ffff700 1 -- 172.21.15.156:0/481571061 learned_addr learned my addr 172.21.15.156:0/481571061 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:52.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f04a4b16700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04a006cdc0 0x7f04a00b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f04a5317700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04a0071040 0x7f04a015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f049ffff700 1 -- 172.21.15.156:0/481571061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04a006cdc0 msgr2=0x7f04a00b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f049ffff700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04a006cdc0 0x7f04a00b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f049ffff700 1 -- 172.21.15.156:0/481571061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04a0071040 msgr2=0x7f04a015a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f049ffff700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04a0071040 0x7f04a015a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f049ffff700 1 -- 172.21.15.156:0/481571061 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0494009550 con 0x7f04a006dee0 2023-11-28T11:08:52.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f04a5317700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04a0071040 0x7f04a015a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:52.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.252+0000 7f049ffff700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04a006dee0 0x7f04a014e2e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f048c00c7f0 tx=0x7f048c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.253+0000 7f049d7fa700 1 -- 172.21.15.156:0/481571061 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c011800 con 0x7f04a006dee0 2023-11-28T11:08:52.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.253+0000 7f049d7fa700 1 -- 172.21.15.156:0/481571061 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c011960 con 0x7f04a006dee0 2023-11-28T11:08:52.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.253+0000 7f04a6d7a700 1 -- 172.21.15.156:0/481571061 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a015d900 con 0x7f04a006dee0 2023-11-28T11:08:52.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.253+0000 7f049d7fa700 1 -- 172.21.15.156:0/481571061 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c011cf0 con 0x7f04a006dee0 2023-11-28T11:08:52.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.253+0000 7f04a6d7a700 1 -- 172.21.15.156:0/481571061 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a015d620 con 0x7f04a006dee0 2023-11-28T11:08:52.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.254+0000 7f04a6d7a700 1 -- 172.21.15.156:0/481571061 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a006e870 con 0x7f04a006dee0 2023-11-28T11:08:52.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.254+0000 7f049d7fa700 1 -- 172.21.15.156:0/481571061 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f048c021070 con 0x7f04a006dee0 2023-11-28T11:08:52.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.254+0000 7f049d7fa700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04840682f0 0x7f048406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.254+0000 7f049d7fa700 1 -- 172.21.15.156:0/481571061 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f048c085a40 con 0x7f04a006dee0 2023-11-28T11:08:52.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.254+0000 7f04a4b16700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04840682f0 0x7f048406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.255+0000 7f04a4b16700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04840682f0 0x7f048406a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0494009130 tx=0x7f0494008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.259+0000 7f049d7fa700 1 -- 172.21.15.156:0/481571061 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f048c0522a0 con 0x7f04a006dee0 2023-11-28T11:08:52.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.440+0000 7f04a6d7a700 1 -- 172.21.15.156:0/481571061 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f04a00cf4a0 con 0x7f04a006dee0 2023-11-28T11:08:52.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.440+0000 7f049d7fa700 1 -- 172.21.15.156:0/481571061 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f048c0558c0 con 0x7f04a006dee0 2023-11-28T11:08:52.442 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:08:52.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.442+0000 7f04a6d7a700 1 -- 172.21.15.156:0/481571061 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04840682f0 msgr2=0x7f048406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.442+0000 7f04a6d7a700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04840682f0 0x7f048406a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0494009130 tx=0x7f0494008030 comp rx=0 tx=0).stop 2023-11-28T11:08:52.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.442+0000 7f04a6d7a700 1 -- 172.21.15.156:0/481571061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04a006dee0 msgr2=0x7f04a014e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.442+0000 7f04a6d7a700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04a006dee0 0x7f04a014e2e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f048c00c7f0 tx=0x7f048c00cbb0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.442+0000 7f04a6d7a700 1 -- 172.21.15.156:0/481571061 shutdown_connections 2023-11-28T11:08:52.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.442+0000 7f04a6d7a700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04a0071040 0x7f04a015a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.442+0000 7f04a6d7a700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04840682f0 0x7f048406a7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.442+0000 7f04a6d7a700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04a006cdc0 0x7f04a00b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.442+0000 7f04a6d7a700 1 --2- 172.21.15.156:0/481571061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04a006dee0 0x7f04a014e2e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.442+0000 7f04a6d7a700 1 -- 172.21.15.156:0/481571061 >> 172.21.15.156:0/481571061 conn(0x7f04a00d0000 msgr2=0x7f04a00cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.442+0000 7f04a6d7a700 1 -- 172.21.15.156:0/481571061 shutdown_connections 2023-11-28T11:08:52.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.442+0000 7f04a6d7a700 1 -- 172.21.15.156:0/481571061 wait complete. 2023-11-28T11:08:52.456 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:52.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.578+0000 7f73e1ed1700 1 Processor -- start 2023-11-28T11:08:52.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.578+0000 7f73e1ed1700 1 -- start start 2023-11-28T11:08:52.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73e1ed1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f73dc0cd8c0 0x7f73dc140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73e1ed1700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f73dc0cdc90 0x7f73dc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73e1ed1700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc13ff00 con 0x7f73dc0b2e70 2023-11-28T11:08:52.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73e1ed1700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73dc1404c0 con 0x7f73dc0cd8c0 2023-11-28T11:08:52.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73e1ed1700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73dc15aa80 con 0x7f73dc0cdc90 2023-11-28T11:08:52.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73db7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f73dc0cd8c0 0x7f73dc140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73dbfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f73dc0b2e70 0x7f73dc06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47782/0 (socket says 172.21.15.156:47782) 2023-11-28T11:08:52.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73dbfff700 1 -- 172.21.15.156:0/4223085632 learned_addr learned my addr 172.21.15.156:0/4223085632 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:52.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73daffd700 1 --2- 172.21.15.156:0/4223085632 >> v2:172.21.15.186:3300/0 conn(0x7f73dc0cdc90 0x7f73dc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73db7fe700 1 -- 172.21.15.156:0/4223085632 >> v2:172.21.15.186:3300/0 conn(0x7f73dc0cdc90 msgr2=0x7f73dc0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73db7fe700 1 --2- 172.21.15.156:0/4223085632 >> v2:172.21.15.186:3300/0 conn(0x7f73dc0cdc90 0x7f73dc0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73db7fe700 1 -- 172.21.15.156:0/4223085632 >> v1:172.21.15.156:6789/0 conn(0x7f73dc0b2e70 legacy=0x7f73dc06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73db7fe700 1 -- 172.21.15.156:0/4223085632 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc0b3310 con 0x7f73dc0cd8c0 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73da7fc700 1 -- 172.21.15.156:0/4223085632 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386370202 0 0) 0x7f73dc13ff00 con 0x7f73dc0b2e70 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73db7fe700 1 --2- 172.21.15.156:0/4223085632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73dc0cd8c0 0x7f73dc140950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73dc071c40 tx=0x7f73d0009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32118de574e22d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.579+0000 7f73da7fc700 1 -- 172.21.15.156:0/4223085632 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0005e30 con 0x7f73dc0cd8c0 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.580+0000 7f73da7fc700 1 -- 172.21.15.156:0/4223085632 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d0006030 con 0x7f73dc0cd8c0 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.580+0000 7f73da7fc700 1 -- 172.21.15.156:0/4223085632 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d00063c0 con 0x7f73dc0cd8c0 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.580+0000 7f73e1ed1700 1 -- 172.21.15.156:0/4223085632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73dc0cd8c0 msgr2=0x7f73dc140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.580+0000 7f73e1ed1700 1 --2- 172.21.15.156:0/4223085632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73dc0cd8c0 0x7f73dc140950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73dc071c40 tx=0x7f73d0009b00 comp rx=0 tx=0).stop 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.580+0000 7f73e1ed1700 1 -- 172.21.15.156:0/4223085632 shutdown_connections 2023-11-28T11:08:52.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.580+0000 7f73e1ed1700 1 --2- 172.21.15.156:0/4223085632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73dc0cd8c0 0x7f73dc140950 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.580+0000 7f73e1ed1700 1 --2- 172.21.15.156:0/4223085632 >> v2:172.21.15.186:3300/0 conn(0x7f73dc0cdc90 0x7f73dc0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.580+0000 7f73e1ed1700 1 -- 172.21.15.156:0/4223085632 >> 172.21.15.156:0/4223085632 conn(0x7f73dc06aed0 msgr2=0x7f73dc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.580+0000 7f73e1ed1700 1 -- 172.21.15.156:0/4223085632 shutdown_connections 2023-11-28T11:08:52.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.580+0000 7f73e1ed1700 1 -- 172.21.15.156:0/4223085632 wait complete. 2023-11-28T11:08:52.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.581+0000 7f73e1ed1700 1 Processor -- start 2023-11-28T11:08:52.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.581+0000 7f73e1ed1700 1 -- start start 2023-11-28T11:08:52.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.582+0000 7f73e1ed1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73dc0b2e70 0x7f73dc15ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.582+0000 7f73e1ed1700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73dc0cd8c0 0x7f73dc140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.582+0000 7f73e1ed1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73dc0cdc90 0x7f73dc15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.582+0000 7f73e1ed1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc0c7810 con 0x7f73dc0cdc90 2023-11-28T11:08:52.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.582+0000 7f73e1ed1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc15b270 con 0x7f73dc0b2e70 2023-11-28T11:08:52.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.582+0000 7f73e1ed1700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc15b550 con 0x7f73dc0cd8c0 2023-11-28T11:08:52.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.582+0000 7f73dbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73dc0cdc90 0x7f73dc15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.582+0000 7f73dbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73dc0cdc90 0x7f73dc15d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32910/0 (socket says 172.21.15.156:32910) 2023-11-28T11:08:52.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.582+0000 7f73dbfff700 1 -- 172.21.15.156:0/2819467754 learned_addr learned my addr 172.21.15.156:0/2819467754 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:52.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.582+0000 7f73daffd700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73dc0cd8c0 0x7f73dc140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.582+0000 7f73db7fe700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73dc0b2e70 0x7f73dc15ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.583+0000 7f73dbfff700 1 -- 172.21.15.156:0/2819467754 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73dc0cd8c0 msgr2=0x7f73dc140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.583+0000 7f73dbfff700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73dc0cd8c0 0x7f73dc140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.583+0000 7f73dbfff700 1 -- 172.21.15.156:0/2819467754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73dc0b2e70 msgr2=0x7f73dc15ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.583+0000 7f73dbfff700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73dc0b2e70 0x7f73dc15ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.583+0000 7f73dbfff700 1 -- 172.21.15.156:0/2819467754 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d0009430 con 0x7f73dc0cdc90 2023-11-28T11:08:52.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.583+0000 7f73dbfff700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73dc0cdc90 0x7f73dc15d4f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f73d400cca0 tx=0x7f73d400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.583+0000 7f73c3fff700 1 -- 172.21.15.156:0/2819467754 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d400cea0 con 0x7f73dc0cdc90 2023-11-28T11:08:52.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.583+0000 7f73c3fff700 1 -- 172.21.15.156:0/2819467754 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d400ade0 con 0x7f73dc0cdc90 2023-11-28T11:08:52.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.583+0000 7f73e1ed1700 1 -- 172.21.15.156:0/2819467754 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73dc140150 con 0x7f73dc0cdc90 2023-11-28T11:08:52.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.583+0000 7f73c3fff700 1 -- 172.21.15.156:0/2819467754 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d4007310 con 0x7f73dc0cdc90 2023-11-28T11:08:52.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.583+0000 7f73e1ed1700 1 -- 172.21.15.156:0/2819467754 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73dc0b35f0 con 0x7f73dc0cdc90 2023-11-28T11:08:52.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.584+0000 7f73e1ed1700 1 -- 172.21.15.156:0/2819467754 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73dc0ce100 con 0x7f73dc0cdc90 2023-11-28T11:08:52.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.584+0000 7f73c3fff700 1 -- 172.21.15.156:0/2819467754 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f73d401a030 con 0x7f73dc0cdc90 2023-11-28T11:08:52.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.585+0000 7f73c3fff700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f73c80682f0 0x7f73c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.585+0000 7f73c3fff700 1 -- 172.21.15.156:0/2819467754 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f73d4085a70 con 0x7f73dc0cdc90 2023-11-28T11:08:52.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.585+0000 7f73db7fe700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f73c80682f0 0x7f73c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.585+0000 7f73db7fe700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f73c80682f0 0x7f73c806a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009b40 tx=0x7f73d0008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.589+0000 7f73c3fff700 1 -- 172.21.15.156:0/2819467754 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f73d40522d0 con 0x7f73dc0cdc90 2023-11-28T11:08:52.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.769+0000 7f73e1ed1700 1 -- 172.21.15.156:0/2819467754 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73dc0d08c0 con 0x7f73dc0cdc90 2023-11-28T11:08:52.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.770+0000 7f73c3fff700 1 -- 172.21.15.156:0/2819467754 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73d4018020 con 0x7f73dc0cdc90 2023-11-28T11:08:52.771 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:08:52.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.772+0000 7f73e1ed1700 1 -- 172.21.15.156:0/2819467754 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f73c80682f0 msgr2=0x7f73c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.772+0000 7f73e1ed1700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f73c80682f0 0x7f73c806a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009b40 tx=0x7f73d0008070 comp rx=0 tx=0).stop 2023-11-28T11:08:52.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.772+0000 7f73e1ed1700 1 -- 172.21.15.156:0/2819467754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73dc0cdc90 msgr2=0x7f73dc15d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.772+0000 7f73e1ed1700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73dc0cdc90 0x7f73dc15d4f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f73d400cca0 tx=0x7f73d400a340 comp rx=0 tx=0).stop 2023-11-28T11:08:52.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.772+0000 7f73e1ed1700 1 -- 172.21.15.156:0/2819467754 shutdown_connections 2023-11-28T11:08:52.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.772+0000 7f73e1ed1700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73dc0cdc90 0x7f73dc15d4f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.772+0000 7f73e1ed1700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73dc0b2e70 0x7f73dc15ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.772+0000 7f73e1ed1700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f73c80682f0 0x7f73c806a7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.772+0000 7f73e1ed1700 1 --2- 172.21.15.156:0/2819467754 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73dc0cd8c0 0x7f73dc140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.772+0000 7f73e1ed1700 1 -- 172.21.15.156:0/2819467754 >> 172.21.15.156:0/2819467754 conn(0x7f73dc06aed0 msgr2=0x7f73dc13ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.772+0000 7f73e1ed1700 1 -- 172.21.15.156:0/2819467754 shutdown_connections 2023-11-28T11:08:52.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.772+0000 7f73e1ed1700 1 -- 172.21.15.156:0/2819467754 wait complete. 2023-11-28T11:08:52.785 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:52.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.908+0000 7f9b5e9a2700 1 Processor -- start 2023-11-28T11:08:52.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.908+0000 7f9b5e9a2700 1 -- start start 2023-11-28T11:08:52.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.909+0000 7f9b5e9a2700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9b5806ba90 0x7f9b580af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.909+0000 7f9b5e9a2700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b580cc5f0 con 0x7f9b580cbdf0 2023-11-28T11:08:52.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.909+0000 7f9b5e9a2700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5815d470 con 0x7f9b5815cef0 2023-11-28T11:08:52.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.909+0000 7f9b5e9a2700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5815d750 con 0x7f9b5806ba90 2023-11-28T11:08:52.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.909+0000 7f9b577fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9b5806ba90 0x7f9b580af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.909+0000 7f9b5cf3f700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f9b580cbdf0 0x7f9b580ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47796/0 (socket says 172.21.15.156:47796) 2023-11-28T11:08:52.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.909+0000 7f9b5cf3f700 1 -- 172.21.15.156:0/358189668 learned_addr learned my addr 172.21.15.156:0/358189668 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:52.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.910+0000 7f9b577fe700 1 -- 172.21.15.156:0/358189668 >> v1:172.21.15.186:6789/0 conn(0x7f9b5815cef0 legacy=0x7f9b5806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.910+0000 7f9b577fe700 1 -- 172.21.15.156:0/358189668 >> v1:172.21.15.156:6789/0 conn(0x7f9b580cbdf0 legacy=0x7f9b580ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.910+0000 7f9b577fe700 1 -- 172.21.15.156:0/358189668 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5815da30 con 0x7f9b5806ba90 2023-11-28T11:08:52.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.910+0000 7f9b56ffd700 1 -- 172.21.15.156:0/358189668 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558938334 0 0) 0x7f9b580cc5f0 con 0x7f9b580cbdf0 2023-11-28T11:08:52.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.910+0000 7f9b577fe700 1 --2- 172.21.15.156:0/358189668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b5806ba90 0x7f9b580af400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009840 tx=0x7f9b48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bf4b2ff8c79e8f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.910+0000 7f9b56ffd700 1 -- 172.21.15.156:0/358189668 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48011830 con 0x7f9b5806ba90 2023-11-28T11:08:52.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.910+0000 7f9b56ffd700 1 -- 172.21.15.156:0/358189668 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b48011990 con 0x7f9b5806ba90 2023-11-28T11:08:52.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.910+0000 7f9b56ffd700 1 -- 172.21.15.156:0/358189668 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48011d20 con 0x7f9b5806ba90 2023-11-28T11:08:52.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.910+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/358189668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b5806ba90 msgr2=0x7f9b580af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.910+0000 7f9b5e9a2700 1 --2- 172.21.15.156:0/358189668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b5806ba90 0x7f9b580af400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009840 tx=0x7f9b48009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:52.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.911+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/358189668 shutdown_connections 2023-11-28T11:08:52.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.911+0000 7f9b5e9a2700 1 --2- 172.21.15.156:0/358189668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b5806ba90 0x7f9b580af400 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.911+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/358189668 >> 172.21.15.156:0/358189668 conn(0x7f9b580d0000 msgr2=0x7f9b580d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.911+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/358189668 shutdown_connections 2023-11-28T11:08:52.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.911+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/358189668 wait complete. 2023-11-28T11:08:52.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.912+0000 7f9b5e9a2700 1 Processor -- start 2023-11-28T11:08:52.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.912+0000 7f9b5e9a2700 1 -- start start 2023-11-28T11:08:52.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b5e9a2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b5806ba90 0x7f9b58155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b5e9a2700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b580cbdf0 0x7f9b580c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b5e9a2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b5815cef0 0x7f9b580b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b5e9a2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580c7da0 con 0x7f9b5806ba90 2023-11-28T11:08:52.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b5e9a2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580c8080 con 0x7f9b5815cef0 2023-11-28T11:08:52.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b5e9a2700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580c8360 con 0x7f9b580cbdf0 2023-11-28T11:08:52.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b57fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b5806ba90 0x7f9b58155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b57fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b5806ba90 0x7f9b58155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32920/0 (socket says 172.21.15.156:32920) 2023-11-28T11:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b57fff700 1 -- 172.21.15.156:0/4118009220 learned_addr learned my addr 172.21.15.156:0/4118009220 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b5cf3f700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b5815cef0 0x7f9b580b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b577fe700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b580cbdf0 0x7f9b580c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b57fff700 1 -- 172.21.15.156:0/4118009220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b580cbdf0 msgr2=0x7f9b580c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b57fff700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b580cbdf0 0x7f9b580c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b57fff700 1 -- 172.21.15.156:0/4118009220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b5815cef0 msgr2=0x7f9b580b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b57fff700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b5815cef0 0x7f9b580b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b57fff700 1 -- 172.21.15.156:0/4118009220 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b480094a0 con 0x7f9b5806ba90 2023-11-28T11:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b577fe700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b580cbdf0 0x7f9b580c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b57fff700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b5806ba90 0x7f9b58155c10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000c450 tx=0x7f9b4000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b54ff9700 1 -- 172.21.15.156:0/4118009220 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b400052d0 con 0x7f9b5806ba90 2023-11-28T11:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b54ff9700 1 -- 172.21.15.156:0/4118009220 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b40005430 con 0x7f9b5806ba90 2023-11-28T11:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/4118009220 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5815dff0 con 0x7f9b5806ba90 2023-11-28T11:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b54ff9700 1 -- 172.21.15.156:0/4118009220 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b400038c0 con 0x7f9b5806ba90 2023-11-28T11:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.913+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/4118009220 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5815dd10 con 0x7f9b5806ba90 2023-11-28T11:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.914+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/4118009220 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b580cad60 con 0x7f9b5806ba90 2023-11-28T11:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.915+0000 7f9b54ff9700 1 -- 172.21.15.156:0/4118009220 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9b40019030 con 0x7f9b5806ba90 2023-11-28T11:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.915+0000 7f9b54ff9700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9b380682f0 0x7f9b3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.915+0000 7f9b54ff9700 1 -- 172.21.15.156:0/4118009220 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9b40084f10 con 0x7f9b5806ba90 2023-11-28T11:08:52.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.915+0000 7f9b577fe700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9b380682f0 0x7f9b3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.916+0000 7f9b577fe700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9b380682f0 0x7f9b3806a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b480090b0 tx=0x7f9b48008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:52.919+0000 7f9b54ff9700 1 -- 172.21.15.156:0/4118009220 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9b40051770 con 0x7f9b5806ba90 2023-11-28T11:08:53.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.102+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/4118009220 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9b580c7ee0 con 0x7f9b5806ba90 2023-11-28T11:08:53.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.102+0000 7f9b54ff9700 1 -- 172.21.15.156:0/4118009220 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9b580c7ee0 con 0x7f9b5806ba90 2023-11-28T11:08:53.104 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:08:53.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.104+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/4118009220 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9b380682f0 msgr2=0x7f9b3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.104+0000 7f9b5e9a2700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9b380682f0 0x7f9b3806a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b480090b0 tx=0x7f9b48008030 comp rx=0 tx=0).stop 2023-11-28T11:08:53.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.104+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/4118009220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b5806ba90 msgr2=0x7f9b58155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.104+0000 7f9b5e9a2700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b5806ba90 0x7f9b58155c10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000c450 tx=0x7f9b4000c760 comp rx=0 tx=0).stop 2023-11-28T11:08:53.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.104+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/4118009220 shutdown_connections 2023-11-28T11:08:53.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.104+0000 7f9b5e9a2700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b5815cef0 0x7f9b580b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.104+0000 7f9b5e9a2700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b5806ba90 0x7f9b58155c10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.104+0000 7f9b5e9a2700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9b380682f0 0x7f9b3806a7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.104+0000 7f9b5e9a2700 1 --2- 172.21.15.156:0/4118009220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b580cbdf0 0x7f9b580c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.104+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/4118009220 >> 172.21.15.156:0/4118009220 conn(0x7f9b580d0000 msgr2=0x7f9b580b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:53.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.105+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/4118009220 shutdown_connections 2023-11-28T11:08:53.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.105+0000 7f9b5e9a2700 1 -- 172.21.15.156:0/4118009220 wait complete. 2023-11-28T11:08:53.118 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:08:53.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.238+0000 7fb348259700 1 Processor -- start 2023-11-28T11:08:53.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.238+0000 7fb348259700 1 -- start start 2023-11-28T11:08:53.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb348259700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400d1330 0x7fb3400b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb348259700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3400b0230 con 0x7fb3400d01b0 2023-11-28T11:08:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb348259700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3400b4570 con 0x7fb3400b20b0 2023-11-28T11:08:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb348259700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34015cec0 con 0x7fb3400d1330 2023-11-28T11:08:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb345ff5700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb3400d01b0 0x7fb3400d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47806/0 (socket says 172.21.15.156:47806) 2023-11-28T11:08:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb345ff5700 1 -- 172.21.15.156:0/852630506 learned_addr learned my addr 172.21.15.156:0/852630506 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575968226 0 0) 0x7fb3400b0230 con 0x7fb3400d01b0 2023-11-28T11:08:53.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb3457f4700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400d1330 0x7fb3400b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb32c0035b0 con 0x7fb3400d01b0 2023-11-28T11:08:53.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1954484701 0 0) 0x7fb32c0035b0 con 0x7fb3400d01b0 2023-11-28T11:08:53.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400d1330 msgr2=0x7fb3400b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:53.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400d1330 0x7fb3400b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 >> v1:172.21.15.186:6789/0 conn(0x7fb3400b20b0 legacy=0x7fb3400b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400b4850 con 0x7fb3400d01b0 2023-11-28T11:08:53.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fb33c001670 con 0x7fb3400d01b0 2023-11-28T11:08:53.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 >> v1:172.21.15.156:6789/0 conn(0x7fb3400d01b0 legacy=0x7fb3400d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400d1330 0x7fb32c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb32c003f70 0x7fb32c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb32c004d90 0x7fb32c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34015cec0 con 0x7fb32c004d90 2023-11-28T11:08:53.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c00ba40 con 0x7fb32c003f70 2023-11-28T11:08:53.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.239+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c00bc00 con 0x7fb3400d1330 2023-11-28T11:08:53.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb345ff5700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400d1330 0x7fb32c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb3457f4700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb32c003f70 0x7fb32c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb3467f6700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb32c004d90 0x7fb32c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb3467f6700 1 -- 172.21.15.156:0/852630506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400d1330 msgr2=0x7fb32c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:53.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb3467f6700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400d1330 0x7fb32c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb3467f6700 1 -- 172.21.15.156:0/852630506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb32c003f70 msgr2=0x7fb32c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:53.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb3467f6700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb32c003f70 0x7fb32c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb3467f6700 1 -- 172.21.15.156:0/852630506 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c0035b0 con 0x7fb32c004d90 2023-11-28T11:08:53.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb345ff5700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400d1330 0x7fb32c003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:53.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb3467f6700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb32c004d90 0x7fb32c005200 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb33800c260 tx=0x7fb32c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4926b9887623cc97 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338001e70 con 0x7fb32c004d90 2023-11-28T11:08:53.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33800ce20 con 0x7fb32c004d90 2023-11-28T11:08:53.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.240+0000 7fb344ff3700 1 -- 172.21.15.156:0/852630506 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338012660 con 0x7fb32c004d90 2023-11-28T11:08:53.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.241+0000 7fb348259700 1 -- 172.21.15.156:0/852630506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb32c004d90 msgr2=0x7fb32c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.241+0000 7fb348259700 1 --2- 172.21.15.156:0/852630506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb32c004d90 0x7fb32c005200 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb33800c260 tx=0x7fb32c005880 comp rx=0 tx=0).stop 2023-11-28T11:08:53.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.241+0000 7fb3467f6700 1 -- 172.21.15.156:0/852630506 reap_dead start 2023-11-28T11:08:53.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.241+0000 7fb348259700 1 -- 172.21.15.156:0/852630506 shutdown_connections 2023-11-28T11:08:53.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.241+0000 7fb348259700 1 -- 172.21.15.156:0/852630506 >> 172.21.15.156:0/852630506 conn(0x7fb34009c3f0 msgr2=0x7fb3400d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:53.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.241+0000 7fb348259700 1 -- 172.21.15.156:0/852630506 shutdown_connections 2023-11-28T11:08:53.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.241+0000 7fb348259700 1 -- 172.21.15.156:0/852630506 wait complete. 2023-11-28T11:08:53.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.242+0000 7fb348259700 1 Processor -- start 2023-11-28T11:08:53.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.242+0000 7fb348259700 1 -- start start 2023-11-28T11:08:53.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb348259700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3401f5450 0x7fb3401f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb348259700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400cde50 0x7fb3400ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb348259700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3401f5aa0 0x7fb3401f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb348259700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400ce800 con 0x7fb3401f5450 2023-11-28T11:08:53.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb348259700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400b7e60 con 0x7fb3401f5aa0 2023-11-28T11:08:53.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb348259700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3401f3470 con 0x7fb3400cde50 2023-11-28T11:08:53.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb345ff5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3401f5450 0x7fb3401f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb345ff5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3401f5450 0x7fb3401f2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32940/0 (socket says 172.21.15.156:32940) 2023-11-28T11:08:53.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb345ff5700 1 -- 172.21.15.156:0/4052919272 learned_addr learned my addr 172.21.15.156:0/4052919272 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:53.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb3467f6700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3401f5aa0 0x7fb3401f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb3457f4700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400cde50 0x7fb3400ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb345ff5700 1 -- 172.21.15.156:0/4052919272 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400cde50 msgr2=0x7fb3400ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb345ff5700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400cde50 0x7fb3400ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb345ff5700 1 -- 172.21.15.156:0/4052919272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3401f5aa0 msgr2=0x7fb3401f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb345ff5700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3401f5aa0 0x7fb3401f5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb345ff5700 1 -- 172.21.15.156:0/4052919272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400afec0 con 0x7fb3401f5450 2023-11-28T11:08:53.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb3457f4700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400cde50 0x7fb3400ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:53.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb345ff5700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3401f5450 0x7fb3401f2f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb33c007550 tx=0x7fb33c007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb3327fc700 1 -- 172.21.15.156:0/4052919272 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c00f870 con 0x7fb3401f5450 2023-11-28T11:08:53.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb3327fc700 1 -- 172.21.15.156:0/4052919272 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c00f9d0 con 0x7fb3401f5450 2023-11-28T11:08:53.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb3327fc700 1 -- 172.21.15.156:0/4052919272 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c00fd40 con 0x7fb3401f5450 2023-11-28T11:08:53.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.243+0000 7fb348259700 1 -- 172.21.15.156:0/4052919272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3400afbe0 con 0x7fb3401f5450 2023-11-28T11:08:53.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.244+0000 7fb348259700 1 -- 172.21.15.156:0/4052919272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3400b2600 con 0x7fb3401f5450 2023-11-28T11:08:53.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.245+0000 7fb348259700 1 -- 172.21.15.156:0/4052919272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb340068360 con 0x7fb3401f5450 2023-11-28T11:08:53.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.245+0000 7fb3327fc700 1 -- 172.21.15.156:0/4052919272 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb33c01d460 con 0x7fb3401f5450 2023-11-28T11:08:53.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.245+0000 7fb3327fc700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb3240682f0 0x7fb32406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.245+0000 7fb3457f4700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb3240682f0 0x7fb32406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.245+0000 7fb3327fc700 1 -- 172.21.15.156:0/4052919272 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb33c086010 con 0x7fb3401f5450 2023-11-28T11:08:53.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.246+0000 7fb3457f4700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb3240682f0 0x7fb32406a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb3401f6b60 tx=0x7fb334008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.249+0000 7fb3327fc700 1 -- 172.21.15.156:0/4052919272 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb33c052810 con 0x7fb3401f5450 2023-11-28T11:08:53.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.445+0000 7fb348259700 1 -- 172.21.15.156:0/4052919272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb340068690 con 0x7fb3401f5450 2023-11-28T11:08:53.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.445+0000 7fb3327fc700 1 -- 172.21.15.156:0/4052919272 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb33c00fea0 con 0x7fb3401f5450 2023-11-28T11:08:53.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.447+0000 7fb348259700 1 -- 172.21.15.156:0/4052919272 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb3240682f0 msgr2=0x7fb32406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.447+0000 7fb348259700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb3240682f0 0x7fb32406a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb3401f6b60 tx=0x7fb334008040 comp rx=0 tx=0).stop 2023-11-28T11:08:53.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.447+0000 7fb348259700 1 -- 172.21.15.156:0/4052919272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3401f5450 msgr2=0x7fb3401f2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.447+0000 7fb348259700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3401f5450 0x7fb3401f2f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb33c007550 tx=0x7fb33c007860 comp rx=0 tx=0).stop 2023-11-28T11:08:53.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.447+0000 7fb348259700 1 -- 172.21.15.156:0/4052919272 shutdown_connections 2023-11-28T11:08:53.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.447+0000 7fb348259700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3401f5aa0 0x7fb3401f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.447+0000 7fb348259700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3401f5450 0x7fb3401f2f30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.447+0000 7fb348259700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb3240682f0 0x7fb32406a7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.447+0000 7fb348259700 1 --2- 172.21.15.156:0/4052919272 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3400cde50 0x7fb3400ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.447+0000 7fb348259700 1 -- 172.21.15.156:0/4052919272 >> 172.21.15.156:0/4052919272 conn(0x7fb34009c3f0 msgr2=0x7fb3400b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:53.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.447+0000 7fb348259700 1 -- 172.21.15.156:0/4052919272 shutdown_connections 2023-11-28T11:08:53.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.447+0000 7fb348259700 1 -- 172.21.15.156:0/4052919272 wait complete. 2023-11-28T11:08:53.450 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T11:08:53.462 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:08:53.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:53.463 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-28T11:08:53.508 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:08:53.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:53.509 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:08:53.602 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:08:53.602 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:53.624 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:08:53.624 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:53.684 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:08:53.685 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:53.745 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:53.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.867+0000 7f046e60e700 1 Processor -- start 2023-11-28T11:08:53.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.867+0000 7f046e60e700 1 -- start start 2023-11-28T11:08:53.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.867+0000 7f046e60e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0468071040 0x7f0468071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.867+0000 7f046e60e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0468160540 0x7f04681609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.867+0000 7f046e60e700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0468071950 con 0x7f046806dee0 2023-11-28T11:08:53.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.867+0000 7f046e60e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046815d060 con 0x7f0468160540 2023-11-28T11:08:53.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.867+0000 7f046e60e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046815d340 con 0x7f0468071040 2023-11-28T11:08:53.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.868+0000 7f046cbab700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0468160540 0x7f04681609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.868+0000 7f046cbab700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0468160540 0x7f04681609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32952/0 (socket says 172.21.15.156:32952) 2023-11-28T11:08:53.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.868+0000 7f046cbab700 1 -- 172.21.15.156:0/1284816718 learned_addr learned my addr 172.21.15.156:0/1284816718 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:53.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.868+0000 7f0467fff700 1 --2- 172.21.15.156:0/1284816718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0468071040 0x7f0468071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.868+0000 7f046cbab700 1 -- 172.21.15.156:0/1284816718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0468071040 msgr2=0x7f0468071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:53.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.868+0000 7f046cbab700 1 --2- 172.21.15.156:0/1284816718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0468071040 0x7f0468071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.868+0000 7f046cbab700 1 -- 172.21.15.156:0/1284816718 >> v1:172.21.15.186:6789/0 conn(0x7f046806dee0 legacy=0x7f046806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.868+0000 7f046cbab700 1 -- 172.21.15.156:0/1284816718 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046815d620 con 0x7f0468160540 2023-11-28T11:08:53.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.868+0000 7f046cbab700 1 --2- 172.21.15.156:0/1284816718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0468160540 0x7f04681609c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0460009e50 tx=0x7f046000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac0c50ae091070e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.868+0000 7f0466ffd700 1 -- 172.21.15.156:0/1284816718 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046000e070 con 0x7f0468160540 2023-11-28T11:08:53.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.868+0000 7f0466ffd700 1 -- 172.21.15.156:0/1284816718 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046000b910 con 0x7f0468160540 2023-11-28T11:08:53.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.868+0000 7f0466ffd700 1 -- 172.21.15.156:0/1284816718 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046000bcc0 con 0x7f0468160540 2023-11-28T11:08:53.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.869+0000 7f046e60e700 1 -- 172.21.15.156:0/1284816718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0468160540 msgr2=0x7f04681609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.869+0000 7f046e60e700 1 --2- 172.21.15.156:0/1284816718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0468160540 0x7f04681609c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0460009e50 tx=0x7f046000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:53.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.869+0000 7f046e60e700 1 -- 172.21.15.156:0/1284816718 shutdown_connections 2023-11-28T11:08:53.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.869+0000 7f046e60e700 1 --2- 172.21.15.156:0/1284816718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0468160540 0x7f04681609c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.869+0000 7f046e60e700 1 --2- 172.21.15.156:0/1284816718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0468071040 0x7f0468071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.869+0000 7f046e60e700 1 -- 172.21.15.156:0/1284816718 >> 172.21.15.156:0/1284816718 conn(0x7f04680d0000 msgr2=0x7f04680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:53.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.869+0000 7f046e60e700 1 -- 172.21.15.156:0/1284816718 shutdown_connections 2023-11-28T11:08:53.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.869+0000 7f046e60e700 1 -- 172.21.15.156:0/1284816718 wait complete. 2023-11-28T11:08:53.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.870+0000 7f046e60e700 1 Processor -- start 2023-11-28T11:08:53.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.870+0000 7f046e60e700 1 -- start start 2023-11-28T11:08:53.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f046e60e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f046806dee0 0x7f046814d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f046e60e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0468071040 0x7f04680c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f046e60e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0468160540 0x7f0468159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f046e60e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0468156eb0 con 0x7f0468071040 2023-11-28T11:08:53.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f046e60e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04680b4ee0 con 0x7f046806dee0 2023-11-28T11:08:53.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f046e60e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046814d880 con 0x7f0468160540 2023-11-28T11:08:53.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f04677fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0468071040 0x7f04680c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f04677fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0468071040 0x7f04680c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32956/0 (socket says 172.21.15.156:32956) 2023-11-28T11:08:53.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f04677fe700 1 -- 172.21.15.156:0/1357903776 learned_addr learned my addr 172.21.15.156:0/1357903776 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:53.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f0467fff700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f046806dee0 0x7f046814d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f046cbab700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0468160540 0x7f0468159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f04677fe700 1 -- 172.21.15.156:0/1357903776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0468160540 msgr2=0x7f0468159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f04677fe700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0468160540 0x7f0468159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f04677fe700 1 -- 172.21.15.156:0/1357903776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f046806dee0 msgr2=0x7f046814d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f04677fe700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f046806dee0 0x7f046814d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.871+0000 7f04677fe700 1 -- 172.21.15.156:0/1357903776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0460009b30 con 0x7f0468071040 2023-11-28T11:08:53.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.872+0000 7f04677fe700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0468071040 0x7f04680c6320 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f045c00c450 tx=0x7f045c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.872+0000 7f0464ff9700 1 -- 172.21.15.156:0/1357903776 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c0052d0 con 0x7f0468071040 2023-11-28T11:08:53.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.872+0000 7f0464ff9700 1 -- 172.21.15.156:0/1357903776 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045c005430 con 0x7f0468071040 2023-11-28T11:08:53.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.872+0000 7f046e60e700 1 -- 172.21.15.156:0/1357903776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046815ccd0 con 0x7f0468071040 2023-11-28T11:08:53.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.872+0000 7f0464ff9700 1 -- 172.21.15.156:0/1357903776 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c0038c0 con 0x7f0468071040 2023-11-28T11:08:53.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.872+0000 7f046e60e700 1 -- 172.21.15.156:0/1357903776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046815d900 con 0x7f0468071040 2023-11-28T11:08:53.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.873+0000 7f046e60e700 1 -- 172.21.15.156:0/1357903776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046806a270 con 0x7f0468071040 2023-11-28T11:08:53.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.873+0000 7f0464ff9700 1 -- 172.21.15.156:0/1357903776 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f045c019030 con 0x7f0468071040 2023-11-28T11:08:53.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.873+0000 7f0464ff9700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04540682f0 0x7f045406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:53.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.874+0000 7f0464ff9700 1 -- 172.21.15.156:0/1357903776 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f045c084fb0 con 0x7f0468071040 2023-11-28T11:08:53.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.874+0000 7f0467fff700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04540682f0 0x7f045406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:53.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.874+0000 7f0467fff700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04540682f0 0x7f045406a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0458009b40 tx=0x7f0458009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:53.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:53.878+0000 7f0464ff9700 1 -- 172.21.15.156:0/1357903776 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f045c051690 con 0x7f0468071040 2023-11-28T11:08:54.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.061+0000 7f046e60e700 1 -- 172.21.15.156:0/1357903776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04680720a0 con 0x7f0468071040 2023-11-28T11:08:54.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.063+0000 7f0464ff9700 1 -- 172.21.15.156:0/1357903776 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f045c017020 con 0x7f0468071040 2023-11-28T11:08:54.064 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:54.064 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:18.354009+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:54.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.065+0000 7f046e60e700 1 -- 172.21.15.156:0/1357903776 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04540682f0 msgr2=0x7f045406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.065+0000 7f046e60e700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04540682f0 0x7f045406a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0458009b40 tx=0x7f0458009210 comp rx=0 tx=0).stop 2023-11-28T11:08:54.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.065+0000 7f046e60e700 1 -- 172.21.15.156:0/1357903776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0468071040 msgr2=0x7f04680c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.065+0000 7f046e60e700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0468071040 0x7f04680c6320 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f045c00c450 tx=0x7f045c00c760 comp rx=0 tx=0).stop 2023-11-28T11:08:54.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.065+0000 7f046e60e700 1 -- 172.21.15.156:0/1357903776 shutdown_connections 2023-11-28T11:08:54.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.065+0000 7f046e60e700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04540682f0 0x7f045406a7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.065+0000 7f046e60e700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0468160540 0x7f0468159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.065+0000 7f046e60e700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f046806dee0 0x7f046814d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.065+0000 7f046e60e700 1 --2- 172.21.15.156:0/1357903776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0468071040 0x7f04680c6320 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.065+0000 7f046e60e700 1 -- 172.21.15.156:0/1357903776 >> 172.21.15.156:0/1357903776 conn(0x7f04680d0000 msgr2=0x7f046815f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.065+0000 7f046e60e700 1 -- 172.21.15.156:0/1357903776 shutdown_connections 2023-11-28T11:08:54.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.065+0000 7f046e60e700 1 -- 172.21.15.156:0/1357903776 wait complete. 2023-11-28T11:08:54.079 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:15.021591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:16.332241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:54.079 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:54.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.201+0000 7fbd77c3f700 1 Processor -- start 2023-11-28T11:08:54.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.201+0000 7fbd77c3f700 1 -- start start 2023-11-28T11:08:54.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd77c3f700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbd70071040 0x7fbd70071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd77c3f700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd70071950 con 0x7fbd70160540 2023-11-28T11:08:54.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd77c3f700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7015d060 con 0x7fbd7006dee0 2023-11-28T11:08:54.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd77c3f700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd7015d340 con 0x7fbd70071040 2023-11-28T11:08:54.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd761dc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fbd70160540 0x7fbd70160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47844/0 (socket says 172.21.15.156:47844) 2023-11-28T11:08:54.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd761dc700 1 -- 172.21.15.156:0/1876279006 learned_addr learned my addr 172.21.15.156:0/1876279006 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:54.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd759db700 1 --2- 172.21.15.156:0/1876279006 >> v2:172.21.15.186:3300/0 conn(0x7fbd70071040 0x7fbd70071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366300197 0 0) 0x7fbd70071950 con 0x7fbd70160540 2023-11-28T11:08:54.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd5c0035b0 con 0x7fbd70160540 2023-11-28T11:08:54.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3271947501 0 0) 0x7fbd5c0035b0 con 0x7fbd70160540 2023-11-28T11:08:54.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 >> v2:172.21.15.186:3300/0 conn(0x7fbd70071040 msgr2=0x7fbd70071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 --2- 172.21.15.156:0/1876279006 >> v2:172.21.15.186:3300/0 conn(0x7fbd70071040 0x7fbd70071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 >> v1:172.21.15.186:6789/0 conn(0x7fbd7006dee0 legacy=0x7fbd7006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7015d620 con 0x7fbd70160540 2023-11-28T11:08:54.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fbd680019f0 con 0x7fbd70160540 2023-11-28T11:08:54.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 >> v1:172.21.15.156:6789/0 conn(0x7fbd70160540 legacy=0x7fbd70160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 --2- 172.21.15.156:0/1876279006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd5c003a00 0x7fbd5c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 --2- 172.21.15.156:0/1876279006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c004850 0x7fbd5c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 --2- 172.21.15.156:0/1876279006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c005320 0x7fbd5c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7015d340 con 0x7fbd5c005320 2023-11-28T11:08:54.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c00c250 con 0x7fbd5c004850 2023-11-28T11:08:54.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.202+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c00c410 con 0x7fbd5c003a00 2023-11-28T11:08:54.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd761dc700 1 --2- 172.21.15.156:0/1876279006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c005320 0x7fbd5c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd759db700 1 --2- 172.21.15.156:0/1876279006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd5c003a00 0x7fbd5c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd751da700 1 --2- 172.21.15.156:0/1876279006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c004850 0x7fbd5c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd761dc700 1 -- 172.21.15.156:0/1876279006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd5c003a00 msgr2=0x7fbd5c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd761dc700 1 --2- 172.21.15.156:0/1876279006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd5c003a00 0x7fbd5c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd761dc700 1 -- 172.21.15.156:0/1876279006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c004850 msgr2=0x7fbd5c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd761dc700 1 --2- 172.21.15.156:0/1876279006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c004850 0x7fbd5c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd761dc700 1 -- 172.21.15.156:0/1876279006 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c0035b0 con 0x7fbd5c005320 2023-11-28T11:08:54.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd761dc700 1 -- 172.21.15.156:0/1876279006 reap_dead start 2023-11-28T11:08:54.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd761dc700 1 --2- 172.21.15.156:0/1876279006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c005320 0x7fbd5c00ba10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800a830 tx=0x7fbd5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d6cff2d2848dfe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6800a2f0 con 0x7fbd5c005320 2023-11-28T11:08:54.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd68012600 con 0x7fbd5c005320 2023-11-28T11:08:54.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.203+0000 7fbd749d9700 1 -- 172.21.15.156:0/1876279006 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd680129b0 con 0x7fbd5c005320 2023-11-28T11:08:54.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.204+0000 7fbd77c3f700 1 -- 172.21.15.156:0/1876279006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c005320 msgr2=0x7fbd5c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.204+0000 7fbd77c3f700 1 --2- 172.21.15.156:0/1876279006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c005320 0x7fbd5c00ba10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800a830 tx=0x7fbd5c005880 comp rx=0 tx=0).stop 2023-11-28T11:08:54.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.204+0000 7fbd77c3f700 1 -- 172.21.15.156:0/1876279006 shutdown_connections 2023-11-28T11:08:54.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.204+0000 7fbd77c3f700 1 --2- 172.21.15.156:0/1876279006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c005320 0x7fbd5c00ba10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.204+0000 7fbd77c3f700 1 -- 172.21.15.156:0/1876279006 >> 172.21.15.156:0/1876279006 conn(0x7fbd700d0000 msgr2=0x7fbd700d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.204+0000 7fbd77c3f700 1 -- 172.21.15.156:0/1876279006 shutdown_connections 2023-11-28T11:08:54.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.204+0000 7fbd77c3f700 1 -- 172.21.15.156:0/1876279006 wait complete. 2023-11-28T11:08:54.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.205+0000 7fbd77c3f700 1 Processor -- start 2023-11-28T11:08:54.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.205+0000 7fbd77c3f700 1 -- start start 2023-11-28T11:08:54.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd77c3f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd5c005320 0x7fbd701485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd77c3f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd70154dd0 0x7fbd701594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd77c3f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd70159a10 0x7fbd700ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd77c3f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700adca0 con 0x7fbd70159a10 2023-11-28T11:08:54.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd77c3f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700acdf0 con 0x7fbd70154dd0 2023-11-28T11:08:54.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd77c3f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700ad090 con 0x7fbd5c005320 2023-11-28T11:08:54.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd761dc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd70159a10 0x7fbd700ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd761dc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd70159a10 0x7fbd700ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32978/0 (socket says 172.21.15.156:32978) 2023-11-28T11:08:54.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd761dc700 1 -- 172.21.15.156:0/496233518 learned_addr learned my addr 172.21.15.156:0/496233518 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:54.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd759db700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd5c005320 0x7fbd701485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd751da700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd70154dd0 0x7fbd701594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd761dc700 1 -- 172.21.15.156:0/496233518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd5c005320 msgr2=0x7fbd701485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd761dc700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd5c005320 0x7fbd701485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd761dc700 1 -- 172.21.15.156:0/496233518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd70154dd0 msgr2=0x7fbd701594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd761dc700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd70154dd0 0x7fbd701594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd761dc700 1 -- 172.21.15.156:0/496233518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7015ccd0 con 0x7fbd70159a10 2023-11-28T11:08:54.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd761dc700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd70159a10 0x7fbd700ae130 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbd700d0c90 tx=0x7fbd68001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.206+0000 7fbd667fc700 1 -- 172.21.15.156:0/496233518 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68001450 con 0x7fbd70159a10 2023-11-28T11:08:54.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.207+0000 7fbd667fc700 1 -- 172.21.15.156:0/496233518 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd680017b0 con 0x7fbd70159a10 2023-11-28T11:08:54.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.207+0000 7fbd667fc700 1 -- 172.21.15.156:0/496233518 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6801e430 con 0x7fbd70159a10 2023-11-28T11:08:54.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.207+0000 7fbd77c3f700 1 -- 172.21.15.156:0/496233518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7015d900 con 0x7fbd70159a10 2023-11-28T11:08:54.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.207+0000 7fbd77c3f700 1 -- 172.21.15.156:0/496233518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7006a270 con 0x7fbd70159a10 2023-11-28T11:08:54.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.207+0000 7fbd77c3f700 1 -- 172.21.15.156:0/496233518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd700adde0 con 0x7fbd70159a10 2023-11-28T11:08:54.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.208+0000 7fbd667fc700 1 -- 172.21.15.156:0/496233518 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbd68010070 con 0x7fbd70159a10 2023-11-28T11:08:54.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.208+0000 7fbd667fc700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbd540682f0 0x7fbd5406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.208+0000 7fbd667fc700 1 -- 172.21.15.156:0/496233518 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbd68087ac0 con 0x7fbd70159a10 2023-11-28T11:08:54.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.208+0000 7fbd759db700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbd540682f0 0x7fbd5406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.209+0000 7fbd759db700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbd540682f0 0x7fbd5406a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009be0 tx=0x7fbd6c009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.212+0000 7fbd667fc700 1 -- 172.21.15.156:0/496233518 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbd68054320 con 0x7fbd70159a10 2023-11-28T11:08:54.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.395+0000 7fbd77c3f700 1 -- 172.21.15.156:0/496233518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd700c6b20 con 0x7fbd70159a10 2023-11-28T11:08:54.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.395+0000 7fbd667fc700 1 -- 172.21.15.156:0/496233518 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd68057940 con 0x7fbd70159a10 2023-11-28T11:08:54.397 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:08:54.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.398+0000 7fbd77c3f700 1 -- 172.21.15.156:0/496233518 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbd540682f0 msgr2=0x7fbd5406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.398+0000 7fbd77c3f700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbd540682f0 0x7fbd5406a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009be0 tx=0x7fbd6c009210 comp rx=0 tx=0).stop 2023-11-28T11:08:54.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.398+0000 7fbd77c3f700 1 -- 172.21.15.156:0/496233518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd70159a10 msgr2=0x7fbd700ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.398+0000 7fbd77c3f700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd70159a10 0x7fbd700ae130 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbd700d0c90 tx=0x7fbd68001d80 comp rx=0 tx=0).stop 2023-11-28T11:08:54.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.398+0000 7fbd77c3f700 1 -- 172.21.15.156:0/496233518 shutdown_connections 2023-11-28T11:08:54.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.398+0000 7fbd77c3f700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd70159a10 0x7fbd700ae130 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.398+0000 7fbd77c3f700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbd540682f0 0x7fbd5406a7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.398+0000 7fbd77c3f700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd5c005320 0x7fbd701485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.398+0000 7fbd77c3f700 1 --2- 172.21.15.156:0/496233518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd70154dd0 0x7fbd701594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.398+0000 7fbd77c3f700 1 -- 172.21.15.156:0/496233518 >> 172.21.15.156:0/496233518 conn(0x7fbd700d0000 msgr2=0x7fbd7015f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.398+0000 7fbd77c3f700 1 -- 172.21.15.156:0/496233518 shutdown_connections 2023-11-28T11:08:54.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.398+0000 7fbd77c3f700 1 -- 172.21.15.156:0/496233518 wait complete. 2023-11-28T11:08:54.411 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:54.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.533+0000 7fdd48f92700 1 Processor -- start 2023-11-28T11:08:54.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.533+0000 7fdd48f92700 1 -- start start 2023-11-28T11:08:54.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.534+0000 7fdd48f92700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdd44140950 0x7fdd440b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.534+0000 7fdd48f92700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdd440b2e70 0x7fdd4406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.534+0000 7fdd48f92700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4413ff00 con 0x7fdd440cd8c0 2023-11-28T11:08:54.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.534+0000 7fdd48f92700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd441404c0 con 0x7fdd440b2e70 2023-11-28T11:08:54.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.534+0000 7fdd48f92700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd44140dc0 con 0x7fdd44140950 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.534+0000 7fdd42d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdd440b2e70 0x7fdd4406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.534+0000 7fdd42d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdd440b2e70 0x7fdd4406f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32984/0 (socket says 172.21.15.156:32984) 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.534+0000 7fdd42d9d700 1 -- 172.21.15.156:0/1290942973 learned_addr learned my addr 172.21.15.156:0/1290942973 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.535+0000 7fdd41d9b700 1 --2- 172.21.15.156:0/1290942973 >> v2:172.21.15.186:3300/0 conn(0x7fdd44140950 0x7fdd440b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.535+0000 7fdd42d9d700 1 -- 172.21.15.156:0/1290942973 >> v2:172.21.15.186:3300/0 conn(0x7fdd44140950 msgr2=0x7fdd440b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.535+0000 7fdd42d9d700 1 --2- 172.21.15.156:0/1290942973 >> v2:172.21.15.186:3300/0 conn(0x7fdd44140950 0x7fdd440b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.535+0000 7fdd42d9d700 1 -- 172.21.15.156:0/1290942973 >> v1:172.21.15.186:6789/0 conn(0x7fdd440cd8c0 legacy=0x7fdd440cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.535+0000 7fdd42d9d700 1 -- 172.21.15.156:0/1290942973 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4415aa80 con 0x7fdd440b2e70 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.535+0000 7fdd42d9d700 1 --2- 172.21.15.156:0/1290942973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd440b2e70 0x7fdd4406f850 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdd44071c40 tx=0x7fdd3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5861f8102e438787 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.535+0000 7fdd4159a700 1 -- 172.21.15.156:0/1290942973 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c00e070 con 0x7fdd440b2e70 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.535+0000 7fdd4159a700 1 -- 172.21.15.156:0/1290942973 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c003a20 con 0x7fdd440b2e70 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.535+0000 7fdd4159a700 1 -- 172.21.15.156:0/1290942973 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c003dd0 con 0x7fdd440b2e70 2023-11-28T11:08:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.535+0000 7fdd48f92700 1 -- 172.21.15.156:0/1290942973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd440b2e70 msgr2=0x7fdd4406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.535+0000 7fdd48f92700 1 --2- 172.21.15.156:0/1290942973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd440b2e70 0x7fdd4406f850 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdd44071c40 tx=0x7fdd3c00b330 comp rx=0 tx=0).stop 2023-11-28T11:08:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.536+0000 7fdd48f92700 1 -- 172.21.15.156:0/1290942973 shutdown_connections 2023-11-28T11:08:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.536+0000 7fdd48f92700 1 --2- 172.21.15.156:0/1290942973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd440b2e70 0x7fdd4406f850 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.536+0000 7fdd48f92700 1 --2- 172.21.15.156:0/1290942973 >> v2:172.21.15.186:3300/0 conn(0x7fdd44140950 0x7fdd440b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.536+0000 7fdd48f92700 1 -- 172.21.15.156:0/1290942973 >> 172.21.15.156:0/1290942973 conn(0x7fdd4406aed0 msgr2=0x7fdd4406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.536+0000 7fdd48f92700 1 -- 172.21.15.156:0/1290942973 shutdown_connections 2023-11-28T11:08:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.536+0000 7fdd48f92700 1 -- 172.21.15.156:0/1290942973 wait complete. 2023-11-28T11:08:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.537+0000 7fdd48f92700 1 Processor -- start 2023-11-28T11:08:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.537+0000 7fdd48f92700 1 -- start start 2023-11-28T11:08:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd48f92700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd440b2e70 0x7fdd4414fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd48f92700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd440cd8c0 0x7fdd441f0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd48f92700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd44140950 0x7fdd440b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd48f92700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440c5ab0 con 0x7fdd44140950 2023-11-28T11:08:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd48f92700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd44150320 con 0x7fdd440b2e70 2023-11-28T11:08:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd48f92700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd441f06a0 con 0x7fdd440cd8c0 2023-11-28T11:08:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd42d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd44140950 0x7fdd440b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd42d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd44140950 0x7fdd440b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:32992/0 (socket says 172.21.15.156:32992) 2023-11-28T11:08:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd42d9d700 1 -- 172.21.15.156:0/1010615632 learned_addr learned my addr 172.21.15.156:0/1010615632 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd4259c700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd440b2e70 0x7fdd4414fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd41d9b700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd440cd8c0 0x7fdd441f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd42d9d700 1 -- 172.21.15.156:0/1010615632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd440cd8c0 msgr2=0x7fdd441f0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd42d9d700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd440cd8c0 0x7fdd441f0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd42d9d700 1 -- 172.21.15.156:0/1010615632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd440b2e70 msgr2=0x7fdd4414fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd42d9d700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd440b2e70 0x7fdd4414fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd42d9d700 1 -- 172.21.15.156:0/1010615632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c009b30 con 0x7fdd44140950 2023-11-28T11:08:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd4259c700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd440b2e70 0x7fdd4414fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd41d9b700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd440cd8c0 0x7fdd441f0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd42d9d700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd44140950 0x7fdd440b5ed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c015040 tx=0x7fdd3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd32ffd700 1 -- 172.21.15.156:0/1010615632 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c004240 con 0x7fdd44140950 2023-11-28T11:08:54.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd32ffd700 1 -- 172.21.15.156:0/1010615632 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c0043a0 con 0x7fdd44140950 2023-11-28T11:08:54.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd48f92700 1 -- 172.21.15.156:0/1010615632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd440b35f0 con 0x7fdd44140950 2023-11-28T11:08:54.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.538+0000 7fdd32ffd700 1 -- 172.21.15.156:0/1010615632 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c004750 con 0x7fdd44140950 2023-11-28T11:08:54.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.539+0000 7fdd48f92700 1 -- 172.21.15.156:0/1010615632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd440b3310 con 0x7fdd44140950 2023-11-28T11:08:54.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.539+0000 7fdd48f92700 1 -- 172.21.15.156:0/1010615632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd44140150 con 0x7fdd44140950 2023-11-28T11:08:54.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.540+0000 7fdd32ffd700 1 -- 172.21.15.156:0/1010615632 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdd3c012430 con 0x7fdd44140950 2023-11-28T11:08:54.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.540+0000 7fdd32ffd700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd240682f0 0x7fdd2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.540+0000 7fdd32ffd700 1 -- 172.21.15.156:0/1010615632 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdd3c0870f0 con 0x7fdd44140950 2023-11-28T11:08:54.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.540+0000 7fdd4259c700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd240682f0 0x7fdd2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.541+0000 7fdd4259c700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd240682f0 0x7fdd2406a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd44157700 tx=0x7fdd34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.544+0000 7fdd32ffd700 1 -- 172.21.15.156:0/1010615632 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdd3c053870 con 0x7fdd44140950 2023-11-28T11:08:54.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.726+0000 7fdd48f92700 1 -- 172.21.15.156:0/1010615632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdd44068690 con 0x7fdd44140950 2023-11-28T11:08:54.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.727+0000 7fdd32ffd700 1 -- 172.21.15.156:0/1010615632 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdd3c020020 con 0x7fdd44140950 2023-11-28T11:08:54.728 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:08:54.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.729+0000 7fdd48f92700 1 -- 172.21.15.156:0/1010615632 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd240682f0 msgr2=0x7fdd2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.729+0000 7fdd48f92700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd240682f0 0x7fdd2406a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd44157700 tx=0x7fdd34009040 comp rx=0 tx=0).stop 2023-11-28T11:08:54.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.729+0000 7fdd48f92700 1 -- 172.21.15.156:0/1010615632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd44140950 msgr2=0x7fdd440b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.729+0000 7fdd48f92700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd44140950 0x7fdd440b5ed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c015040 tx=0x7fdd3c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.729+0000 7fdd48f92700 1 -- 172.21.15.156:0/1010615632 shutdown_connections 2023-11-28T11:08:54.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.729+0000 7fdd48f92700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd44140950 0x7fdd440b5ed0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.729+0000 7fdd48f92700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd440b2e70 0x7fdd4414fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.729+0000 7fdd48f92700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd240682f0 0x7fdd2406a7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.729+0000 7fdd48f92700 1 --2- 172.21.15.156:0/1010615632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd440cd8c0 0x7fdd441f0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.729+0000 7fdd48f92700 1 -- 172.21.15.156:0/1010615632 >> 172.21.15.156:0/1010615632 conn(0x7fdd4406aed0 msgr2=0x7fdd440d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.729+0000 7fdd48f92700 1 -- 172.21.15.156:0/1010615632 shutdown_connections 2023-11-28T11:08:54.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.729+0000 7fdd48f92700 1 -- 172.21.15.156:0/1010615632 wait complete. 2023-11-28T11:08:54.742 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:54.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.866+0000 7f2a612b6700 1 Processor -- start 2023-11-28T11:08:54.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.866+0000 7f2a612b6700 1 -- start start 2023-11-28T11:08:54.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.866+0000 7f2a612b6700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2a5c160740 0x7f2a5c0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a612b6700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5c06bbb0 con 0x7f2a5c06b060 2023-11-28T11:08:54.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a612b6700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5c0cc030 con 0x7f2a5c160b10 2023-11-28T11:08:54.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a612b6700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c0ca720 con 0x7f2a5c160740 2023-11-28T11:08:54.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a5affd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2a5c160740 0x7f2a5c0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a5b7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2a5c06b060 0x7f2a5c06b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47878/0 (socket says 172.21.15.156:47878) 2023-11-28T11:08:54.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a5b7fe700 1 -- 172.21.15.156:0/3771298269 learned_addr learned my addr 172.21.15.156:0/3771298269 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:54.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a5affd700 1 -- 172.21.15.156:0/3771298269 >> v1:172.21.15.186:6789/0 conn(0x7f2a5c160b10 legacy=0x7f2a5c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a5affd700 1 -- 172.21.15.156:0/3771298269 >> v1:172.21.15.156:6789/0 conn(0x7f2a5c06b060 legacy=0x7f2a5c06b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a5affd700 1 -- 172.21.15.156:0/3771298269 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c0c9e10 con 0x7f2a5c160740 2023-11-28T11:08:54.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a59ffb700 1 -- 172.21.15.156:0/3771298269 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408160956 0 0) 0x7f2a5c06bbb0 con 0x7f2a5c06b060 2023-11-28T11:08:54.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a5affd700 1 --2- 172.21.15.156:0/3771298269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a5c160740 0x7f2a5c0cbaf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009840 tx=0x7f2a4c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a52308ccab7d8c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a59ffb700 1 -- 172.21.15.156:0/3771298269 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c011830 con 0x7f2a5c160740 2023-11-28T11:08:54.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a59ffb700 1 -- 172.21.15.156:0/3771298269 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c011990 con 0x7f2a5c160740 2023-11-28T11:08:54.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.867+0000 7f2a59ffb700 1 -- 172.21.15.156:0/3771298269 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c011d20 con 0x7f2a5c160740 2023-11-28T11:08:54.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.868+0000 7f2a612b6700 1 -- 172.21.15.156:0/3771298269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a5c160740 msgr2=0x7f2a5c0cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.868+0000 7f2a612b6700 1 --2- 172.21.15.156:0/3771298269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a5c160740 0x7f2a5c0cbaf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009840 tx=0x7f2a4c009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:54.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.868+0000 7f2a612b6700 1 -- 172.21.15.156:0/3771298269 shutdown_connections 2023-11-28T11:08:54.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.868+0000 7f2a612b6700 1 --2- 172.21.15.156:0/3771298269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a5c160740 0x7f2a5c0cbaf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.868+0000 7f2a612b6700 1 -- 172.21.15.156:0/3771298269 >> 172.21.15.156:0/3771298269 conn(0x7f2a5c0b12b0 msgr2=0x7f2a5c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:54.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.868+0000 7f2a612b6700 1 -- 172.21.15.156:0/3771298269 shutdown_connections 2023-11-28T11:08:54.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.868+0000 7f2a612b6700 1 -- 172.21.15.156:0/3771298269 wait complete. 2023-11-28T11:08:54.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.869+0000 7f2a612b6700 1 Processor -- start 2023-11-28T11:08:54.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.869+0000 7f2a612b6700 1 -- start start 2023-11-28T11:08:54.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a612b6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a5c06b060 0x7f2a5c0c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a612b6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a5c160740 0x7f2a5c0c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a612b6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a5c160b10 0x7f2a5c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a612b6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0b10c0 con 0x7f2a5c160b10 2023-11-28T11:08:54.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a612b6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0c96f0 con 0x7f2a5c160740 2023-11-28T11:08:54.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a612b6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0c3830 con 0x7f2a5c06b060 2023-11-28T11:08:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a5b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a5c160b10 0x7f2a5c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a5b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a5c160b10 0x7f2a5c06da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33004/0 (socket says 172.21.15.156:33004) 2023-11-28T11:08:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a5b7fe700 1 -- 172.21.15.156:0/2855417839 learned_addr learned my addr 172.21.15.156:0/2855417839 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a5a7fc700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a5c160740 0x7f2a5c0c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a5affd700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a5c06b060 0x7f2a5c0c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a5b7fe700 1 -- 172.21.15.156:0/2855417839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a5c06b060 msgr2=0x7f2a5c0c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a5b7fe700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a5c06b060 0x7f2a5c0c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a5b7fe700 1 -- 172.21.15.156:0/2855417839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a5c160740 msgr2=0x7f2a5c0c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a5b7fe700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a5c160740 0x7f2a5c0c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a5b7fe700 1 -- 172.21.15.156:0/2855417839 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c0094a0 con 0x7f2a5c160b10 2023-11-28T11:08:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.870+0000 7f2a5b7fe700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a5c160b10 0x7f2a5c06da70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400cb90 tx=0x7f2a5400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.871+0000 7f2a437fe700 1 -- 172.21.15.156:0/2855417839 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400ab30 con 0x7f2a5c160b10 2023-11-28T11:08:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.871+0000 7f2a437fe700 1 -- 172.21.15.156:0/2855417839 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5400ac90 con 0x7f2a5c160b10 2023-11-28T11:08:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.871+0000 7f2a612b6700 1 -- 172.21.15.156:0/2855417839 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5c0ca0d0 con 0x7f2a5c160b10 2023-11-28T11:08:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.871+0000 7f2a437fe700 1 -- 172.21.15.156:0/2855417839 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540070d0 con 0x7f2a5c160b10 2023-11-28T11:08:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.871+0000 7f2a612b6700 1 -- 172.21.15.156:0/2855417839 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c06efc0 con 0x7f2a5c160b10 2023-11-28T11:08:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.872+0000 7f2a612b6700 1 -- 172.21.15.156:0/2855417839 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c0ca3b0 con 0x7f2a5c160b10 2023-11-28T11:08:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.872+0000 7f2a437fe700 1 -- 172.21.15.156:0/2855417839 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2a5401a030 con 0x7f2a5c160b10 2023-11-28T11:08:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.872+0000 7f2a437fe700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2a480682f0 0x7f2a4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.873+0000 7f2a437fe700 1 -- 172.21.15.156:0/2855417839 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2a54085900 con 0x7f2a5c160b10 2023-11-28T11:08:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.873+0000 7f2a5affd700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2a480682f0 0x7f2a4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.873+0000 7f2a5affd700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2a480682f0 0x7f2a4806a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0090b0 tx=0x7f2a4c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:54.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:54.877+0000 7f2a437fe700 1 -- 172.21.15.156:0/2855417839 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2a54052160 con 0x7f2a5c160b10 2023-11-28T11:08:55.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.061+0000 7f2a612b6700 1 -- 172.21.15.156:0/2855417839 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a5c0745c0 con 0x7f2a5c160b10 2023-11-28T11:08:55.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.061+0000 7f2a437fe700 1 -- 172.21.15.156:0/2855417839 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a54018020 con 0x7f2a5c160b10 2023-11-28T11:08:55.062 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:08:55.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.063+0000 7f2a612b6700 1 -- 172.21.15.156:0/2855417839 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2a480682f0 msgr2=0x7f2a4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.063+0000 7f2a612b6700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2a480682f0 0x7f2a4806a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0090b0 tx=0x7f2a4c008030 comp rx=0 tx=0).stop 2023-11-28T11:08:55.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.063+0000 7f2a612b6700 1 -- 172.21.15.156:0/2855417839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a5c160b10 msgr2=0x7f2a5c06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.063+0000 7f2a612b6700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a5c160b10 0x7f2a5c06da70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400cb90 tx=0x7f2a5400cf50 comp rx=0 tx=0).stop 2023-11-28T11:08:55.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.063+0000 7f2a612b6700 1 -- 172.21.15.156:0/2855417839 shutdown_connections 2023-11-28T11:08:55.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.063+0000 7f2a612b6700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a5c160b10 0x7f2a5c06da70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.063+0000 7f2a612b6700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2a480682f0 0x7f2a4806a7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.063+0000 7f2a612b6700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a5c06b060 0x7f2a5c0c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.063+0000 7f2a612b6700 1 --2- 172.21.15.156:0/2855417839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a5c160740 0x7f2a5c0c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.063+0000 7f2a612b6700 1 -- 172.21.15.156:0/2855417839 >> 172.21.15.156:0/2855417839 conn(0x7f2a5c0b12b0 msgr2=0x7f2a5c0ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.063+0000 7f2a612b6700 1 -- 172.21.15.156:0/2855417839 shutdown_connections 2023-11-28T11:08:55.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.063+0000 7f2a612b6700 1 -- 172.21.15.156:0/2855417839 wait complete. 2023-11-28T11:08:55.079 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:55.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.201+0000 7f214d71d700 1 Processor -- start 2023-11-28T11:08:55.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.201+0000 7f214d71d700 1 -- start start 2023-11-28T11:08:55.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.202+0000 7f214d71d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f21480cd8c0 0x7f2148140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.202+0000 7f214d71d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f21480cdc90 0x7f21480b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.202+0000 7f214d71d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214813ff00 con 0x7f21480b2e70 2023-11-28T11:08:55.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.202+0000 7f214d71d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21481404c0 con 0x7f21480cd8c0 2023-11-28T11:08:55.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.202+0000 7f214d71d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214815aa80 con 0x7f21480cdc90 2023-11-28T11:08:55.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.202+0000 7f2146ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f21480cd8c0 0x7f2148140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.202+0000 7f21477fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f21480b2e70 0x7f214806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47892/0 (socket says 172.21.15.156:47892) 2023-11-28T11:08:55.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.202+0000 7f21477fe700 1 -- 172.21.15.156:0/1839744933 learned_addr learned my addr 172.21.15.156:0/1839744933 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:55.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.202+0000 7f21467fc700 1 --2- 172.21.15.156:0/1839744933 >> v2:172.21.15.186:3300/0 conn(0x7f21480cdc90 0x7f21480b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.203+0000 7f2146ffd700 1 -- 172.21.15.156:0/1839744933 >> v2:172.21.15.186:3300/0 conn(0x7f21480cdc90 msgr2=0x7f21480b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:55.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.203+0000 7f2146ffd700 1 --2- 172.21.15.156:0/1839744933 >> v2:172.21.15.186:3300/0 conn(0x7f21480cdc90 0x7f21480b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.203+0000 7f2146ffd700 1 -- 172.21.15.156:0/1839744933 >> v1:172.21.15.156:6789/0 conn(0x7f21480b2e70 legacy=0x7f214806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.203+0000 7f2146ffd700 1 -- 172.21.15.156:0/1839744933 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21480b3310 con 0x7f21480cd8c0 2023-11-28T11:08:55.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.203+0000 7f2145ffb700 1 -- 172.21.15.156:0/1839744933 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815129092 0 0) 0x7f214813ff00 con 0x7f21480b2e70 2023-11-28T11:08:55.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.203+0000 7f2146ffd700 1 --2- 172.21.15.156:0/1839744933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21480cd8c0 0x7f2148140950 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2148071c40 tx=0x7f213c009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd13625d930e03c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.203+0000 7f2145ffb700 1 -- 172.21.15.156:0/1839744933 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c005e30 con 0x7f21480cd8c0 2023-11-28T11:08:55.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.203+0000 7f2145ffb700 1 -- 172.21.15.156:0/1839744933 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c006030 con 0x7f21480cd8c0 2023-11-28T11:08:55.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.203+0000 7f2145ffb700 1 -- 172.21.15.156:0/1839744933 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c0063c0 con 0x7f21480cd8c0 2023-11-28T11:08:55.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.204+0000 7f214d71d700 1 -- 172.21.15.156:0/1839744933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21480cd8c0 msgr2=0x7f2148140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.204+0000 7f214d71d700 1 --2- 172.21.15.156:0/1839744933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21480cd8c0 0x7f2148140950 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2148071c40 tx=0x7f213c009b00 comp rx=0 tx=0).stop 2023-11-28T11:08:55.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.204+0000 7f214d71d700 1 -- 172.21.15.156:0/1839744933 shutdown_connections 2023-11-28T11:08:55.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.204+0000 7f214d71d700 1 --2- 172.21.15.156:0/1839744933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21480cd8c0 0x7f2148140950 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.204+0000 7f214d71d700 1 --2- 172.21.15.156:0/1839744933 >> v2:172.21.15.186:3300/0 conn(0x7f21480cdc90 0x7f21480b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.204+0000 7f214d71d700 1 -- 172.21.15.156:0/1839744933 >> 172.21.15.156:0/1839744933 conn(0x7f214806aed0 msgr2=0x7f214806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.204+0000 7f214d71d700 1 -- 172.21.15.156:0/1839744933 shutdown_connections 2023-11-28T11:08:55.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.204+0000 7f214d71d700 1 -- 172.21.15.156:0/1839744933 wait complete. 2023-11-28T11:08:55.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.205+0000 7f214d71d700 1 Processor -- start 2023-11-28T11:08:55.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.205+0000 7f214d71d700 1 -- start start 2023-11-28T11:08:55.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f214d71d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21480b2e70 0x7f214806d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f214d71d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21480cd8c0 0x7f2148153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f214d71d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21480cdc90 0x7f21481586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f214d71d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2148158260 con 0x7f21480b2e70 2023-11-28T11:08:55.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f214d71d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21481544c0 con 0x7f21480cdc90 2023-11-28T11:08:55.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f214d71d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214815d570 con 0x7f21480cd8c0 2023-11-28T11:08:55.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f2146ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21480b2e70 0x7f214806d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f2146ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21480b2e70 0x7f214806d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33016/0 (socket says 172.21.15.156:33016) 2023-11-28T11:08:55.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f2146ffd700 1 -- 172.21.15.156:0/3268609671 learned_addr learned my addr 172.21.15.156:0/3268609671 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:55.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f21477fe700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21480cdc90 0x7f21481586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f21467fc700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21480cd8c0 0x7f2148153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f2146ffd700 1 -- 172.21.15.156:0/3268609671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21480cd8c0 msgr2=0x7f2148153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f2146ffd700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21480cd8c0 0x7f2148153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f2146ffd700 1 -- 172.21.15.156:0/3268609671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21480cdc90 msgr2=0x7f21481586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f2146ffd700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21480cdc90 0x7f21481586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f2146ffd700 1 -- 172.21.15.156:0/3268609671 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c009430 con 0x7f21480b2e70 2023-11-28T11:08:55.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f21467fc700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21480cd8c0 0x7f2148153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:55.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f2146ffd700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21480b2e70 0x7f214806d010 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f213c000c00 tx=0x7f213c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f212f7fe700 1 -- 172.21.15.156:0/3268609671 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c0068f0 con 0x7f21480b2e70 2023-11-28T11:08:55.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f212f7fe700 1 -- 172.21.15.156:0/3268609671 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c020070 con 0x7f21480b2e70 2023-11-28T11:08:55.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f214d71d700 1 -- 172.21.15.156:0/3268609671 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2148140150 con 0x7f21480b2e70 2023-11-28T11:08:55.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f212f7fe700 1 -- 172.21.15.156:0/3268609671 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c004280 con 0x7f21480b2e70 2023-11-28T11:08:55.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.206+0000 7f214d71d700 1 -- 172.21.15.156:0/3268609671 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21480b35f0 con 0x7f21480b2e70 2023-11-28T11:08:55.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.207+0000 7f214d71d700 1 -- 172.21.15.156:0/3268609671 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21480ce100 con 0x7f21480b2e70 2023-11-28T11:08:55.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.208+0000 7f212f7fe700 1 -- 172.21.15.156:0/3268609671 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f213c02b020 con 0x7f21480b2e70 2023-11-28T11:08:55.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.208+0000 7f212f7fe700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f21340682f0 0x7f213406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.208+0000 7f212f7fe700 1 -- 172.21.15.156:0/3268609671 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f213c0872b0 con 0x7f21480b2e70 2023-11-28T11:08:55.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.208+0000 7f21467fc700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f21340682f0 0x7f213406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.209+0000 7f21467fc700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f21340682f0 0x7f213406a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2148154880 tx=0x7f2138008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.212+0000 7f212f7fe700 1 -- 172.21.15.156:0/3268609671 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f213c053b10 con 0x7f21480b2e70 2023-11-28T11:08:55.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.395+0000 7f214d71d700 1 -- 172.21.15.156:0/3268609671 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2148158540 con 0x7f21480b2e70 2023-11-28T11:08:55.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.395+0000 7f212f7fe700 1 -- 172.21.15.156:0/3268609671 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f213c057130 con 0x7f21480b2e70 2023-11-28T11:08:55.396 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:08:55.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.397+0000 7f214d71d700 1 -- 172.21.15.156:0/3268609671 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f21340682f0 msgr2=0x7f213406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.397+0000 7f214d71d700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f21340682f0 0x7f213406a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2148154880 tx=0x7f2138008040 comp rx=0 tx=0).stop 2023-11-28T11:08:55.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.397+0000 7f214d71d700 1 -- 172.21.15.156:0/3268609671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21480b2e70 msgr2=0x7f214806d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.397+0000 7f214d71d700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21480b2e70 0x7f214806d010 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f213c000c00 tx=0x7f213c004750 comp rx=0 tx=0).stop 2023-11-28T11:08:55.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.397+0000 7f214d71d700 1 -- 172.21.15.156:0/3268609671 shutdown_connections 2023-11-28T11:08:55.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.397+0000 7f214d71d700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21480cdc90 0x7f21481586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.397+0000 7f214d71d700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21480b2e70 0x7f214806d010 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.397+0000 7f214d71d700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f21340682f0 0x7f213406a7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.397+0000 7f214d71d700 1 --2- 172.21.15.156:0/3268609671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21480cd8c0 0x7f2148153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.397+0000 7f214d71d700 1 -- 172.21.15.156:0/3268609671 >> 172.21.15.156:0/3268609671 conn(0x7f214806aed0 msgr2=0x7f21480cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.397+0000 7f214d71d700 1 -- 172.21.15.156:0/3268609671 shutdown_connections 2023-11-28T11:08:55.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.397+0000 7f214d71d700 1 -- 172.21.15.156:0/3268609671 wait complete. 2023-11-28T11:08:55.411 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:08:55.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.531+0000 7fd7a1525700 1 Processor -- start 2023-11-28T11:08:55.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.531+0000 7fd7a1525700 1 -- start start 2023-11-28T11:08:55.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.532+0000 7fd7a1525700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd79c070c50 0x7fd79c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.532+0000 7fd7a1525700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79c15e1a0 con 0x7fd79c158200 2023-11-28T11:08:55.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.532+0000 7fd7a1525700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79c0b0fd0 con 0x7fd79c0b0bb0 2023-11-28T11:08:55.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.532+0000 7fd7a1525700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79c15f070 con 0x7fd79c070c50 2023-11-28T11:08:55.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.532+0000 7fd79a7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd79c070c50 0x7fd79c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.532+0000 7fd79affd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd79c158200 0x7fd79c0caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47902/0 (socket says 172.21.15.156:47902) 2023-11-28T11:08:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.532+0000 7fd79affd700 1 -- 172.21.15.156:0/2584989466 learned_addr learned my addr 172.21.15.156:0/2584989466 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.532+0000 7fd79a7fc700 1 -- 172.21.15.156:0/2584989466 >> v1:172.21.15.186:6789/0 conn(0x7fd79c0b0bb0 legacy=0x7fd79c15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.532+0000 7fd79a7fc700 1 -- 172.21.15.156:0/2584989466 >> v1:172.21.15.156:6789/0 conn(0x7fd79c158200 legacy=0x7fd79c0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.532+0000 7fd799ffb700 1 -- 172.21.15.156:0/2584989466 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4184377689 0 0) 0x7fd79c15e1a0 con 0x7fd79c158200 2023-11-28T11:08:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.532+0000 7fd79a7fc700 1 -- 172.21.15.156:0/2584989466 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c0cb180 con 0x7fd79c070c50 2023-11-28T11:08:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.533+0000 7fd79a7fc700 1 --2- 172.21.15.156:0/2584989466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd79c070c50 0x7fd79c0b0670 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009840 tx=0x7fd78c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75c4587384338f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.533+0000 7fd799ffb700 1 -- 172.21.15.156:0/2584989466 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c011830 con 0x7fd79c070c50 2023-11-28T11:08:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.533+0000 7fd799ffb700 1 -- 172.21.15.156:0/2584989466 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c011990 con 0x7fd79c070c50 2023-11-28T11:08:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.533+0000 7fd799ffb700 1 -- 172.21.15.156:0/2584989466 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c011d20 con 0x7fd79c070c50 2023-11-28T11:08:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.533+0000 7fd7a1525700 1 -- 172.21.15.156:0/2584989466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd79c070c50 msgr2=0x7fd79c0b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.533+0000 7fd7a1525700 1 --2- 172.21.15.156:0/2584989466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd79c070c50 0x7fd79c0b0670 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009840 tx=0x7fd78c009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.533+0000 7fd7a1525700 1 -- 172.21.15.156:0/2584989466 shutdown_connections 2023-11-28T11:08:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.533+0000 7fd7a1525700 1 --2- 172.21.15.156:0/2584989466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd79c070c50 0x7fd79c0b0670 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.533+0000 7fd7a1525700 1 -- 172.21.15.156:0/2584989466 >> 172.21.15.156:0/2584989466 conn(0x7fd79c0cb670 msgr2=0x7fd79c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.533+0000 7fd7a1525700 1 -- 172.21.15.156:0/2584989466 shutdown_connections 2023-11-28T11:08:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.533+0000 7fd7a1525700 1 -- 172.21.15.156:0/2584989466 wait complete. 2023-11-28T11:08:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.535+0000 7fd7a1525700 1 Processor -- start 2023-11-28T11:08:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.535+0000 7fd7a1525700 1 -- start start 2023-11-28T11:08:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.535+0000 7fd7a1525700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd79c070c50 0x7fd79c1f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.535+0000 7fd7a1525700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd79c0b0bb0 0x7fd79c0addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.535+0000 7fd7a1525700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd79c158200 0x7fd79c156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.535+0000 7fd7a1525700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c1f6560 con 0x7fd79c070c50 2023-11-28T11:08:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.535+0000 7fd7a1525700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c1f5cc0 con 0x7fd79c0b0bb0 2023-11-28T11:08:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.535+0000 7fd7a1525700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c1f5fa0 con 0x7fd79c158200 2023-11-28T11:08:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.535+0000 7fd79affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd79c070c50 0x7fd79c1f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd79affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd79c070c50 0x7fd79c1f56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33030/0 (socket says 172.21.15.156:33030) 2023-11-28T11:08:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd79affd700 1 -- 172.21.15.156:0/2885735087 learned_addr learned my addr 172.21.15.156:0/2885735087 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd79a7fc700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd79c0b0bb0 0x7fd79c0addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd79b7fe700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd79c158200 0x7fd79c156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd79affd700 1 -- 172.21.15.156:0/2885735087 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd79c158200 msgr2=0x7fd79c156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd79affd700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd79c158200 0x7fd79c156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd79affd700 1 -- 172.21.15.156:0/2885735087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd79c0b0bb0 msgr2=0x7fd79c0addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd79affd700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd79c0b0bb0 0x7fd79c0addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd79affd700 1 -- 172.21.15.156:0/2885735087 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c0094a0 con 0x7fd79c070c50 2023-11-28T11:08:55.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd79b7fe700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd79c158200 0x7fd79c156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:55.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd79affd700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd79c070c50 0x7fd79c1f56e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd79000c520 tx=0x7fd79000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd7837fe700 1 -- 172.21.15.156:0/2885735087 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd790004c80 con 0x7fd79c070c50 2023-11-28T11:08:55.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd7837fe700 1 -- 172.21.15.156:0/2885735087 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd790004e20 con 0x7fd79c070c50 2023-11-28T11:08:55.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd7a1525700 1 -- 172.21.15.156:0/2885735087 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79c1585d0 con 0x7fd79c070c50 2023-11-28T11:08:55.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd7837fe700 1 -- 172.21.15.156:0/2885735087 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd790007310 con 0x7fd79c070c50 2023-11-28T11:08:55.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.536+0000 7fd7a1525700 1 -- 172.21.15.156:0/2885735087 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79c0710f0 con 0x7fd79c070c50 2023-11-28T11:08:55.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.537+0000 7fd7a1525700 1 -- 172.21.15.156:0/2885735087 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79c15e3f0 con 0x7fd79c070c50 2023-11-28T11:08:55.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.538+0000 7fd7837fe700 1 -- 172.21.15.156:0/2885735087 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd79001a020 con 0x7fd79c070c50 2023-11-28T11:08:55.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.538+0000 7fd7837fe700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7880682f0 0x7fd78806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.538+0000 7fd7837fe700 1 -- 172.21.15.156:0/2885735087 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd790085ea0 con 0x7fd79c070c50 2023-11-28T11:08:55.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.538+0000 7fd79a7fc700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7880682f0 0x7fd78806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.539+0000 7fd79a7fc700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7880682f0 0x7fd78806a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd79c159510 tx=0x7fd78c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.542+0000 7fd7837fe700 1 -- 172.21.15.156:0/2885735087 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd7900527a0 con 0x7fd79c070c50 2023-11-28T11:08:55.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.739+0000 7fd7a1525700 1 -- 172.21.15.156:0/2885735087 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd79c06fd60 con 0x7fd79c070c50 2023-11-28T11:08:55.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.739+0000 7fd7837fe700 1 -- 172.21.15.156:0/2885735087 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd790055dc0 con 0x7fd79c070c50 2023-11-28T11:08:55.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.742+0000 7fd7a1525700 1 -- 172.21.15.156:0/2885735087 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7880682f0 msgr2=0x7fd78806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.742+0000 7fd7a1525700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7880682f0 0x7fd78806a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd79c159510 tx=0x7fd78c01c040 comp rx=0 tx=0).stop 2023-11-28T11:08:55.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.742+0000 7fd7a1525700 1 -- 172.21.15.156:0/2885735087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd79c070c50 msgr2=0x7fd79c1f56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.742+0000 7fd7a1525700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd79c070c50 0x7fd79c1f56e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd79000c520 tx=0x7fd79000c8e0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.742+0000 7fd7a1525700 1 -- 172.21.15.156:0/2885735087 shutdown_connections 2023-11-28T11:08:55.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.742+0000 7fd7a1525700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7880682f0 0x7fd78806a7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.742+0000 7fd7a1525700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd79c158200 0x7fd79c156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.742+0000 7fd7a1525700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd79c070c50 0x7fd79c1f56e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.742+0000 7fd7a1525700 1 --2- 172.21.15.156:0/2885735087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd79c0b0bb0 0x7fd79c0addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.742+0000 7fd7a1525700 1 -- 172.21.15.156:0/2885735087 >> 172.21.15.156:0/2885735087 conn(0x7fd79c0cb670 msgr2=0x7fd79c143900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.742+0000 7fd7a1525700 1 -- 172.21.15.156:0/2885735087 shutdown_connections 2023-11-28T11:08:55.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:55.742+0000 7fd7a1525700 1 -- 172.21.15.156:0/2885735087 wait complete. 2023-11-28T11:08:55.745 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T11:08:55.755 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-28T11:08:55.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:55.755 DEBUG:teuthology.orchestra.run.smithi186:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-28T11:08:55.814 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T11:08:55.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:55.815 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T11:08:55.845 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:08:55.846 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:55.864 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:08:55.864 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:55.921 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:08:55.922 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:55.979 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:56.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.139+0000 7fdf4d056700 1 Processor -- start 2023-11-28T11:08:56.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.139+0000 7fdf4d056700 1 -- start start 2023-11-28T11:08:56.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf4d056700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdf480ae9d0 0x7fdf4815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf4d056700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf480aee40 con 0x7fdf4806e500 2023-11-28T11:08:56.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf4d056700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4806c930 con 0x7fdf4806a5e0 2023-11-28T11:08:56.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf4d056700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4815bf10 con 0x7fdf480ae9d0 2023-11-28T11:08:56.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf46d9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fdf4806e500 0x7fdf4806e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47916/0 (socket says 172.21.15.156:47916) 2023-11-28T11:08:56.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf46d9d700 1 -- 172.21.15.156:0/2748681643 learned_addr learned my addr 172.21.15.156:0/2748681643 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:56.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074533584 0 0) 0x7fdf480aee40 con 0x7fdf4806e500 2023-11-28T11:08:56.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf4659c700 1 --2- 172.21.15.156:0/2748681643 >> v2:172.21.15.186:3300/0 conn(0x7fdf480ae9d0 0x7fdf4815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf300035b0 con 0x7fdf4806e500 2023-11-28T11:08:56.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3894273972 0 0) 0x7fdf300035b0 con 0x7fdf4806e500 2023-11-28T11:08:56.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 >> v2:172.21.15.186:3300/0 conn(0x7fdf480ae9d0 msgr2=0x7fdf4815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf45d9b700 1 --2- 172.21.15.156:0/2748681643 >> v2:172.21.15.186:3300/0 conn(0x7fdf480ae9d0 0x7fdf4815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 >> v1:172.21.15.186:6789/0 conn(0x7fdf4806a5e0 legacy=0x7fdf4806a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.140+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4806cc10 con 0x7fdf4806e500 2023-11-28T11:08:56.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fdf3c001670 con 0x7fdf4806e500 2023-11-28T11:08:56.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 >> v1:172.21.15.156:6789/0 conn(0x7fdf4806e500 legacy=0x7fdf4806e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf45d9b700 1 --2- 172.21.15.156:0/2748681643 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf30003a00 0x7fdf30003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf45d9b700 1 --2- 172.21.15.156:0/2748681643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 0x7fdf30004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf45d9b700 1 --2- 172.21.15.156:0/2748681643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf30005320 0x7fdf3000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4815bf10 con 0x7fdf30004850 2023-11-28T11:08:56.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3000c250 con 0x7fdf30005320 2023-11-28T11:08:56.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3000c410 con 0x7fdf30003a00 2023-11-28T11:08:56.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf4659c700 1 --2- 172.21.15.156:0/2748681643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 0x7fdf30004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf46d9d700 1 --2- 172.21.15.156:0/2748681643 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf30003a00 0x7fdf30003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf4759e700 1 --2- 172.21.15.156:0/2748681643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf30005320 0x7fdf3000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf4659c700 1 -- 172.21.15.156:0/2748681643 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf30003a00 msgr2=0x7fdf30003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf4659c700 1 --2- 172.21.15.156:0/2748681643 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf30003a00 0x7fdf30003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf4659c700 1 -- 172.21.15.156:0/2748681643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf30005320 msgr2=0x7fdf3000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf4659c700 1 --2- 172.21.15.156:0/2748681643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf30005320 0x7fdf3000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf4659c700 1 -- 172.21.15.156:0/2748681643 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf300035b0 con 0x7fdf30004850 2023-11-28T11:08:56.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf4759e700 1 -- 172.21.15.156:0/2748681643 reap_dead start 2023-11-28T11:08:56.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf4659c700 1 --2- 172.21.15.156:0/2748681643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 0x7fdf30004cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdf480707b0 tx=0x7fdf30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cff2eb736ad3001 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3800e6c0 con 0x7fdf30004850 2023-11-28T11:08:56.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3800e820 con 0x7fdf30004850 2023-11-28T11:08:56.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.141+0000 7fdf45d9b700 1 -- 172.21.15.156:0/2748681643 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3800eb90 con 0x7fdf30004850 2023-11-28T11:08:56.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.142+0000 7fdf4d056700 1 -- 172.21.15.156:0/2748681643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 msgr2=0x7fdf30004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.142+0000 7fdf4d056700 1 --2- 172.21.15.156:0/2748681643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 0x7fdf30004cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdf480707b0 tx=0x7fdf30005880 comp rx=0 tx=0).stop 2023-11-28T11:08:56.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.142+0000 7fdf4d056700 1 -- 172.21.15.156:0/2748681643 shutdown_connections 2023-11-28T11:08:56.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.142+0000 7fdf4d056700 1 --2- 172.21.15.156:0/2748681643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 0x7fdf30004cc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.142+0000 7fdf4d056700 1 -- 172.21.15.156:0/2748681643 >> 172.21.15.156:0/2748681643 conn(0x7fdf480b12b0 msgr2=0x7fdf480b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.142+0000 7fdf4d056700 1 -- 172.21.15.156:0/2748681643 shutdown_connections 2023-11-28T11:08:56.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.142+0000 7fdf4d056700 1 -- 172.21.15.156:0/2748681643 wait complete. 2023-11-28T11:08:56.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.143+0000 7fdf4d056700 1 Processor -- start 2023-11-28T11:08:56.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.143+0000 7fdf4d056700 1 -- start start 2023-11-28T11:08:56.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf4d056700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 0x7fdf4814ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf4d056700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf481e4ff0 0x7fdf481e5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf4d056700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf480c8ea0 0x7fdf480c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf4d056700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480c7540 con 0x7fdf30004850 2023-11-28T11:08:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf4d056700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf481e59a0 con 0x7fdf481e4ff0 2023-11-28T11:08:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf4d056700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf481e6ce0 con 0x7fdf480c8ea0 2023-11-28T11:08:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf46d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 0x7fdf4814ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf46d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 0x7fdf4814ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33050/0 (socket says 172.21.15.156:33050) 2023-11-28T11:08:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf46d9d700 1 -- 172.21.15.156:0/16988834 learned_addr learned my addr 172.21.15.156:0/16988834 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf4659c700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf481e4ff0 0x7fdf481e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf4759e700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf480c8ea0 0x7fdf480c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf46d9d700 1 -- 172.21.15.156:0/16988834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf480c8ea0 msgr2=0x7fdf480c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf46d9d700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf480c8ea0 0x7fdf480c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf46d9d700 1 -- 172.21.15.156:0/16988834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf481e4ff0 msgr2=0x7fdf481e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf46d9d700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf481e4ff0 0x7fdf481e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.144+0000 7fdf46d9d700 1 -- 172.21.15.156:0/16988834 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4806cef0 con 0x7fdf30004850 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.145+0000 7fdf46d9d700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 0x7fdf4814ed50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00e780 tx=0x7fdf3c00eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.145+0000 7fdf2f7fe700 1 -- 172.21.15.156:0/16988834 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c012590 con 0x7fdf30004850 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.145+0000 7fdf2f7fe700 1 -- 172.21.15.156:0/16988834 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c012710 con 0x7fdf30004850 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.145+0000 7fdf4d056700 1 -- 172.21.15.156:0/16988834 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf480664c0 con 0x7fdf30004850 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.145+0000 7fdf2f7fe700 1 -- 172.21.15.156:0/16988834 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c012ac0 con 0x7fdf30004850 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.145+0000 7fdf4d056700 1 -- 172.21.15.156:0/16988834 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4815ac80 con 0x7fdf30004850 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.146+0000 7fdf4d056700 1 -- 172.21.15.156:0/16988834 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf48065fc0 con 0x7fdf30004850 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.146+0000 7fdf2f7fe700 1 -- 172.21.15.156:0/16988834 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdf3c012c20 con 0x7fdf30004850 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.147+0000 7fdf2f7fe700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdf340682f0 0x7fdf3406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.147+0000 7fdf2f7fe700 1 -- 172.21.15.156:0/16988834 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdf3c085c80 con 0x7fdf30004850 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.147+0000 7fdf4659c700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdf340682f0 0x7fdf3406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.147+0000 7fdf4659c700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdf340682f0 0x7fdf3406a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdf38000c00 tx=0x7fdf3801e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.151+0000 7fdf2f7fe700 1 -- 172.21.15.156:0/16988834 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdf3c052460 con 0x7fdf30004850 2023-11-28T11:08:56.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.335+0000 7fdf4d056700 1 -- 172.21.15.156:0/16988834 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf4806ba50 con 0x7fdf30004850 2023-11-28T11:08:56.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.336+0000 7fdf2f7fe700 1 -- 172.21.15.156:0/16988834 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17955 (secure 0 0 0) 0x7fdf3c055a80 con 0x7fdf30004850 2023-11-28T11:08:56.338 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:56.338 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:18.354009+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:56.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.338+0000 7fdf4d056700 1 -- 172.21.15.156:0/16988834 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdf340682f0 msgr2=0x7fdf3406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.338+0000 7fdf4d056700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdf340682f0 0x7fdf3406a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdf38000c00 tx=0x7fdf3801e040 comp rx=0 tx=0).stop 2023-11-28T11:08:56.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.338+0000 7fdf4d056700 1 -- 172.21.15.156:0/16988834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 msgr2=0x7fdf4814ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.338+0000 7fdf4d056700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 0x7fdf4814ed50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00e780 tx=0x7fdf3c00eb40 comp rx=0 tx=0).stop 2023-11-28T11:08:56.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.338+0000 7fdf4d056700 1 -- 172.21.15.156:0/16988834 shutdown_connections 2023-11-28T11:08:56.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.338+0000 7fdf4d056700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdf340682f0 0x7fdf3406a7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.338+0000 7fdf4d056700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf480c8ea0 0x7fdf480c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.338+0000 7fdf4d056700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf30004850 0x7fdf4814ed50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.338+0000 7fdf4d056700 1 --2- 172.21.15.156:0/16988834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf481e4ff0 0x7fdf481e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.338+0000 7fdf4d056700 1 -- 172.21.15.156:0/16988834 >> 172.21.15.156:0/16988834 conn(0x7fdf480b12b0 msgr2=0x7fdf480b49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.338+0000 7fdf4d056700 1 -- 172.21.15.156:0/16988834 shutdown_connections 2023-11-28T11:08:56.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.338+0000 7fdf4d056700 1 -- 172.21.15.156:0/16988834 wait complete. 2023-11-28T11:08:56.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:15.021591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:16.332241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:56.352 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:56.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.475+0000 7efd9bc64700 1 Processor -- start 2023-11-28T11:08:56.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.475+0000 7efd9bc64700 1 -- start start 2023-11-28T11:08:56.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9bc64700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd94071040 0x7efd94071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9bc64700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efd94160540 0x7efd941609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9bc64700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd94071950 con 0x7efd9406dee0 2023-11-28T11:08:56.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9bc64700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9415d060 con 0x7efd94160540 2023-11-28T11:08:56.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9bc64700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9415d340 con 0x7efd94071040 2023-11-28T11:08:56.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9a201700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efd94160540 0x7efd941609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9a201700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efd94160540 0x7efd941609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33062/0 (socket says 172.21.15.156:33062) 2023-11-28T11:08:56.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9a201700 1 -- 172.21.15.156:0/3112076150 learned_addr learned my addr 172.21.15.156:0/3112076150 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:56.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd99a00700 1 --2- 172.21.15.156:0/3112076150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd94071040 0x7efd94071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9a201700 1 -- 172.21.15.156:0/3112076150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd94071040 msgr2=0x7efd94071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9a201700 1 --2- 172.21.15.156:0/3112076150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd94071040 0x7efd94071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9a201700 1 -- 172.21.15.156:0/3112076150 >> v1:172.21.15.186:6789/0 conn(0x7efd9406dee0 legacy=0x7efd9406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9a201700 1 -- 172.21.15.156:0/3112076150 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9415d620 con 0x7efd94160540 2023-11-28T11:08:56.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.476+0000 7efd9a201700 1 --2- 172.21.15.156:0/3112076150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd94160540 0x7efd941609c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009e50 tx=0x7efd8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36e706121718867 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.477+0000 7efd989fe700 1 -- 172.21.15.156:0/3112076150 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00e070 con 0x7efd94160540 2023-11-28T11:08:56.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.477+0000 7efd989fe700 1 -- 172.21.15.156:0/3112076150 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c00b9c0 con 0x7efd94160540 2023-11-28T11:08:56.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.477+0000 7efd989fe700 1 -- 172.21.15.156:0/3112076150 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00bd70 con 0x7efd94160540 2023-11-28T11:08:56.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.477+0000 7efd9bc64700 1 -- 172.21.15.156:0/3112076150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd94160540 msgr2=0x7efd941609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.477+0000 7efd9bc64700 1 --2- 172.21.15.156:0/3112076150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd94160540 0x7efd941609c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009e50 tx=0x7efd8c00b330 comp rx=0 tx=0).stop 2023-11-28T11:08:56.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.477+0000 7efd9bc64700 1 -- 172.21.15.156:0/3112076150 shutdown_connections 2023-11-28T11:08:56.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.477+0000 7efd9bc64700 1 --2- 172.21.15.156:0/3112076150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd94160540 0x7efd941609c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.477+0000 7efd9bc64700 1 --2- 172.21.15.156:0/3112076150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd94071040 0x7efd94071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.477+0000 7efd9bc64700 1 -- 172.21.15.156:0/3112076150 >> 172.21.15.156:0/3112076150 conn(0x7efd940d0000 msgr2=0x7efd940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.477+0000 7efd9bc64700 1 -- 172.21.15.156:0/3112076150 shutdown_connections 2023-11-28T11:08:56.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.477+0000 7efd9bc64700 1 -- 172.21.15.156:0/3112076150 wait complete. 2023-11-28T11:08:56.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.479+0000 7efd9bc64700 1 Processor -- start 2023-11-28T11:08:56.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.479+0000 7efd9bc64700 1 -- start start 2023-11-28T11:08:56.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.479+0000 7efd9bc64700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd9406dee0 0x7efd94152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.479+0000 7efd9bc64700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd94071040 0x7efd94159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.479+0000 7efd9bc64700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd94160540 0x7efd940c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.479+0000 7efd9bc64700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd940b4c50 con 0x7efd94071040 2023-11-28T11:08:56.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.479+0000 7efd9bc64700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd940c4d60 con 0x7efd9406dee0 2023-11-28T11:08:56.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.479+0000 7efd9bc64700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd941530e0 con 0x7efd94160540 2023-11-28T11:08:56.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.479+0000 7efd991ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd94071040 0x7efd94159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.479+0000 7efd991ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd94071040 0x7efd94159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33068/0 (socket says 172.21.15.156:33068) 2023-11-28T11:08:56.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd991ff700 1 -- 172.21.15.156:0/988219011 learned_addr learned my addr 172.21.15.156:0/988219011 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:56.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd99a00700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd9406dee0 0x7efd94152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd991ff700 1 -- 172.21.15.156:0/988219011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd94160540 msgr2=0x7efd940c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd991ff700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd94160540 0x7efd940c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd991ff700 1 -- 172.21.15.156:0/988219011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd9406dee0 msgr2=0x7efd94152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd991ff700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd9406dee0 0x7efd94152ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd991ff700 1 -- 172.21.15.156:0/988219011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c009b30 con 0x7efd94071040 2023-11-28T11:08:56.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd991ff700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd94071040 0x7efd94159100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd8400c480 tx=0x7efd8400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd8a7fc700 1 -- 172.21.15.156:0/988219011 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd840052d0 con 0x7efd94071040 2023-11-28T11:08:56.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd8a7fc700 1 -- 172.21.15.156:0/988219011 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd84005430 con 0x7efd94071040 2023-11-28T11:08:56.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd9bc64700 1 -- 172.21.15.156:0/988219011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9415ccd0 con 0x7efd94071040 2023-11-28T11:08:56.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd8a7fc700 1 -- 172.21.15.156:0/988219011 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd840038c0 con 0x7efd94071040 2023-11-28T11:08:56.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.480+0000 7efd9bc64700 1 -- 172.21.15.156:0/988219011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9415d900 con 0x7efd94071040 2023-11-28T11:08:56.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.481+0000 7efd9bc64700 1 -- 172.21.15.156:0/988219011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9406a270 con 0x7efd94071040 2023-11-28T11:08:56.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.481+0000 7efd8a7fc700 1 -- 172.21.15.156:0/988219011 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efd84019030 con 0x7efd94071040 2023-11-28T11:08:56.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.482+0000 7efd8a7fc700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7efd780682f0 0x7efd7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.482+0000 7efd99a00700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7efd780682f0 0x7efd7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.482+0000 7efd8a7fc700 1 -- 172.21.15.156:0/988219011 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efd84084fa0 con 0x7efd94071040 2023-11-28T11:08:56.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.482+0000 7efd99a00700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7efd780682f0 0x7efd7806a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd941585f0 tx=0x7efd90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.486+0000 7efd8a7fc700 1 -- 172.21.15.156:0/988219011 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efd84051800 con 0x7efd94071040 2023-11-28T11:08:56.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.672+0000 7efd9bc64700 1 -- 172.21.15.156:0/988219011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd940b4e00 con 0x7efd94071040 2023-11-28T11:08:56.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.672+0000 7efd8a7fc700 1 -- 172.21.15.156:0/988219011 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd940b4e00 con 0x7efd94071040 2023-11-28T11:08:56.674 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.674+0000 7efd9bc64700 1 -- 172.21.15.156:0/988219011 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7efd780682f0 msgr2=0x7efd7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.674+0000 7efd9bc64700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7efd780682f0 0x7efd7806a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd941585f0 tx=0x7efd90008040 comp rx=0 tx=0).stop 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.674+0000 7efd9bc64700 1 -- 172.21.15.156:0/988219011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd94071040 msgr2=0x7efd94159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.674+0000 7efd9bc64700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd94071040 0x7efd94159100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd8400c480 tx=0x7efd8400c840 comp rx=0 tx=0).stop 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.674+0000 7efd9bc64700 1 -- 172.21.15.156:0/988219011 shutdown_connections 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.675+0000 7efd9bc64700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7efd780682f0 0x7efd7806a7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.675+0000 7efd9bc64700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd94160540 0x7efd940c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.675+0000 7efd9bc64700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd9406dee0 0x7efd94152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.675+0000 7efd9bc64700 1 --2- 172.21.15.156:0/988219011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd94071040 0x7efd94159100 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.675+0000 7efd9bc64700 1 -- 172.21.15.156:0/988219011 >> 172.21.15.156:0/988219011 conn(0x7efd940d0000 msgr2=0x7efd9415f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.675+0000 7efd9bc64700 1 -- 172.21.15.156:0/988219011 shutdown_connections 2023-11-28T11:08:56.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.675+0000 7efd9bc64700 1 -- 172.21.15.156:0/988219011 wait complete. 2023-11-28T11:08:56.690 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:56.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.814+0000 7f20f1227700 1 Processor -- start 2023-11-28T11:08:56.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.814+0000 7f20f1227700 1 -- start start 2023-11-28T11:08:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.815+0000 7f20f1227700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f20ec071040 0x7f20ec071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.815+0000 7f20f1227700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec160540 0x7f20ec1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.815+0000 7f20f1227700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ec071950 con 0x7f20ec06dee0 2023-11-28T11:08:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.815+0000 7f20f1227700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec15d060 con 0x7f20ec071040 2023-11-28T11:08:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.815+0000 7f20f1227700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec15d340 con 0x7f20ec160540 2023-11-28T11:08:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.815+0000 7f20ea59c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f20ec06dee0 0x7f20ec06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:47950/0 (socket says 172.21.15.156:47950) 2023-11-28T11:08:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.815+0000 7f20ea59c700 1 -- 172.21.15.156:0/3427955618 learned_addr learned my addr 172.21.15.156:0/3427955618 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.815+0000 7f20ead9d700 1 --2- 172.21.15.156:0/3427955618 >> v2:172.21.15.186:3300/0 conn(0x7f20ec071040 0x7f20ec071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.815+0000 7f20eb59e700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec160540 0x7f20ec1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.815+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895644594 0 0) 0x7f20ec071950 con 0x7f20ec06dee0 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.815+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20d40035b0 con 0x7f20ec06dee0 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2365571759 0 0) 0x7f20d40035b0 con 0x7f20ec06dee0 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec160540 msgr2=0x7f20ec1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec160540 0x7f20ec1609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 >> v2:172.21.15.186:3300/0 conn(0x7f20ec071040 msgr2=0x7f20ec071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 --2- 172.21.15.156:0/3427955618 >> v2:172.21.15.186:3300/0 conn(0x7f20ec071040 0x7f20ec071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec15d620 con 0x7f20ec06dee0 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f20dc001670 con 0x7f20ec06dee0 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20ead9d700 1 --2- 172.21.15.156:0/3427955618 >> v2:172.21.15.186:3300/0 conn(0x7f20ec071040 0x7f20ec071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20eb59e700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec160540 0x7f20ec1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 >> v1:172.21.15.156:6789/0 conn(0x7f20ec06dee0 legacy=0x7f20ec06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec160540 0x7f20d4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4003f40 0x7f20d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20d4004d90 0x7f20d4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec15d060 con 0x7f20d4003f40 2023-11-28T11:08:56.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec15d340 con 0x7f20d4004d90 2023-11-28T11:08:56.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d400ba90 con 0x7f20ec160540 2023-11-28T11:08:56.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20ea59c700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4003f40 0x7f20d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20eb59e700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20d4004d90 0x7f20d4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20ead9d700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec160540 0x7f20d4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20ea59c700 1 -- 172.21.15.156:0/3427955618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec160540 msgr2=0x7f20d4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20ea59c700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec160540 0x7f20d4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20ea59c700 1 -- 172.21.15.156:0/3427955618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20d4004d90 msgr2=0x7f20d4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20ea59c700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20d4004d90 0x7f20d4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20ea59c700 1 -- 172.21.15.156:0/3427955618 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4009a70 con 0x7f20d4003f40 2023-11-28T11:08:56.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20ea59c700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4003f40 0x7f20d4004850 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00c320 tx=0x7f20d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bee42a2df31e80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00ce60 con 0x7f20d4003f40 2023-11-28T11:08:56.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc005960 con 0x7f20d4003f40 2023-11-28T11:08:56.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.816+0000 7f20e9d9b700 1 -- 172.21.15.156:0/3427955618 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc005cf0 con 0x7f20d4003f40 2023-11-28T11:08:56.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.817+0000 7f20f1227700 1 -- 172.21.15.156:0/3427955618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4003f40 msgr2=0x7f20d4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.817+0000 7f20f1227700 1 --2- 172.21.15.156:0/3427955618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4003f40 0x7f20d4004850 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00c320 tx=0x7f20d4005880 comp rx=0 tx=0).stop 2023-11-28T11:08:56.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.817+0000 7f20eb59e700 1 -- 172.21.15.156:0/3427955618 reap_dead start 2023-11-28T11:08:56.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.817+0000 7f20f1227700 1 -- 172.21.15.156:0/3427955618 shutdown_connections 2023-11-28T11:08:56.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.817+0000 7f20f1227700 1 -- 172.21.15.156:0/3427955618 >> 172.21.15.156:0/3427955618 conn(0x7f20ec0d0000 msgr2=0x7f20ec0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.817+0000 7f20f1227700 1 -- 172.21.15.156:0/3427955618 shutdown_connections 2023-11-28T11:08:56.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.817+0000 7f20f1227700 1 -- 172.21.15.156:0/3427955618 wait complete. 2023-11-28T11:08:56.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.818+0000 7f20f1227700 1 Processor -- start 2023-11-28T11:08:56.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.819+0000 7f20f1227700 1 -- start start 2023-11-28T11:08:56.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.819+0000 7f20f1227700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4003f40 0x7f20ec1535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.819+0000 7f20f1227700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ec1f65b0 0x7f20ec0c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.819+0000 7f20f1227700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec1e67d0 0x7f20ec1e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.819+0000 7f20f1227700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1e71b0 con 0x7f20d4003f40 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.819+0000 7f20f1227700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec153b30 con 0x7f20ec1f65b0 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.819+0000 7f20f1227700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec0b0920 con 0x7f20ec1e67d0 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.819+0000 7f20ead9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4003f40 0x7f20ec1535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.819+0000 7f20ead9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4003f40 0x7f20ec1535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33084/0 (socket says 172.21.15.156:33084) 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.819+0000 7f20ead9d700 1 -- 172.21.15.156:0/3646327589 learned_addr learned my addr 172.21.15.156:0/3646327589 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20ea59c700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ec1f65b0 0x7f20ec0c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20eb59e700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec1e67d0 0x7f20ec1e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20ead9d700 1 -- 172.21.15.156:0/3646327589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec1e67d0 msgr2=0x7f20ec1e6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20ead9d700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec1e67d0 0x7f20ec1e6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20ead9d700 1 -- 172.21.15.156:0/3646327589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ec1f65b0 msgr2=0x7f20ec0c3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20ead9d700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ec1f65b0 0x7f20ec0c3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20ead9d700 1 -- 172.21.15.156:0/3646327589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec15ccd0 con 0x7f20d4003f40 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20ead9d700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4003f40 0x7f20ec1535f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f20e0011d00 tx=0x7f20e000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20d37fe700 1 -- 172.21.15.156:0/3646327589 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000fa80 con 0x7f20d4003f40 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20d37fe700 1 -- 172.21.15.156:0/3646327589 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e000fbe0 con 0x7f20d4003f40 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20d37fe700 1 -- 172.21.15.156:0/3646327589 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0004020 con 0x7f20d4003f40 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20f1227700 1 -- 172.21.15.156:0/3646327589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec15d900 con 0x7f20d4003f40 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.820+0000 7f20f1227700 1 -- 172.21.15.156:0/3646327589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec06a270 con 0x7f20d4003f40 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.821+0000 7f20f1227700 1 -- 172.21.15.156:0/3646327589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec06f4a0 con 0x7f20d4003f40 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.821+0000 7f20d37fe700 1 -- 172.21.15.156:0/3646327589 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f20e0024070 con 0x7f20d4003f40 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.822+0000 7f20d37fe700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20d80682f0 0x7f20d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.822+0000 7f20d37fe700 1 -- 172.21.15.156:0/3646327589 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f20e0086110 con 0x7f20d4003f40 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.822+0000 7f20ea59c700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20d80682f0 0x7f20d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.822+0000 7f20ea59c700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20d80682f0 0x7f20d806a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20dc000c00 tx=0x7f20dc00a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:56.826+0000 7f20d37fe700 1 -- 172.21.15.156:0/3646327589 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f20e0052970 con 0x7f20d4003f40 2023-11-28T11:08:57.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.011+0000 7f20f1227700 1 -- 172.21.15.156:0/3646327589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f20ec153cd0 con 0x7f20d4003f40 2023-11-28T11:08:57.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.011+0000 7f20d37fe700 1 -- 172.21.15.156:0/3646327589 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f20e00220a0 con 0x7f20d4003f40 2023-11-28T11:08:57.012 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:08:57.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.013+0000 7f20f1227700 1 -- 172.21.15.156:0/3646327589 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20d80682f0 msgr2=0x7f20d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.013+0000 7f20f1227700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20d80682f0 0x7f20d806a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20dc000c00 tx=0x7f20dc00a000 comp rx=0 tx=0).stop 2023-11-28T11:08:57.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.013+0000 7f20f1227700 1 -- 172.21.15.156:0/3646327589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4003f40 msgr2=0x7f20ec1535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.013+0000 7f20f1227700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4003f40 0x7f20ec1535f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f20e0011d00 tx=0x7f20e000f520 comp rx=0 tx=0).stop 2023-11-28T11:08:57.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.013+0000 7f20f1227700 1 -- 172.21.15.156:0/3646327589 shutdown_connections 2023-11-28T11:08:57.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.013+0000 7f20f1227700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20d80682f0 0x7f20d806a7a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.013+0000 7f20f1227700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ec1e67d0 0x7f20ec1e6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.013+0000 7f20f1227700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4003f40 0x7f20ec1535f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.013+0000 7f20f1227700 1 --2- 172.21.15.156:0/3646327589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ec1f65b0 0x7f20ec0c3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.013+0000 7f20f1227700 1 -- 172.21.15.156:0/3646327589 >> 172.21.15.156:0/3646327589 conn(0x7f20ec0d0000 msgr2=0x7f20ec0cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.013+0000 7f20f1227700 1 -- 172.21.15.156:0/3646327589 shutdown_connections 2023-11-28T11:08:57.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.013+0000 7f20f1227700 1 -- 172.21.15.156:0/3646327589 wait complete. 2023-11-28T11:08:57.027 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:57.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.150+0000 7f30b6691700 1 Processor -- start 2023-11-28T11:08:57.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.150+0000 7f30b6691700 1 -- start start 2023-11-28T11:08:57.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.150+0000 7f30b6691700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30b00cd8c0 0x7f30b0140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.150+0000 7f30b6691700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f30b00cdc90 0x7f30b00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.150+0000 7f30b6691700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b013ff00 con 0x7f30b00b2e70 2023-11-28T11:08:57.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.150+0000 7f30b6691700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b01404c0 con 0x7f30b00cdc90 2023-11-28T11:08:57.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.150+0000 7f30b6691700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b015aa80 con 0x7f30b00cd8c0 2023-11-28T11:08:57.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.151+0000 7f30af7fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f30b00cdc90 0x7f30b00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.151+0000 7f30affff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30b00cd8c0 0x7f30b0140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.151+0000 7f30affff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30b00cd8c0 0x7f30b0140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36956/0 (socket says 172.21.15.156:36956) 2023-11-28T11:08:57.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.151+0000 7f30affff700 1 -- 172.21.15.156:0/1687115062 learned_addr learned my addr 172.21.15.156:0/1687115062 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:57.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.151+0000 7f30affff700 1 -- 172.21.15.156:0/1687115062 >> v2:172.21.15.186:3300/0 conn(0x7f30b00cdc90 msgr2=0x7f30b00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:57.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.151+0000 7f30affff700 1 --2- 172.21.15.156:0/1687115062 >> v2:172.21.15.186:3300/0 conn(0x7f30b00cdc90 0x7f30b00b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.151+0000 7f30affff700 1 -- 172.21.15.156:0/1687115062 >> v1:172.21.15.186:6789/0 conn(0x7f30b00b2e70 legacy=0x7f30b006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.151+0000 7f30affff700 1 -- 172.21.15.156:0/1687115062 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b00b3310 con 0x7f30b00cd8c0 2023-11-28T11:08:57.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.151+0000 7f30af7fe700 1 --2- 172.21.15.156:0/1687115062 >> v2:172.21.15.186:3300/0 conn(0x7f30b00cdc90 0x7f30b00b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:57.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.151+0000 7f30affff700 1 --2- 172.21.15.156:0/1687115062 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30b00cd8c0 0x7f30b0140950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f30a0006fd0 tx=0x7f30a0009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b800faf4b0b6b3c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.152+0000 7f30aeffd700 1 -- 172.21.15.156:0/1687115062 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0005e30 con 0x7f30b00cd8c0 2023-11-28T11:08:57.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.152+0000 7f30aeffd700 1 -- 172.21.15.156:0/1687115062 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a0004030 con 0x7f30b00cd8c0 2023-11-28T11:08:57.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.152+0000 7f30aeffd700 1 -- 172.21.15.156:0/1687115062 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a00043e0 con 0x7f30b00cd8c0 2023-11-28T11:08:57.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.152+0000 7f30b6691700 1 -- 172.21.15.156:0/1687115062 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30b00cd8c0 msgr2=0x7f30b0140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.152+0000 7f30b6691700 1 --2- 172.21.15.156:0/1687115062 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30b00cd8c0 0x7f30b0140950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f30a0006fd0 tx=0x7f30a0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.152+0000 7f30b6691700 1 -- 172.21.15.156:0/1687115062 shutdown_connections 2023-11-28T11:08:57.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.152+0000 7f30b6691700 1 --2- 172.21.15.156:0/1687115062 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30b00cd8c0 0x7f30b0140950 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.152+0000 7f30b6691700 1 --2- 172.21.15.156:0/1687115062 >> v2:172.21.15.186:3300/0 conn(0x7f30b00cdc90 0x7f30b00b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.152+0000 7f30b6691700 1 -- 172.21.15.156:0/1687115062 >> 172.21.15.156:0/1687115062 conn(0x7f30b006aed0 msgr2=0x7f30b006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.152+0000 7f30b6691700 1 -- 172.21.15.156:0/1687115062 shutdown_connections 2023-11-28T11:08:57.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.152+0000 7f30b6691700 1 -- 172.21.15.156:0/1687115062 wait complete. 2023-11-28T11:08:57.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.153+0000 7f30b6691700 1 Processor -- start 2023-11-28T11:08:57.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.154+0000 7f30b6691700 1 -- start start 2023-11-28T11:08:57.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.154+0000 7f30b6691700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30b00b2e70 0x7f30b015d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.154+0000 7f30b6691700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b00cd8c0 0x7f30b00c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.154+0000 7f30b6691700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30b00cdc90 0x7f30b0154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.154+0000 7f30b6691700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b015c650 con 0x7f30b00cd8c0 2023-11-28T11:08:57.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.154+0000 7f30b6691700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00c7880 con 0x7f30b00cdc90 2023-11-28T11:08:57.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.154+0000 7f30b6691700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00b6be0 con 0x7f30b00b2e70 2023-11-28T11:08:57.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30af7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b00cd8c0 0x7f30b00c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30af7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b00cd8c0 0x7f30b00c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33098/0 (socket says 172.21.15.156:33098) 2023-11-28T11:08:57.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30af7fe700 1 -- 172.21.15.156:0/1055258044 learned_addr learned my addr 172.21.15.156:0/1055258044 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:57.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30affff700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30b00b2e70 0x7f30b015d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30b4c2e700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30b00cdc90 0x7f30b0154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30af7fe700 1 -- 172.21.15.156:0/1055258044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30b00b2e70 msgr2=0x7f30b015d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30af7fe700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30b00b2e70 0x7f30b015d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30af7fe700 1 -- 172.21.15.156:0/1055258044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30b00cdc90 msgr2=0x7f30b0154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30af7fe700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30b00cdc90 0x7f30b0154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30af7fe700 1 -- 172.21.15.156:0/1055258044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30980094a0 con 0x7f30b00cd8c0 2023-11-28T11:08:57.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30af7fe700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b00cd8c0 0x7f30b00c7180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f309800ebb0 tx=0x7f309800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30acff9700 1 -- 172.21.15.156:0/1055258044 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309800c850 con 0x7f30b00cd8c0 2023-11-28T11:08:57.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30acff9700 1 -- 172.21.15.156:0/1055258044 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309800c9b0 con 0x7f30b00cd8c0 2023-11-28T11:08:57.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30b6691700 1 -- 172.21.15.156:0/1055258044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a0009500 con 0x7f30b00cd8c0 2023-11-28T11:08:57.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30acff9700 1 -- 172.21.15.156:0/1055258044 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309800cd40 con 0x7f30b00cd8c0 2023-11-28T11:08:57.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.155+0000 7f30b6691700 1 -- 172.21.15.156:0/1055258044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b0140150 con 0x7f30b00cd8c0 2023-11-28T11:08:57.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.156+0000 7f30b6691700 1 -- 172.21.15.156:0/1055258044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b00b35f0 con 0x7f30b00cd8c0 2023-11-28T11:08:57.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.157+0000 7f30acff9700 1 -- 172.21.15.156:0/1055258044 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3098019030 con 0x7f30b00cd8c0 2023-11-28T11:08:57.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.157+0000 7f30acff9700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3090068320 0x7f309006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.157+0000 7f30acff9700 1 -- 172.21.15.156:0/1055258044 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3098085b20 con 0x7f30b00cd8c0 2023-11-28T11:08:57.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.157+0000 7f30affff700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3090068320 0x7f309006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.158+0000 7f30affff700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3090068320 0x7f309006a7d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f30a0006fd0 tx=0x7f30a0008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.161+0000 7f30acff9700 1 -- 172.21.15.156:0/1055258044 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3098052380 con 0x7f30b00cd8c0 2023-11-28T11:08:57.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.343+0000 7f30b6691700 1 -- 172.21.15.156:0/1055258044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30b00ce100 con 0x7f30b00cd8c0 2023-11-28T11:08:57.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.343+0000 7f30acff9700 1 -- 172.21.15.156:0/1055258044 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3098017020 con 0x7f30b00cd8c0 2023-11-28T11:08:57.345 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:08:57.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.345+0000 7f30b6691700 1 -- 172.21.15.156:0/1055258044 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3090068320 msgr2=0x7f309006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.345+0000 7f30b6691700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3090068320 0x7f309006a7d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f30a0006fd0 tx=0x7f30a0008070 comp rx=0 tx=0).stop 2023-11-28T11:08:57.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.345+0000 7f30b6691700 1 -- 172.21.15.156:0/1055258044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b00cd8c0 msgr2=0x7f30b00c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.345+0000 7f30b6691700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b00cd8c0 0x7f30b00c7180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f309800ebb0 tx=0x7f309800eec0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.345+0000 7f30b6691700 1 -- 172.21.15.156:0/1055258044 shutdown_connections 2023-11-28T11:08:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.346+0000 7f30b6691700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30b00cdc90 0x7f30b0154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.346+0000 7f30b6691700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3090068320 0x7f309006a7d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.346+0000 7f30b6691700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30b00b2e70 0x7f30b015d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.346+0000 7f30b6691700 1 --2- 172.21.15.156:0/1055258044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b00cd8c0 0x7f30b00c7180 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.346+0000 7f30b6691700 1 -- 172.21.15.156:0/1055258044 >> 172.21.15.156:0/1055258044 conn(0x7f30b006aed0 msgr2=0x7f30b013dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.346+0000 7f30b6691700 1 -- 172.21.15.156:0/1055258044 shutdown_connections 2023-11-28T11:08:57.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.346+0000 7f30b6691700 1 -- 172.21.15.156:0/1055258044 wait complete. 2023-11-28T11:08:57.359 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:57.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.482+0000 7ff990b8f700 1 Processor -- start 2023-11-28T11:08:57.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.482+0000 7ff990b8f700 1 -- start start 2023-11-28T11:08:57.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.483+0000 7ff990b8f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff98c06dee0 0x7ff98c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.483+0000 7ff990b8f700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff98c160540 0x7ff98c160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.483+0000 7ff990b8f700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98c15d060 con 0x7ff98c071040 2023-11-28T11:08:57.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.483+0000 7ff990b8f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98c15d340 con 0x7ff98c06dee0 2023-11-28T11:08:57.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.483+0000 7ff990b8f700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98c15d620 con 0x7ff98c160540 2023-11-28T11:08:57.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.484+0000 7ff989d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff98c06dee0 0x7ff98c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.484+0000 7ff989d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff98c06dee0 0x7ff98c06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33106/0 (socket says 172.21.15.156:33106) 2023-11-28T11:08:57.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.484+0000 7ff989d9b700 1 -- 172.21.15.156:0/3083404509 learned_addr learned my addr 172.21.15.156:0/3083404509 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:57.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.484+0000 7ff989d9b700 1 -- 172.21.15.156:0/3083404509 >> v2:172.21.15.186:3300/0 conn(0x7ff98c160540 msgr2=0x7ff98c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:57.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.484+0000 7ff98ad9d700 1 --2- 172.21.15.156:0/3083404509 >> v2:172.21.15.186:3300/0 conn(0x7ff98c160540 0x7ff98c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.484+0000 7ff989d9b700 1 --2- 172.21.15.156:0/3083404509 >> v2:172.21.15.186:3300/0 conn(0x7ff98c160540 0x7ff98c160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.484+0000 7ff989d9b700 1 -- 172.21.15.156:0/3083404509 >> v1:172.21.15.186:6789/0 conn(0x7ff98c071040 legacy=0x7ff98c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:57.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.484+0000 7ff989d9b700 1 -- 172.21.15.156:0/3083404509 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c15d900 con 0x7ff98c06dee0 2023-11-28T11:08:57.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.484+0000 7ff989d9b700 1 --2- 172.21.15.156:0/3083404509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98c06dee0 0x7ff98c06e330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0c3a10 tx=0x7ff97c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4283636075d62492 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.484+0000 7ff98959a700 1 -- 172.21.15.156:0/3083404509 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c004230 con 0x7ff98c06dee0 2023-11-28T11:08:57.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.484+0000 7ff98959a700 1 -- 172.21.15.156:0/3083404509 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97c004390 con 0x7ff98c06dee0 2023-11-28T11:08:57.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.484+0000 7ff98959a700 1 -- 172.21.15.156:0/3083404509 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c0036a0 con 0x7ff98c06dee0 2023-11-28T11:08:57.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.485+0000 7ff990b8f700 1 -- 172.21.15.156:0/3083404509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98c06dee0 msgr2=0x7ff98c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.485+0000 7ff990b8f700 1 --2- 172.21.15.156:0/3083404509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98c06dee0 0x7ff98c06e330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0c3a10 tx=0x7ff97c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.485+0000 7ff990b8f700 1 -- 172.21.15.156:0/3083404509 shutdown_connections 2023-11-28T11:08:57.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.485+0000 7ff990b8f700 1 --2- 172.21.15.156:0/3083404509 >> v2:172.21.15.186:3300/0 conn(0x7ff98c160540 0x7ff98c160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.485+0000 7ff990b8f700 1 --2- 172.21.15.156:0/3083404509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98c06dee0 0x7ff98c06e330 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.485+0000 7ff990b8f700 1 -- 172.21.15.156:0/3083404509 >> 172.21.15.156:0/3083404509 conn(0x7ff98c0d0000 msgr2=0x7ff98c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.485+0000 7ff990b8f700 1 -- 172.21.15.156:0/3083404509 shutdown_connections 2023-11-28T11:08:57.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.485+0000 7ff990b8f700 1 -- 172.21.15.156:0/3083404509 wait complete. 2023-11-28T11:08:57.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.486+0000 7ff990b8f700 1 Processor -- start 2023-11-28T11:08:57.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.486+0000 7ff990b8f700 1 -- start start 2023-11-28T11:08:57.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff990b8f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98c06dee0 0x7ff98c14cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff990b8f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff98c071040 0x7ff98c0c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff990b8f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff98c160540 0x7ff98c0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff990b8f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c0b3130 con 0x7ff98c06dee0 2023-11-28T11:08:57.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff990b8f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c1f5ed0 con 0x7ff98c071040 2023-11-28T11:08:57.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff990b8f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c1f61b0 con 0x7ff98c160540 2023-11-28T11:08:57.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff98a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98c06dee0 0x7ff98c14cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff98a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98c06dee0 0x7ff98c14cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33110/0 (socket says 172.21.15.156:33110) 2023-11-28T11:08:57.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff98a59c700 1 -- 172.21.15.156:0/4229152485 learned_addr learned my addr 172.21.15.156:0/4229152485 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:57.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff989d9b700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff98c071040 0x7ff98c0c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff98ad9d700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff98c160540 0x7ff98c0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff98a59c700 1 -- 172.21.15.156:0/4229152485 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff98c160540 msgr2=0x7ff98c0b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff98a59c700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff98c160540 0x7ff98c0b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff98a59c700 1 -- 172.21.15.156:0/4229152485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff98c071040 msgr2=0x7ff98c0c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff98a59c700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff98c071040 0x7ff98c0c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff98a59c700 1 -- 172.21.15.156:0/4229152485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c009550 con 0x7ff98c06dee0 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.487+0000 7ff98ad9d700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff98c160540 0x7ff98c0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.488+0000 7ff98a59c700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98c06dee0 0x7ff98c14cd60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff97400b470 tx=0x7ff97400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.488+0000 7ff982ffd700 1 -- 172.21.15.156:0/4229152485 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974011830 con 0x7ff98c06dee0 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.488+0000 7ff982ffd700 1 -- 172.21.15.156:0/4229152485 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9740119b0 con 0x7ff98c06dee0 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.488+0000 7ff990b8f700 1 -- 172.21.15.156:0/4229152485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98c15ccd0 con 0x7ff98c06dee0 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.488+0000 7ff982ffd700 1 -- 172.21.15.156:0/4229152485 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974011d60 con 0x7ff98c06dee0 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.488+0000 7ff990b8f700 1 -- 172.21.15.156:0/4229152485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98c06e870 con 0x7ff98c06dee0 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.489+0000 7ff990b8f700 1 -- 172.21.15.156:0/4229152485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98c06a270 con 0x7ff98c06dee0 2023-11-28T11:08:57.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.489+0000 7ff982ffd700 1 -- 172.21.15.156:0/4229152485 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff974021070 con 0x7ff98c06dee0 2023-11-28T11:08:57.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.489+0000 7ff982ffd700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff96c0682f0 0x7ff96c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.490+0000 7ff982ffd700 1 -- 172.21.15.156:0/4229152485 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff974085af0 con 0x7ff98c06dee0 2023-11-28T11:08:57.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.490+0000 7ff989d9b700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff96c0682f0 0x7ff96c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.490+0000 7ff989d9b700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff96c0682f0 0x7ff96c06a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009130 tx=0x7ff97c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.494+0000 7ff982ffd700 1 -- 172.21.15.156:0/4229152485 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff974052350 con 0x7ff98c06dee0 2023-11-28T11:08:57.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.676+0000 7ff990b8f700 1 -- 172.21.15.156:0/4229152485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff98c0720a0 con 0x7ff98c06dee0 2023-11-28T11:08:57.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.676+0000 7ff982ffd700 1 -- 172.21.15.156:0/4229152485 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff9740150a0 con 0x7ff98c06dee0 2023-11-28T11:08:57.678 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:08:57.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.678+0000 7ff990b8f700 1 -- 172.21.15.156:0/4229152485 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff96c0682f0 msgr2=0x7ff96c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.678+0000 7ff990b8f700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff96c0682f0 0x7ff96c06a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009130 tx=0x7ff97c008030 comp rx=0 tx=0).stop 2023-11-28T11:08:57.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.678+0000 7ff990b8f700 1 -- 172.21.15.156:0/4229152485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98c06dee0 msgr2=0x7ff98c14cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.678+0000 7ff990b8f700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98c06dee0 0x7ff98c14cd60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff97400b470 tx=0x7ff97400b780 comp rx=0 tx=0).stop 2023-11-28T11:08:57.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.678+0000 7ff990b8f700 1 -- 172.21.15.156:0/4229152485 shutdown_connections 2023-11-28T11:08:57.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.678+0000 7ff990b8f700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff96c0682f0 0x7ff96c06a7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.678+0000 7ff990b8f700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff98c160540 0x7ff98c0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.679+0000 7ff990b8f700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98c06dee0 0x7ff98c14cd60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.679+0000 7ff990b8f700 1 --2- 172.21.15.156:0/4229152485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff98c071040 0x7ff98c0c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.679+0000 7ff990b8f700 1 -- 172.21.15.156:0/4229152485 >> 172.21.15.156:0/4229152485 conn(0x7ff98c0d0000 msgr2=0x7ff98c15f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.679+0000 7ff990b8f700 1 -- 172.21.15.156:0/4229152485 shutdown_connections 2023-11-28T11:08:57.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.679+0000 7ff990b8f700 1 -- 172.21.15.156:0/4229152485 wait complete. 2023-11-28T11:08:57.692 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:08:57.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.812+0000 7f2ba0cae700 1 Processor -- start 2023-11-28T11:08:57.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.812+0000 7f2ba0cae700 1 -- start start 2023-11-28T11:08:57.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.813+0000 7f2ba0cae700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2b9c06dee0 0x7f2b9c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.813+0000 7f2ba0cae700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c15d060 con 0x7f2b9c160540 2023-11-28T11:08:57.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.813+0000 7f2ba0cae700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c15d340 con 0x7f2b9c071040 2023-11-28T11:08:57.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.813+0000 7f2ba0cae700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9c15d620 con 0x7f2b9c06dee0 2023-11-28T11:08:57.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.813+0000 7f2b99d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2b9c06dee0 0x7f2b9c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.813+0000 7f2b9ad9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2b9c160540 0x7f2b9c160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48000/0 (socket says 172.21.15.156:48000) 2023-11-28T11:08:57.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.813+0000 7f2b9ad9d700 1 -- 172.21.15.156:0/1046973348 learned_addr learned my addr 172.21.15.156:0/1046973348 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:57.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.813+0000 7f2b99d9b700 1 -- 172.21.15.156:0/1046973348 >> v1:172.21.15.186:6789/0 conn(0x7f2b9c071040 legacy=0x7f2b9c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:57.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.813+0000 7f2b99d9b700 1 -- 172.21.15.156:0/1046973348 >> v1:172.21.15.156:6789/0 conn(0x7f2b9c160540 legacy=0x7f2b9c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.813+0000 7f2b9959a700 1 -- 172.21.15.156:0/1046973348 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058935558 0 0) 0x7f2b9c15d060 con 0x7f2b9c160540 2023-11-28T11:08:57.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.813+0000 7f2b99d9b700 1 -- 172.21.15.156:0/1046973348 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c15d900 con 0x7f2b9c06dee0 2023-11-28T11:08:57.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.814+0000 7f2b99d9b700 1 --2- 172.21.15.156:0/1046973348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c06dee0 0x7f2b9c06e330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009840 tx=0x7f2b8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c29e8dd16470712 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.814+0000 7f2b9959a700 1 -- 172.21.15.156:0/1046973348 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c004230 con 0x7f2b9c06dee0 2023-11-28T11:08:57.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.814+0000 7f2b9959a700 1 -- 172.21.15.156:0/1046973348 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c004390 con 0x7f2b9c06dee0 2023-11-28T11:08:57.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.814+0000 7f2b9959a700 1 -- 172.21.15.156:0/1046973348 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c0036a0 con 0x7f2b9c06dee0 2023-11-28T11:08:57.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.814+0000 7f2ba0cae700 1 -- 172.21.15.156:0/1046973348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c06dee0 msgr2=0x7f2b9c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.814+0000 7f2ba0cae700 1 --2- 172.21.15.156:0/1046973348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c06dee0 0x7f2b9c06e330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009840 tx=0x7f2b8c009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:57.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.814+0000 7f2ba0cae700 1 -- 172.21.15.156:0/1046973348 shutdown_connections 2023-11-28T11:08:57.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.814+0000 7f2ba0cae700 1 --2- 172.21.15.156:0/1046973348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c06dee0 0x7f2b9c06e330 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.814+0000 7f2ba0cae700 1 -- 172.21.15.156:0/1046973348 >> 172.21.15.156:0/1046973348 conn(0x7f2b9c0d0000 msgr2=0x7f2b9c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.814+0000 7f2ba0cae700 1 -- 172.21.15.156:0/1046973348 shutdown_connections 2023-11-28T11:08:57.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.814+0000 7f2ba0cae700 1 -- 172.21.15.156:0/1046973348 wait complete. 2023-11-28T11:08:57.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.816+0000 7f2ba0cae700 1 Processor -- start 2023-11-28T11:08:57.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.816+0000 7f2ba0cae700 1 -- start start 2023-11-28T11:08:57.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.817+0000 7f2ba0cae700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b9c06dee0 0x7f2b9c0b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.817+0000 7f2ba0cae700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b9c071040 0x7f2b9c0ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.817+0000 7f2ba0cae700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c160540 0x7f2b9c0c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.817+0000 7f2ba0cae700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0ae760 con 0x7f2b9c160540 2023-11-28T11:08:57.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.817+0000 7f2ba0cae700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0b0030 con 0x7f2b9c071040 2023-11-28T11:08:57.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.817+0000 7f2ba0cae700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0b0310 con 0x7f2b9c06dee0 2023-11-28T11:08:57.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.817+0000 7f2b9ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c160540 0x7f2b9c0c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.817+0000 7f2b9ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c160540 0x7f2b9c0c4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33128/0 (socket says 172.21.15.156:33128) 2023-11-28T11:08:57.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.817+0000 7f2b9ad9d700 1 -- 172.21.15.156:0/2983068306 learned_addr learned my addr 172.21.15.156:0/2983068306 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:57.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.817+0000 7f2b99d9b700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b9c071040 0x7f2b9c0ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.817+0000 7f2b9a59c700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b9c06dee0 0x7f2b9c0b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.818+0000 7f2b9ad9d700 1 -- 172.21.15.156:0/2983068306 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b9c06dee0 msgr2=0x7f2b9c0b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.818+0000 7f2b9ad9d700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b9c06dee0 0x7f2b9c0b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.818+0000 7f2b9ad9d700 1 -- 172.21.15.156:0/2983068306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b9c071040 msgr2=0x7f2b9c0ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.818+0000 7f2b9ad9d700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b9c071040 0x7f2b9c0ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.818+0000 7f2b9ad9d700 1 -- 172.21.15.156:0/2983068306 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c0094a0 con 0x7f2b9c160540 2023-11-28T11:08:57.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.818+0000 7f2b99d9b700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b9c071040 0x7f2b9c0ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:57.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.818+0000 7f2b9ad9d700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c160540 0x7f2b9c0c4970 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400cba0 tx=0x7f2b9400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.818+0000 7f2b8affd700 1 -- 172.21.15.156:0/2983068306 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9400ab80 con 0x7f2b9c160540 2023-11-28T11:08:57.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.818+0000 7f2b8affd700 1 -- 172.21.15.156:0/2983068306 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9400ad00 con 0x7f2b9c160540 2023-11-28T11:08:57.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.818+0000 7f2ba0cae700 1 -- 172.21.15.156:0/2983068306 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c15ccd0 con 0x7f2b9c160540 2023-11-28T11:08:57.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.818+0000 7f2b8affd700 1 -- 172.21.15.156:0/2983068306 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94007310 con 0x7f2b9c160540 2023-11-28T11:08:57.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.818+0000 7f2ba0cae700 1 -- 172.21.15.156:0/2983068306 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c06e870 con 0x7f2b9c160540 2023-11-28T11:08:57.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.819+0000 7f2ba0cae700 1 -- 172.21.15.156:0/2983068306 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c06a270 con 0x7f2b9c160540 2023-11-28T11:08:57.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.819+0000 7f2b8affd700 1 -- 172.21.15.156:0/2983068306 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2b94010070 con 0x7f2b9c160540 2023-11-28T11:08:57.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.820+0000 7f2b8affd700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2b7c068320 0x7f2b7c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.820+0000 7f2b8affd700 1 -- 172.21.15.156:0/2983068306 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b940855d0 con 0x7f2b9c160540 2023-11-28T11:08:57.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.820+0000 7f2b9a59c700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2b7c068320 0x7f2b7c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.820+0000 7f2b9a59c700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2b7c068320 0x7f2b7c06a7d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009870 tx=0x7f2b90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:57.824+0000 7f2b8affd700 1 -- 172.21.15.156:0/2983068306 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2b94051e30 con 0x7f2b9c160540 2023-11-28T11:08:58.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.025+0000 7f2ba0cae700 1 -- 172.21.15.156:0/2983068306 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b9c0c6ad0 con 0x7f2b9c160540 2023-11-28T11:08:58.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.025+0000 7f2b8affd700 1 -- 172.21.15.156:0/2983068306 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2b94055450 con 0x7f2b9c160540 2023-11-28T11:08:58.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.027+0000 7f2ba0cae700 1 -- 172.21.15.156:0/2983068306 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2b7c068320 msgr2=0x7f2b7c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.027+0000 7f2ba0cae700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2b7c068320 0x7f2b7c06a7d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009870 tx=0x7f2b90008040 comp rx=0 tx=0).stop 2023-11-28T11:08:58.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.027+0000 7f2ba0cae700 1 -- 172.21.15.156:0/2983068306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c160540 msgr2=0x7f2b9c0c4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.027+0000 7f2ba0cae700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c160540 0x7f2b9c0c4970 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400cba0 tx=0x7f2b9400cf60 comp rx=0 tx=0).stop 2023-11-28T11:08:58.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.027+0000 7f2ba0cae700 1 -- 172.21.15.156:0/2983068306 shutdown_connections 2023-11-28T11:08:58.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.027+0000 7f2ba0cae700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c160540 0x7f2b9c0c4970 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.027+0000 7f2ba0cae700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2b7c068320 0x7f2b7c06a7d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.027+0000 7f2ba0cae700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b9c06dee0 0x7f2b9c0b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.027+0000 7f2ba0cae700 1 --2- 172.21.15.156:0/2983068306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b9c071040 0x7f2b9c0ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.027+0000 7f2ba0cae700 1 -- 172.21.15.156:0/2983068306 >> 172.21.15.156:0/2983068306 conn(0x7f2b9c0d0000 msgr2=0x7f2b9c0cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.027+0000 7f2ba0cae700 1 -- 172.21.15.156:0/2983068306 shutdown_connections 2023-11-28T11:08:58.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.028+0000 7f2ba0cae700 1 -- 172.21.15.156:0/2983068306 wait complete. 2023-11-28T11:08:58.030 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T11:08:58.041 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-28T11:08:58.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:58.042 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-28T11:08:58.078 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:08:58.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:58.079 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T11:08:58.137 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:08:58.137 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:58.160 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:08:58.160 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:58.220 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:08:58.220 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:58.281 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:58.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.402+0000 7f569d26d700 1 Processor -- start 2023-11-28T11:08:58.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.402+0000 7f569d26d700 1 -- start start 2023-11-28T11:08:58.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f569d26d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f56980cd7c0 0x7f56980cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f569d26d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56980d1950 con 0x7f5698161340 2023-11-28T11:08:58.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f569d26d700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56980ce140 con 0x7f569815c9b0 2023-11-28T11:08:58.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f569d26d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56980b5800 con 0x7f56980cd7c0 2023-11-28T11:08:58.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f56967fc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5698161340 0x7f56980d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48012/0 (socket says 172.21.15.156:48012) 2023-11-28T11:08:58.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f56967fc700 1 -- 172.21.15.156:0/2108603565 learned_addr learned my addr 172.21.15.156:0/2108603565 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:58.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815785480 0 0) 0x7f56980d1950 con 0x7f5698161340 2023-11-28T11:08:58.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56800035b0 con 0x7f5698161340 2023-11-28T11:08:58.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f56977fe700 1 --2- 172.21.15.156:0/2108603565 >> v2:172.21.15.186:3300/0 conn(0x7f56980cd7c0 0x7f56980cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4006847860 0 0) 0x7f56800035b0 con 0x7f5698161340 2023-11-28T11:08:58.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 >> v2:172.21.15.186:3300/0 conn(0x7f56980cd7c0 msgr2=0x7f56980cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:58.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 --2- 172.21.15.156:0/2108603565 >> v2:172.21.15.186:3300/0 conn(0x7f56980cd7c0 0x7f56980cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 >> v1:172.21.15.186:6789/0 conn(0x7f569815c9b0 legacy=0x7f56980b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56980d24f0 con 0x7f5698161340 2023-11-28T11:08:58.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f568c001670 con 0x7f5698161340 2023-11-28T11:08:58.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 >> v1:172.21.15.156:6789/0 conn(0x7f5698161340 legacy=0x7f56980d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 --2- 172.21.15.156:0/2108603565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5680003a00 0x7f5680003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 --2- 172.21.15.156:0/2108603565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5680004850 0x7f5680004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 --2- 172.21.15.156:0/2108603565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5680005320 0x7f568000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56980b5800 con 0x7f5680005320 2023-11-28T11:08:58.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568000c1e0 con 0x7f5680003a00 2023-11-28T11:08:58.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568000c3a0 con 0x7f5680004850 2023-11-28T11:08:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f56977fe700 1 --2- 172.21.15.156:0/2108603565 >> v2:172.21.15.186:3300/0 conn(0x7f56980cd7c0 0x7f56980cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f56977fe700 1 --2- 172.21.15.156:0/2108603565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5680005320 0x7f568000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.403+0000 7f5696ffd700 1 --2- 172.21.15.156:0/2108603565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5680003a00 0x7f5680003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f56967fc700 1 --2- 172.21.15.156:0/2108603565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5680004850 0x7f5680004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f56977fe700 1 -- 172.21.15.156:0/2108603565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5680004850 msgr2=0x7f5680004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f56977fe700 1 --2- 172.21.15.156:0/2108603565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5680004850 0x7f5680004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f56977fe700 1 -- 172.21.15.156:0/2108603565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5680003a00 msgr2=0x7f5680003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f56977fe700 1 --2- 172.21.15.156:0/2108603565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5680003a00 0x7f5680003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f56977fe700 1 -- 172.21.15.156:0/2108603565 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56800035b0 con 0x7f5680005320 2023-11-28T11:08:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f56967fc700 1 --2- 172.21.15.156:0/2108603565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5680004850 0x7f5680004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f56977fe700 1 -- 172.21.15.156:0/2108603565 reap_dead start 2023-11-28T11:08:58.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f56977fe700 1 --2- 172.21.15.156:0/2108603565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5680005320 0x7f568000ba10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f56900073e0 tx=0x7f5680005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64ba80648dd6218 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690009490 con 0x7f5680005320 2023-11-28T11:08:58.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5690014400 con 0x7f5680005320 2023-11-28T11:08:58.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f5695ffb700 1 -- 172.21.15.156:0/2108603565 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56900147b0 con 0x7f5680005320 2023-11-28T11:08:58.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f569d26d700 1 -- 172.21.15.156:0/2108603565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5680005320 msgr2=0x7f568000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.404+0000 7f569d26d700 1 --2- 172.21.15.156:0/2108603565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5680005320 0x7f568000ba10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f56900073e0 tx=0x7f5680005880 comp rx=0 tx=0).stop 2023-11-28T11:08:58.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.405+0000 7f569d26d700 1 -- 172.21.15.156:0/2108603565 shutdown_connections 2023-11-28T11:08:58.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.405+0000 7f569d26d700 1 --2- 172.21.15.156:0/2108603565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5680005320 0x7f568000ba10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.405+0000 7f569d26d700 1 -- 172.21.15.156:0/2108603565 >> 172.21.15.156:0/2108603565 conn(0x7f5698072600 msgr2=0x7f56980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.405+0000 7f569d26d700 1 -- 172.21.15.156:0/2108603565 shutdown_connections 2023-11-28T11:08:58.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.405+0000 7f569d26d700 1 -- 172.21.15.156:0/2108603565 wait complete. 2023-11-28T11:08:58.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.406+0000 7f569d26d700 1 Processor -- start 2023-11-28T11:08:58.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.406+0000 7f569d26d700 1 -- start start 2023-11-28T11:08:58.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f569d26d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5680005320 0x7f56980cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f569d26d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5698157300 0x7f56980ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f569d26d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f569815a780 0x7f5698156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f569d26d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56980cb680 con 0x7f5698157300 2023-11-28T11:08:58.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f569d26d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56980caa70 con 0x7f569815a780 2023-11-28T11:08:58.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f569d26d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56980cad50 con 0x7f5680005320 2023-11-28T11:08:58.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f56967fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5698157300 0x7f56980ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f56967fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5698157300 0x7f56980ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33146/0 (socket says 172.21.15.156:33146) 2023-11-28T11:08:58.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f56967fc700 1 -- 172.21.15.156:0/562773399 learned_addr learned my addr 172.21.15.156:0/562773399 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:58.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f56977fe700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f569815a780 0x7f5698156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f5696ffd700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5680005320 0x7f56980cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f56967fc700 1 -- 172.21.15.156:0/562773399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5680005320 msgr2=0x7f56980cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f56967fc700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5680005320 0x7f56980cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f56967fc700 1 -- 172.21.15.156:0/562773399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f569815a780 msgr2=0x7f5698156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f56967fc700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f569815a780 0x7f5698156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f56967fc700 1 -- 172.21.15.156:0/562773399 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56980d1300 con 0x7f5698157300 2023-11-28T11:08:58.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f56977fe700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f569815a780 0x7f5698156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:58.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f56967fc700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5698157300 0x7f56980ca240 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f568c00ea70 tx=0x7f568c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f567f7fe700 1 -- 172.21.15.156:0/562773399 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c005e70 con 0x7f5698157300 2023-11-28T11:08:58.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f567f7fe700 1 -- 172.21.15.156:0/562773399 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c00c7e0 con 0x7f5698157300 2023-11-28T11:08:58.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f569d26d700 1 -- 172.21.15.156:0/562773399 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56980b5520 con 0x7f5698157300 2023-11-28T11:08:58.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f567f7fe700 1 -- 172.21.15.156:0/562773399 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c00cb50 con 0x7f5698157300 2023-11-28T11:08:58.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.407+0000 7f569d26d700 1 -- 172.21.15.156:0/562773399 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56980d15e0 con 0x7f5698157300 2023-11-28T11:08:58.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.408+0000 7f569d26d700 1 -- 172.21.15.156:0/562773399 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56980745c0 con 0x7f5698157300 2023-11-28T11:08:58.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.409+0000 7f567f7fe700 1 -- 172.21.15.156:0/562773399 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f568c00ccb0 con 0x7f5698157300 2023-11-28T11:08:58.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.409+0000 7f567f7fe700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f56840682f0 0x7f568406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.409+0000 7f567f7fe700 1 -- 172.21.15.156:0/562773399 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f568c085c50 con 0x7f5698157300 2023-11-28T11:08:58.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.409+0000 7f5696ffd700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f56840682f0 0x7f568406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.410+0000 7f5696ffd700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f56840682f0 0x7f568406a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5688007fd0 tx=0x7f5688009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.413+0000 7f567f7fe700 1 -- 172.21.15.156:0/562773399 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f568c052510 con 0x7f5698157300 2023-11-28T11:08:58.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.596+0000 7f569d26d700 1 -- 172.21.15.156:0/562773399 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56980d08a0 con 0x7f5698157300 2023-11-28T11:08:58.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.597+0000 7f567f7fe700 1 -- 172.21.15.156:0/562773399 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f568c055b30 con 0x7f5698157300 2023-11-28T11:08:58.598 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:08:58.598 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:08:18.354009+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:58.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.599+0000 7f569d26d700 1 -- 172.21.15.156:0/562773399 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f56840682f0 msgr2=0x7f568406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.599+0000 7f569d26d700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f56840682f0 0x7f568406a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5688007fd0 tx=0x7f5688009040 comp rx=0 tx=0).stop 2023-11-28T11:08:58.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.599+0000 7f569d26d700 1 -- 172.21.15.156:0/562773399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5698157300 msgr2=0x7f56980ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.599+0000 7f569d26d700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5698157300 0x7f56980ca240 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f568c00ea70 tx=0x7f568c00ee30 comp rx=0 tx=0).stop 2023-11-28T11:08:58.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.599+0000 7f569d26d700 1 -- 172.21.15.156:0/562773399 shutdown_connections 2023-11-28T11:08:58.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.599+0000 7f569d26d700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f569815a780 0x7f5698156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.599+0000 7f569d26d700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f56840682f0 0x7f568406a7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.599+0000 7f569d26d700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5680005320 0x7f56980cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.599+0000 7f569d26d700 1 --2- 172.21.15.156:0/562773399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5698157300 0x7f56980ca240 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.599+0000 7f569d26d700 1 -- 172.21.15.156:0/562773399 >> 172.21.15.156:0/562773399 conn(0x7f5698072600 msgr2=0x7f56980b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.599+0000 7f569d26d700 1 -- 172.21.15.156:0/562773399 shutdown_connections 2023-11-28T11:08:58.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.599+0000 7f569d26d700 1 -- 172.21.15.156:0/562773399 wait complete. 2023-11-28T11:08:58.613 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:15.021591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:16.332241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:58.613 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.735+0000 7fe40c314700 1 Processor -- start 2023-11-28T11:08:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.736+0000 7fe40c314700 1 -- start start 2023-11-28T11:08:58.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.736+0000 7fe40c314700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe404071040 0x7fe404071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.736+0000 7fe40c314700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40406dee0 0x7fe40406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.736+0000 7fe40c314700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe404071950 con 0x7fe40406cdc0 2023-11-28T11:08:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.736+0000 7fe40c314700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe404161550 con 0x7fe404071040 2023-11-28T11:08:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe40c314700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40415d060 con 0x7fe40406dee0 2023-11-28T11:08:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe40a0b0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe404071040 0x7fe404071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe40a8b1700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe40406cdc0 0x7fe40406d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48034/0 (socket says 172.21.15.156:48034) 2023-11-28T11:08:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe40a8b1700 1 -- 172.21.15.156:0/864334470 learned_addr learned my addr 172.21.15.156:0/864334470 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe4098af700 1 --2- 172.21.15.156:0/864334470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40406dee0 0x7fe40406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe40a0b0700 1 -- 172.21.15.156:0/864334470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40406dee0 msgr2=0x7fe40406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe40a0b0700 1 --2- 172.21.15.156:0/864334470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40406dee0 0x7fe40406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe40a0b0700 1 -- 172.21.15.156:0/864334470 >> v1:172.21.15.156:6789/0 conn(0x7fe40406cdc0 legacy=0x7fe40406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe40a0b0700 1 -- 172.21.15.156:0/864334470 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40415d340 con 0x7fe404071040 2023-11-28T11:08:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe4090ae700 1 -- 172.21.15.156:0/864334470 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475833293 0 0) 0x7fe404071950 con 0x7fe40406cdc0 2023-11-28T11:08:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe40a0b0700 1 --2- 172.21.15.156:0/864334470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe404071040 0x7fe404071410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009840 tx=0x7fe3f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67236395ec323d66 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe4090ae700 1 -- 172.21.15.156:0/864334470 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4011830 con 0x7fe404071040 2023-11-28T11:08:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe4090ae700 1 -- 172.21.15.156:0/864334470 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f4011990 con 0x7fe404071040 2023-11-28T11:08:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.737+0000 7fe4090ae700 1 -- 172.21.15.156:0/864334470 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4011d20 con 0x7fe404071040 2023-11-28T11:08:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.738+0000 7fe40c314700 1 -- 172.21.15.156:0/864334470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe404071040 msgr2=0x7fe404071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.738+0000 7fe40c314700 1 --2- 172.21.15.156:0/864334470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe404071040 0x7fe404071410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009840 tx=0x7fe3f4009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.738+0000 7fe40c314700 1 -- 172.21.15.156:0/864334470 shutdown_connections 2023-11-28T11:08:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.738+0000 7fe40c314700 1 --2- 172.21.15.156:0/864334470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe404071040 0x7fe404071410 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.738+0000 7fe40c314700 1 --2- 172.21.15.156:0/864334470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40406dee0 0x7fe40406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.738+0000 7fe40c314700 1 -- 172.21.15.156:0/864334470 >> 172.21.15.156:0/864334470 conn(0x7fe4040d0000 msgr2=0x7fe4040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.738+0000 7fe40c314700 1 -- 172.21.15.156:0/864334470 shutdown_connections 2023-11-28T11:08:58.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.738+0000 7fe40c314700 1 -- 172.21.15.156:0/864334470 wait complete. 2023-11-28T11:08:58.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.739+0000 7fe40c314700 1 Processor -- start 2023-11-28T11:08:58.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.739+0000 7fe40c314700 1 -- start start 2023-11-28T11:08:58.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40c314700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe40406cdc0 0x7fe404154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40c314700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40406dee0 0x7fe4040c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40c314700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe404071040 0x7fe40415f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40c314700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4040c57a0 con 0x7fe40406cdc0 2023-11-28T11:08:58.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40c314700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe404154a90 con 0x7fe40406dee0 2023-11-28T11:08:58.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40c314700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4040ad4c0 con 0x7fe404071040 2023-11-28T11:08:58.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40a0b0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe40406cdc0 0x7fe404154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40a0b0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe40406cdc0 0x7fe404154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33158/0 (socket says 172.21.15.156:33158) 2023-11-28T11:08:58.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40a0b0700 1 -- 172.21.15.156:0/236542688 learned_addr learned my addr 172.21.15.156:0/236542688 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:58.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40a8b1700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe404071040 0x7fe40415f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe4098af700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40406dee0 0x7fe4040c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40a0b0700 1 -- 172.21.15.156:0/236542688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe404071040 msgr2=0x7fe40415f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40a0b0700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe404071040 0x7fe40415f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40a0b0700 1 -- 172.21.15.156:0/236542688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40406dee0 msgr2=0x7fe4040c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40a0b0700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40406dee0 0x7fe4040c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.740+0000 7fe40a0b0700 1 -- 172.21.15.156:0/236542688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f40094a0 con 0x7fe40406cdc0 2023-11-28T11:08:58.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.741+0000 7fe40a0b0700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe40406cdc0 0x7fe404154550 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40090b0 tx=0x7fe3f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.741+0000 7fe3faffd700 1 -- 172.21.15.156:0/236542688 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40036a0 con 0x7fe40406cdc0 2023-11-28T11:08:58.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.741+0000 7fe3faffd700 1 -- 172.21.15.156:0/236542688 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f401f070 con 0x7fe40406cdc0 2023-11-28T11:08:58.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.741+0000 7fe40c314700 1 -- 172.21.15.156:0/236542688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40415d900 con 0x7fe40406cdc0 2023-11-28T11:08:58.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.741+0000 7fe3faffd700 1 -- 172.21.15.156:0/236542688 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4003a60 con 0x7fe40406cdc0 2023-11-28T11:08:58.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.741+0000 7fe40c314700 1 -- 172.21.15.156:0/236542688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40415d620 con 0x7fe40406cdc0 2023-11-28T11:08:58.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.742+0000 7fe40c314700 1 -- 172.21.15.156:0/236542688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40406e870 con 0x7fe40406cdc0 2023-11-28T11:08:58.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.742+0000 7fe3faffd700 1 -- 172.21.15.156:0/236542688 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe3f400f050 con 0x7fe40406cdc0 2023-11-28T11:08:58.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.743+0000 7fe3faffd700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe3e80682f0 0x7fe3e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.743+0000 7fe4098af700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe3e80682f0 0x7fe3e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.743+0000 7fe3faffd700 1 -- 172.21.15.156:0/236542688 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3f4021070 con 0x7fe40406cdc0 2023-11-28T11:08:58.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.743+0000 7fe4098af700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe3e80682f0 0x7fe3e806a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4040ce0d0 tx=0x7fe400008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.747+0000 7fe3faffd700 1 -- 172.21.15.156:0/236542688 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe3f4053d80 con 0x7fe40406cdc0 2023-11-28T11:08:58.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.929+0000 7fe40c314700 1 -- 172.21.15.156:0/236542688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4040720a0 con 0x7fe40406cdc0 2023-11-28T11:08:58.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.929+0000 7fe3faffd700 1 -- 172.21.15.156:0/236542688 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3f40573a0 con 0x7fe40406cdc0 2023-11-28T11:08:58.931 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:08:58.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.931+0000 7fe40c314700 1 -- 172.21.15.156:0/236542688 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe3e80682f0 msgr2=0x7fe3e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.931+0000 7fe40c314700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe3e80682f0 0x7fe3e806a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4040ce0d0 tx=0x7fe400008040 comp rx=0 tx=0).stop 2023-11-28T11:08:58.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.931+0000 7fe40c314700 1 -- 172.21.15.156:0/236542688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe40406cdc0 msgr2=0x7fe404154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.931+0000 7fe40c314700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe40406cdc0 0x7fe404154550 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40090b0 tx=0x7fe3f4004750 comp rx=0 tx=0).stop 2023-11-28T11:08:58.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.931+0000 7fe40c314700 1 -- 172.21.15.156:0/236542688 shutdown_connections 2023-11-28T11:08:58.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.931+0000 7fe40c314700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe3e80682f0 0x7fe3e806a7a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.931+0000 7fe40c314700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe404071040 0x7fe40415f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.931+0000 7fe40c314700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe40406cdc0 0x7fe404154550 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.932+0000 7fe40c314700 1 --2- 172.21.15.156:0/236542688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40406dee0 0x7fe4040c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.932+0000 7fe40c314700 1 -- 172.21.15.156:0/236542688 >> 172.21.15.156:0/236542688 conn(0x7fe4040d0000 msgr2=0x7fe40406d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.932+0000 7fe40c314700 1 -- 172.21.15.156:0/236542688 shutdown_connections 2023-11-28T11:08:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:58.932+0000 7fe40c314700 1 -- 172.21.15.156:0/236542688 wait complete. 2023-11-28T11:08:58.945 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.067+0000 7f35cdb83700 1 Processor -- start 2023-11-28T11:08:59.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.067+0000 7f35cdb83700 1 -- start start 2023-11-28T11:08:59.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35cdb83700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f35c80cd8c0 0x7f35c8140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35cdb83700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cdc90 0x7f35c80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35cdb83700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c813ff00 con 0x7f35c80b2e70 2023-11-28T11:08:59.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35cdb83700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c81404c0 con 0x7f35c80cd8c0 2023-11-28T11:08:59.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35cdb83700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c815aa80 con 0x7f35c80cdc90 2023-11-28T11:08:59.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35c6ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cdc90 0x7f35c80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35c77fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f35c80cd8c0 0x7f35c8140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35c6ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cdc90 0x7f35c80b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37030/0 (socket says 172.21.15.156:37030) 2023-11-28T11:08:59.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35c6ffd700 1 -- 172.21.15.156:0/4104372990 learned_addr learned my addr 172.21.15.156:0/4104372990 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:59.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35c7fff700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f35c80b2e70 0x7f35c806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:58408/0 (socket says 172.21.15.156:58408) 2023-11-28T11:08:59.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35c6ffd700 1 -- 172.21.15.156:0/4104372990 >> v2:172.21.15.186:3300/0 conn(0x7f35c80cd8c0 msgr2=0x7f35c8140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35c6ffd700 1 --2- 172.21.15.156:0/4104372990 >> v2:172.21.15.186:3300/0 conn(0x7f35c80cd8c0 0x7f35c8140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35c6ffd700 1 -- 172.21.15.156:0/4104372990 >> v1:172.21.15.186:6789/0 conn(0x7f35c80b2e70 legacy=0x7f35c806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.068+0000 7f35c6ffd700 1 -- 172.21.15.156:0/4104372990 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c80b3310 con 0x7f35c80cdc90 2023-11-28T11:08:59.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.069+0000 7f35c6ffd700 1 --2- 172.21.15.156:0/4104372990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cdc90 0x7f35c80b2930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35b80098f0 tx=0x7f35b8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab690392495875d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.069+0000 7f35c67fc700 1 -- 172.21.15.156:0/4104372990 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8011830 con 0x7f35c80cdc90 2023-11-28T11:08:59.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.069+0000 7f35c67fc700 1 -- 172.21.15.156:0/4104372990 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b8011990 con 0x7f35c80cdc90 2023-11-28T11:08:59.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.069+0000 7f35c67fc700 1 -- 172.21.15.156:0/4104372990 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8011d20 con 0x7f35c80cdc90 2023-11-28T11:08:59.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.069+0000 7f35cdb83700 1 -- 172.21.15.156:0/4104372990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cdc90 msgr2=0x7f35c80b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.069+0000 7f35cdb83700 1 --2- 172.21.15.156:0/4104372990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cdc90 0x7f35c80b2930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35b80098f0 tx=0x7f35b8009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:59.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.069+0000 7f35cdb83700 1 -- 172.21.15.156:0/4104372990 shutdown_connections 2023-11-28T11:08:59.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.069+0000 7f35cdb83700 1 --2- 172.21.15.156:0/4104372990 >> v2:172.21.15.186:3300/0 conn(0x7f35c80cd8c0 0x7f35c8140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.069+0000 7f35cdb83700 1 --2- 172.21.15.156:0/4104372990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cdc90 0x7f35c80b2930 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.069+0000 7f35cdb83700 1 -- 172.21.15.156:0/4104372990 >> 172.21.15.156:0/4104372990 conn(0x7f35c806aed0 msgr2=0x7f35c806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.069+0000 7f35cdb83700 1 -- 172.21.15.156:0/4104372990 shutdown_connections 2023-11-28T11:08:59.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.069+0000 7f35cdb83700 1 -- 172.21.15.156:0/4104372990 wait complete. 2023-11-28T11:08:59.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.071+0000 7f35cdb83700 1 Processor -- start 2023-11-28T11:08:59.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.071+0000 7f35cdb83700 1 -- start start 2023-11-28T11:08:59.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.071+0000 7f35cdb83700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80b2e70 0x7f35c8143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.071+0000 7f35cdb83700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cd8c0 0x7f35c81ef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.071+0000 7f35cdb83700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35c80cdc90 0x7f35c813e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.071+0000 7f35cdb83700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c8143fa0 con 0x7f35c80b2e70 2023-11-28T11:08:59.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.071+0000 7f35cdb83700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80cf000 con 0x7f35c80cdc90 2023-11-28T11:08:59.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.071+0000 7f35cdb83700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c81435d0 con 0x7f35c80cd8c0 2023-11-28T11:08:59.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35c77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80b2e70 0x7f35c8143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35c77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80b2e70 0x7f35c8143a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33174/0 (socket says 172.21.15.156:33174) 2023-11-28T11:08:59.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35c77fe700 1 -- 172.21.15.156:0/3162597702 learned_addr learned my addr 172.21.15.156:0/3162597702 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:59.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35c6ffd700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cd8c0 0x7f35c81ef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35c7fff700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35c80cdc90 0x7f35c813e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35c77fe700 1 -- 172.21.15.156:0/3162597702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cd8c0 msgr2=0x7f35c81ef9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35c77fe700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cd8c0 0x7f35c81ef9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35c77fe700 1 -- 172.21.15.156:0/3162597702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35c80cdc90 msgr2=0x7f35c813e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35c77fe700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35c80cdc90 0x7f35c813e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35c77fe700 1 -- 172.21.15.156:0/3162597702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b8009550 con 0x7f35c80b2e70 2023-11-28T11:08:59.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35c6ffd700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cd8c0 0x7f35c81ef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:59.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35c77fe700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80b2e70 0x7f35c8143a60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f35b000e9c0 tx=0x7f35b000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35bffff700 1 -- 172.21.15.156:0/3162597702 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00096f0 con 0x7f35c80b2e70 2023-11-28T11:08:59.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35bffff700 1 -- 172.21.15.156:0/3162597702 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b000ee60 con 0x7f35c80b2e70 2023-11-28T11:08:59.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35cdb83700 1 -- 172.21.15.156:0/3162597702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c8140150 con 0x7f35c80b2e70 2023-11-28T11:08:59.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35bffff700 1 -- 172.21.15.156:0/3162597702 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b000ca50 con 0x7f35c80b2e70 2023-11-28T11:08:59.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.072+0000 7f35cdb83700 1 -- 172.21.15.156:0/3162597702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c80b35f0 con 0x7f35c80b2e70 2023-11-28T11:08:59.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.073+0000 7f35cdb83700 1 -- 172.21.15.156:0/3162597702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c80ce100 con 0x7f35c80b2e70 2023-11-28T11:08:59.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.074+0000 7f35bffff700 1 -- 172.21.15.156:0/3162597702 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f35b0023070 con 0x7f35c80b2e70 2023-11-28T11:08:59.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.074+0000 7f35bffff700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35a80682f0 0x7f35a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.074+0000 7f35bffff700 1 -- 172.21.15.156:0/3162597702 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f35b0085e60 con 0x7f35c80b2e70 2023-11-28T11:08:59.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.074+0000 7f35c6ffd700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35a80682f0 0x7f35a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.074+0000 7f35c6ffd700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35a80682f0 0x7f35a806a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f35b80098f0 tx=0x7f35b8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.078+0000 7f35bffff700 1 -- 172.21.15.156:0/3162597702 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f35b00526c0 con 0x7f35c80b2e70 2023-11-28T11:08:59.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.258+0000 7f35cdb83700 1 -- 172.21.15.156:0/3162597702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f35c81438b0 con 0x7f35c80b2e70 2023-11-28T11:08:59.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.259+0000 7f35bffff700 1 -- 172.21.15.156:0/3162597702 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f35b0055ce0 con 0x7f35c80b2e70 2023-11-28T11:08:59.260 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:08:59.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.261+0000 7f35cdb83700 1 -- 172.21.15.156:0/3162597702 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35a80682f0 msgr2=0x7f35a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.261+0000 7f35cdb83700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35a80682f0 0x7f35a806a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f35b80098f0 tx=0x7f35b8008030 comp rx=0 tx=0).stop 2023-11-28T11:08:59.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.261+0000 7f35cdb83700 1 -- 172.21.15.156:0/3162597702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80b2e70 msgr2=0x7f35c8143a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.261+0000 7f35cdb83700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80b2e70 0x7f35c8143a60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f35b000e9c0 tx=0x7f35b000ecd0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.261+0000 7f35cdb83700 1 -- 172.21.15.156:0/3162597702 shutdown_connections 2023-11-28T11:08:59.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.261+0000 7f35cdb83700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35c80cdc90 0x7f35c813e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.261+0000 7f35cdb83700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80b2e70 0x7f35c8143a60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.261+0000 7f35cdb83700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35a80682f0 0x7f35a806a7a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.261+0000 7f35cdb83700 1 --2- 172.21.15.156:0/3162597702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35c80cd8c0 0x7f35c81ef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.261+0000 7f35cdb83700 1 -- 172.21.15.156:0/3162597702 >> 172.21.15.156:0/3162597702 conn(0x7f35c806aed0 msgr2=0x7f35c8158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.261+0000 7f35cdb83700 1 -- 172.21.15.156:0/3162597702 shutdown_connections 2023-11-28T11:08:59.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.261+0000 7f35cdb83700 1 -- 172.21.15.156:0/3162597702 wait complete. 2023-11-28T11:08:59.274 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:59.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.397+0000 7fc67d788700 1 Processor -- start 2023-11-28T11:08:59.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.397+0000 7fc67d788700 1 -- start start 2023-11-28T11:08:59.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.397+0000 7fc67d788700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc678158ce0 0x7fc67815d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.397+0000 7fc67d788700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc67806fb30 0x7fc6780cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc67d788700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc67806bf10 con 0x7fc6780ce2e0 2023-11-28T11:08:59.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc67d788700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc67815d730 con 0x7fc678158ce0 2023-11-28T11:08:59.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc67d788700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6780ce6f0 con 0x7fc67806fb30 2023-11-28T11:08:59.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc676ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc678158ce0 0x7fc67815d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc676ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc678158ce0 0x7fc67815d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33180/0 (socket says 172.21.15.156:33180) 2023-11-28T11:08:59.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc676ffd700 1 -- 172.21.15.156:0/1337902863 learned_addr learned my addr 172.21.15.156:0/1337902863 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:59.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc6767fc700 1 --2- 172.21.15.156:0/1337902863 >> v2:172.21.15.186:3300/0 conn(0x7fc67806fb30 0x7fc6780cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc676ffd700 1 -- 172.21.15.156:0/1337902863 >> v2:172.21.15.186:3300/0 conn(0x7fc67806fb30 msgr2=0x7fc6780cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc676ffd700 1 --2- 172.21.15.156:0/1337902863 >> v2:172.21.15.186:3300/0 conn(0x7fc67806fb30 0x7fc6780cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc676ffd700 1 -- 172.21.15.156:0/1337902863 >> v1:172.21.15.186:6789/0 conn(0x7fc6780ce2e0 legacy=0x7fc67806c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc676ffd700 1 -- 172.21.15.156:0/1337902863 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6781590b0 con 0x7fc678158ce0 2023-11-28T11:08:59.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc676ffd700 1 --2- 172.21.15.156:0/1337902863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc678158ce0 0x7fc67815d1f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc67806da60 tx=0x7fc668009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed22ba93d1877029 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc675ffb700 1 -- 172.21.15.156:0/1337902863 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668004230 con 0x7fc678158ce0 2023-11-28T11:08:59.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc675ffb700 1 -- 172.21.15.156:0/1337902863 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc668004390 con 0x7fc678158ce0 2023-11-28T11:08:59.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.398+0000 7fc675ffb700 1 -- 172.21.15.156:0/1337902863 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6680036a0 con 0x7fc678158ce0 2023-11-28T11:08:59.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.399+0000 7fc67d788700 1 -- 172.21.15.156:0/1337902863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc678158ce0 msgr2=0x7fc67815d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.399+0000 7fc67d788700 1 --2- 172.21.15.156:0/1337902863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc678158ce0 0x7fc67815d1f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc67806da60 tx=0x7fc668009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.399+0000 7fc67d788700 1 -- 172.21.15.156:0/1337902863 shutdown_connections 2023-11-28T11:08:59.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.399+0000 7fc67d788700 1 --2- 172.21.15.156:0/1337902863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc678158ce0 0x7fc67815d1f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.399+0000 7fc67d788700 1 --2- 172.21.15.156:0/1337902863 >> v2:172.21.15.186:3300/0 conn(0x7fc67806fb30 0x7fc6780cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.399+0000 7fc67d788700 1 -- 172.21.15.156:0/1337902863 >> 172.21.15.156:0/1337902863 conn(0x7fc6780cb670 msgr2=0x7fc6780cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.399+0000 7fc67d788700 1 -- 172.21.15.156:0/1337902863 shutdown_connections 2023-11-28T11:08:59.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.399+0000 7fc67d788700 1 -- 172.21.15.156:0/1337902863 wait complete. 2023-11-28T11:08:59.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.400+0000 7fc67d788700 1 Processor -- start 2023-11-28T11:08:59.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.400+0000 7fc67d788700 1 -- start start 2023-11-28T11:08:59.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc67d788700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc67806fb30 0x7fc67806c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc67d788700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6780ce2e0 0x7fc67814f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc67d788700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc678158ce0 0x7fc6781459d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc67d788700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc678158900 con 0x7fc678158ce0 2023-11-28T11:08:59.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc67d788700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67806cdf0 con 0x7fc67806fb30 2023-11-28T11:08:59.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc67d788700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780c5510 con 0x7fc6780ce2e0 2023-11-28T11:08:59.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc6777fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc678158ce0 0x7fc6781459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc6777fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc678158ce0 0x7fc6781459d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33188/0 (socket says 172.21.15.156:33188) 2023-11-28T11:08:59.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc6777fe700 1 -- 172.21.15.156:0/3330991653 learned_addr learned my addr 172.21.15.156:0/3330991653 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:59.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc676ffd700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc67806fb30 0x7fc67806c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc6767fc700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6780ce2e0 0x7fc67814f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc6777fe700 1 -- 172.21.15.156:0/3330991653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6780ce2e0 msgr2=0x7fc67814f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc6777fe700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6780ce2e0 0x7fc67814f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc6777fe700 1 -- 172.21.15.156:0/3330991653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc67806fb30 msgr2=0x7fc67806c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc6777fe700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc67806fb30 0x7fc67806c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.401+0000 7fc6777fe700 1 -- 172.21.15.156:0/3330991653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc668009550 con 0x7fc678158ce0 2023-11-28T11:08:59.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.402+0000 7fc6777fe700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc678158ce0 0x7fc6781459d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc67000cc40 tx=0x7fc67000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.402+0000 7fc65f7fe700 1 -- 172.21.15.156:0/3330991653 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67000aa80 con 0x7fc678158ce0 2023-11-28T11:08:59.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.402+0000 7fc65f7fe700 1 -- 172.21.15.156:0/3330991653 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67000abe0 con 0x7fc678158ce0 2023-11-28T11:08:59.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.402+0000 7fc67d788700 1 -- 172.21.15.156:0/3330991653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6780cb3b0 con 0x7fc678158ce0 2023-11-28T11:08:59.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.402+0000 7fc65f7fe700 1 -- 172.21.15.156:0/3330991653 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc670012420 con 0x7fc678158ce0 2023-11-28T11:08:59.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.402+0000 7fc67d788700 1 -- 172.21.15.156:0/3330991653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6780cb0d0 con 0x7fc678158ce0 2023-11-28T11:08:59.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.403+0000 7fc67d788700 1 -- 172.21.15.156:0/3330991653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc67806b8c0 con 0x7fc678158ce0 2023-11-28T11:08:59.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.403+0000 7fc65f7fe700 1 -- 172.21.15.156:0/3330991653 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc670022070 con 0x7fc678158ce0 2023-11-28T11:08:59.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.403+0000 7fc65f7fe700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc6640682f0 0x7fc66406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.404+0000 7fc676ffd700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc6640682f0 0x7fc66406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.404+0000 7fc65f7fe700 1 -- 172.21.15.156:0/3330991653 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6700861b0 con 0x7fc678158ce0 2023-11-28T11:08:59.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.404+0000 7fc676ffd700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc6640682f0 0x7fc66406a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc668009130 tx=0x7fc668008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.408+0000 7fc65f7fe700 1 -- 172.21.15.156:0/3330991653 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc670052a10 con 0x7fc678158ce0 2023-11-28T11:08:59.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.590+0000 7fc67d788700 1 -- 172.21.15.156:0/3330991653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6780745c0 con 0x7fc678158ce0 2023-11-28T11:08:59.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.591+0000 7fc65f7fe700 1 -- 172.21.15.156:0/3330991653 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc670056030 con 0x7fc678158ce0 2023-11-28T11:08:59.597 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:08:59.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.593+0000 7fc67d788700 1 -- 172.21.15.156:0/3330991653 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc6640682f0 msgr2=0x7fc66406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.593+0000 7fc67d788700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc6640682f0 0x7fc66406a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc668009130 tx=0x7fc668008030 comp rx=0 tx=0).stop 2023-11-28T11:08:59.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.593+0000 7fc67d788700 1 -- 172.21.15.156:0/3330991653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc678158ce0 msgr2=0x7fc6781459d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.593+0000 7fc67d788700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc678158ce0 0x7fc6781459d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc67000cc40 tx=0x7fc67000cf50 comp rx=0 tx=0).stop 2023-11-28T11:08:59.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.593+0000 7fc67d788700 1 -- 172.21.15.156:0/3330991653 shutdown_connections 2023-11-28T11:08:59.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.593+0000 7fc67d788700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc678158ce0 0x7fc6781459d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.593+0000 7fc67d788700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc67806fb30 0x7fc67806c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.593+0000 7fc67d788700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc6640682f0 0x7fc66406a7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.593+0000 7fc67d788700 1 --2- 172.21.15.156:0/3330991653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6780ce2e0 0x7fc67814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.593+0000 7fc67d788700 1 -- 172.21.15.156:0/3330991653 >> 172.21.15.156:0/3330991653 conn(0x7fc6780cb670 msgr2=0x7fc6780ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.593+0000 7fc67d788700 1 -- 172.21.15.156:0/3330991653 shutdown_connections 2023-11-28T11:08:59.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.593+0000 7fc67d788700 1 -- 172.21.15.156:0/3330991653 wait complete. 2023-11-28T11:08:59.606 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:59.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.728+0000 7f4c69410700 1 Processor -- start 2023-11-28T11:08:59.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.728+0000 7f4c69410700 1 -- start start 2023-11-28T11:08:59.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c69410700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c6415c9b0 0x7f4c640b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c69410700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4c64161340 0x7f4c640d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c69410700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c640d1950 con 0x7f4c640d2320 2023-11-28T11:08:59.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c69410700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c640b5800 con 0x7f4c64161340 2023-11-28T11:08:59.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c69410700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640b5520 con 0x7f4c6415c9b0 2023-11-28T11:08:59.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c627fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4c64161340 0x7f4c640d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c637fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4c640d2320 0x7f4c640cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48074/0 (socket says 172.21.15.156:48074) 2023-11-28T11:08:59.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c637fe700 1 -- 172.21.15.156:0/2344642812 learned_addr learned my addr 172.21.15.156:0/2344642812 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:08:59.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c62ffd700 1 --2- 172.21.15.156:0/2344642812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c6415c9b0 0x7f4c640b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c627fc700 1 -- 172.21.15.156:0/2344642812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c6415c9b0 msgr2=0x7f4c640b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c627fc700 1 --2- 172.21.15.156:0/2344642812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c6415c9b0 0x7f4c640b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c627fc700 1 -- 172.21.15.156:0/2344642812 >> v1:172.21.15.156:6789/0 conn(0x7f4c640d2320 legacy=0x7f4c640cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c627fc700 1 -- 172.21.15.156:0/2344642812 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c640d1300 con 0x7f4c64161340 2023-11-28T11:08:59.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c62ffd700 1 --2- 172.21.15.156:0/2344642812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c6415c9b0 0x7f4c640b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:59.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.729+0000 7f4c627fc700 1 --2- 172.21.15.156:0/2344642812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c64161340 0x7f4c640d1de0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009840 tx=0x7f4c54009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b130982f172519 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.730+0000 7f4c61ffb700 1 -- 172.21.15.156:0/2344642812 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54011830 con 0x7f4c64161340 2023-11-28T11:08:59.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.730+0000 7f4c61ffb700 1 -- 172.21.15.156:0/2344642812 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c54011990 con 0x7f4c64161340 2023-11-28T11:08:59.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.730+0000 7f4c61ffb700 1 -- 172.21.15.156:0/2344642812 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54011d20 con 0x7f4c64161340 2023-11-28T11:08:59.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.730+0000 7f4c69410700 1 -- 172.21.15.156:0/2344642812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c64161340 msgr2=0x7f4c640d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.730+0000 7f4c69410700 1 --2- 172.21.15.156:0/2344642812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c64161340 0x7f4c640d1de0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009840 tx=0x7f4c54009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:59.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.730+0000 7f4c69410700 1 -- 172.21.15.156:0/2344642812 shutdown_connections 2023-11-28T11:08:59.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.730+0000 7f4c69410700 1 --2- 172.21.15.156:0/2344642812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c6415c9b0 0x7f4c640b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.730+0000 7f4c69410700 1 --2- 172.21.15.156:0/2344642812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c64161340 0x7f4c640d1de0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.730+0000 7f4c69410700 1 -- 172.21.15.156:0/2344642812 >> 172.21.15.156:0/2344642812 conn(0x7f4c64072600 msgr2=0x7f4c640cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.730+0000 7f4c69410700 1 -- 172.21.15.156:0/2344642812 shutdown_connections 2023-11-28T11:08:59.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.730+0000 7f4c69410700 1 -- 172.21.15.156:0/2344642812 wait complete. 2023-11-28T11:08:59.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.732+0000 7f4c69410700 1 Processor -- start 2023-11-28T11:08:59.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.732+0000 7f4c69410700 1 -- start start 2023-11-28T11:08:59.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c69410700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c640d2320 0x7f4c64151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c69410700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c6415c9b0 0x7f4c6414fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c69410700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c64161340 0x7f4c64159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c69410700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c641527a0 con 0x7f4c640d2320 2023-11-28T11:08:59.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c69410700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c64151e90 con 0x7f4c64161340 2023-11-28T11:08:59.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c69410700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640c8680 con 0x7f4c6415c9b0 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c62ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c640d2320 0x7f4c64151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c62ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c640d2320 0x7f4c64151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33198/0 (socket says 172.21.15.156:33198) 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c62ffd700 1 -- 172.21.15.156:0/4125920030 learned_addr learned my addr 172.21.15.156:0/4125920030 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c627fc700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c6415c9b0 0x7f4c6414fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c637fe700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c64161340 0x7f4c64159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c62ffd700 1 -- 172.21.15.156:0/4125920030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c6415c9b0 msgr2=0x7f4c6414fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c62ffd700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c6415c9b0 0x7f4c6414fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c62ffd700 1 -- 172.21.15.156:0/4125920030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c64161340 msgr2=0x7f4c64159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c62ffd700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c64161340 0x7f4c64159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c62ffd700 1 -- 172.21.15.156:0/4125920030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c540094a0 con 0x7f4c640d2320 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c637fe700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c64161340 0x7f4c64159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.733+0000 7f4c62ffd700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c640d2320 0x7f4c64151500 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800c750 tx=0x7f4c5800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.734+0000 7f4c4b7fe700 1 -- 172.21.15.156:0/4125920030 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58011800 con 0x7f4c640d2320 2023-11-28T11:08:59.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.734+0000 7f4c4b7fe700 1 -- 172.21.15.156:0/4125920030 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c58011960 con 0x7f4c640d2320 2023-11-28T11:08:59.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.734+0000 7f4c4b7fe700 1 -- 172.21.15.156:0/4125920030 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58011cd0 con 0x7f4c640d2320 2023-11-28T11:08:59.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.734+0000 7f4c69410700 1 -- 172.21.15.156:0/4125920030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c640b5130 con 0x7f4c640d2320 2023-11-28T11:08:59.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.734+0000 7f4c69410700 1 -- 172.21.15.156:0/4125920030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c640d15e0 con 0x7f4c640d2320 2023-11-28T11:08:59.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.735+0000 7f4c69410700 1 -- 172.21.15.156:0/4125920030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c640b52c0 con 0x7f4c640d2320 2023-11-28T11:08:59.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.735+0000 7f4c4b7fe700 1 -- 172.21.15.156:0/4125920030 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4c58020070 con 0x7f4c640d2320 2023-11-28T11:08:59.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.735+0000 7f4c4b7fe700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4c500682f0 0x7f4c5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.736+0000 7f4c627fc700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4c500682f0 0x7f4c5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.736+0000 7f4c4b7fe700 1 -- 172.21.15.156:0/4125920030 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c58085c00 con 0x7f4c640d2320 2023-11-28T11:08:59.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.736+0000 7f4c627fc700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4c500682f0 0x7f4c5006a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4c540090b0 tx=0x7f4c54008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.740+0000 7f4c4b7fe700 1 -- 172.21.15.156:0/4125920030 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4c58052370 con 0x7f4c640d2320 2023-11-28T11:08:59.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.924+0000 7f4c69410700 1 -- 172.21.15.156:0/4125920030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c640745c0 con 0x7f4c640d2320 2023-11-28T11:08:59.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.924+0000 7f4c4b7fe700 1 -- 172.21.15.156:0/4125920030 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c58055990 con 0x7f4c640d2320 2023-11-28T11:08:59.926 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:08:59.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.926+0000 7f4c69410700 1 -- 172.21.15.156:0/4125920030 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4c500682f0 msgr2=0x7f4c5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.926+0000 7f4c69410700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4c500682f0 0x7f4c5006a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4c540090b0 tx=0x7f4c54008030 comp rx=0 tx=0).stop 2023-11-28T11:08:59.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.926+0000 7f4c69410700 1 -- 172.21.15.156:0/4125920030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c640d2320 msgr2=0x7f4c64151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.926+0000 7f4c69410700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c640d2320 0x7f4c64151500 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800c750 tx=0x7f4c5800cb10 comp rx=0 tx=0).stop 2023-11-28T11:08:59.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.927+0000 7f4c69410700 1 -- 172.21.15.156:0/4125920030 shutdown_connections 2023-11-28T11:08:59.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.927+0000 7f4c69410700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c64161340 0x7f4c64159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.927+0000 7f4c69410700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c640d2320 0x7f4c64151500 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.927+0000 7f4c69410700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4c500682f0 0x7f4c5006a7a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.927+0000 7f4c69410700 1 --2- 172.21.15.156:0/4125920030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c6415c9b0 0x7f4c6414fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.927+0000 7f4c69410700 1 -- 172.21.15.156:0/4125920030 >> 172.21.15.156:0/4125920030 conn(0x7f4c64072600 msgr2=0x7f4c640b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.927+0000 7f4c69410700 1 -- 172.21.15.156:0/4125920030 shutdown_connections 2023-11-28T11:08:59.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:08:59.927+0000 7f4c69410700 1 -- 172.21.15.156:0/4125920030 wait complete. 2023-11-28T11:08:59.940 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:00.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.063+0000 7fac075fd700 1 Processor -- start 2023-11-28T11:09:00.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.063+0000 7fac075fd700 1 -- start start 2023-11-28T11:09:00.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fac075fd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fac0006dee0 0x7fac0006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fac075fd700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0015d060 con 0x7fac00071040 2023-11-28T11:09:00.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fac075fd700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0015d340 con 0x7fac00160540 2023-11-28T11:09:00.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fac075fd700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac0015d620 con 0x7fac0006dee0 2023-11-28T11:09:00.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fac05399700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fac00071040 0x7fac00071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48084/0 (socket says 172.21.15.156:48084) 2023-11-28T11:09:00.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fac05399700 1 -- 172.21.15.156:0/1222448127 learned_addr learned my addr 172.21.15.156:0/1222448127 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:00.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fac04b98700 1 --2- 172.21.15.156:0/1222448127 >> v2:172.21.15.186:3300/0 conn(0x7fac0006dee0 0x7fac0006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1170025384 0 0) 0x7fac0015d060 con 0x7fac00071040 2023-11-28T11:09:00.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabec0035b0 con 0x7fac00071040 2023-11-28T11:09:00.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1907213548 0 0) 0x7fabec0035b0 con 0x7fac00071040 2023-11-28T11:09:00.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 >> v2:172.21.15.186:3300/0 conn(0x7fac0006dee0 msgr2=0x7fac0006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:00.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fabf7fff700 1 --2- 172.21.15.156:0/1222448127 >> v2:172.21.15.186:3300/0 conn(0x7fac0006dee0 0x7fac0006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 >> v1:172.21.15.186:6789/0 conn(0x7fac00160540 legacy=0x7fac00160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0015d900 con 0x7fac00071040 2023-11-28T11:09:00.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fabfc001670 con 0x7fac00071040 2023-11-28T11:09:00.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 >> v1:172.21.15.156:6789/0 conn(0x7fac00071040 legacy=0x7fac00071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fabf7fff700 1 --2- 172.21.15.156:0/1222448127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 0x7fabec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.064+0000 7fabf7fff700 1 --2- 172.21.15.156:0/1222448127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabec004850 0x7fabec004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fabf7fff700 1 --2- 172.21.15.156:0/1222448127 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabec005320 0x7fabec00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0015d620 con 0x7fabec003a00 2023-11-28T11:09:00.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec00c250 con 0x7fabec004850 2023-11-28T11:09:00.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec00c410 con 0x7fabec005320 2023-11-28T11:09:00.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fac05399700 1 --2- 172.21.15.156:0/1222448127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 0x7fabec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fac04b98700 1 --2- 172.21.15.156:0/1222448127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabec004850 0x7fabec004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fac05b9a700 1 --2- 172.21.15.156:0/1222448127 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabec005320 0x7fabec00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fac05399700 1 -- 172.21.15.156:0/1222448127 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabec005320 msgr2=0x7fabec00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:00.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fac05399700 1 --2- 172.21.15.156:0/1222448127 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabec005320 0x7fabec00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fac05399700 1 -- 172.21.15.156:0/1222448127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabec004850 msgr2=0x7fabec004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:00.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fac05399700 1 --2- 172.21.15.156:0/1222448127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabec004850 0x7fabec004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fac05399700 1 -- 172.21.15.156:0/1222448127 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec0035b0 con 0x7fabec003a00 2023-11-28T11:09:00.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fac05b9a700 1 --2- 172.21.15.156:0/1222448127 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabec005320 0x7fabec00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:00.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fac05b9a700 1 -- 172.21.15.156:0/1222448127 reap_dead start 2023-11-28T11:09:00.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fac05399700 1 --2- 172.21.15.156:0/1222448127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 0x7fabec003e10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00c240 tx=0x7fabec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e94f905127f33b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc007c10 con 0x7fabec003a00 2023-11-28T11:09:00.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc00ce00 con 0x7fabec003a00 2023-11-28T11:09:00.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.065+0000 7fabf7fff700 1 -- 172.21.15.156:0/1222448127 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc005ba0 con 0x7fabec003a00 2023-11-28T11:09:00.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.066+0000 7fac075fd700 1 -- 172.21.15.156:0/1222448127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 msgr2=0x7fabec003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.066+0000 7fac075fd700 1 --2- 172.21.15.156:0/1222448127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 0x7fabec003e10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00c240 tx=0x7fabec005880 comp rx=0 tx=0).stop 2023-11-28T11:09:00.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.066+0000 7fac075fd700 1 -- 172.21.15.156:0/1222448127 shutdown_connections 2023-11-28T11:09:00.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.066+0000 7fac075fd700 1 --2- 172.21.15.156:0/1222448127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 0x7fabec003e10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.066+0000 7fac075fd700 1 -- 172.21.15.156:0/1222448127 >> 172.21.15.156:0/1222448127 conn(0x7fac000d0000 msgr2=0x7fac000d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.066+0000 7fac075fd700 1 -- 172.21.15.156:0/1222448127 shutdown_connections 2023-11-28T11:09:00.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.066+0000 7fac075fd700 1 -- 172.21.15.156:0/1222448127 wait complete. 2023-11-28T11:09:00.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.067+0000 7fac075fd700 1 Processor -- start 2023-11-28T11:09:00.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.067+0000 7fac075fd700 1 -- start start 2023-11-28T11:09:00.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.068+0000 7fac075fd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 0x7fac000c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.068+0000 7fac075fd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac000b2e30 0x7fac001577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.068+0000 7fac075fd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac000ad7b0 0x7fac000adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.068+0000 7fac075fd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac000c5240 con 0x7fabec003a00 2023-11-28T11:09:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.068+0000 7fac075fd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac000ae190 con 0x7fac000b2e30 2023-11-28T11:09:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.068+0000 7fac075fd700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac000b3e40 con 0x7fac000ad7b0 2023-11-28T11:09:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.068+0000 7fac05399700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 0x7fac000c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.068+0000 7fac05399700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 0x7fac000c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33218/0 (socket says 172.21.15.156:33218) 2023-11-28T11:09:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.068+0000 7fac05399700 1 -- 172.21.15.156:0/1244064435 learned_addr learned my addr 172.21.15.156:0/1244064435 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.069+0000 7fac04b98700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac000b2e30 0x7fac001577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.069+0000 7fac05399700 1 -- 172.21.15.156:0/1244064435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac000ad7b0 msgr2=0x7fac000adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.069+0000 7fac05399700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac000ad7b0 0x7fac000adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.069+0000 7fac05399700 1 -- 172.21.15.156:0/1244064435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac000b2e30 msgr2=0x7fac001577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.069+0000 7fac05399700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac000b2e30 0x7fac001577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.069+0000 7fac05399700 1 -- 172.21.15.156:0/1244064435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0006a270 con 0x7fabec003a00 2023-11-28T11:09:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.069+0000 7fac05399700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 0x7fac000c6700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00c830 tx=0x7fabfc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.069+0000 7fabf5ffb700 1 -- 172.21.15.156:0/1244064435 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc004c80 con 0x7fabec003a00 2023-11-28T11:09:00.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.069+0000 7fabf5ffb700 1 -- 172.21.15.156:0/1244064435 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc004e00 con 0x7fabec003a00 2023-11-28T11:09:00.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.069+0000 7fac075fd700 1 -- 172.21.15.156:0/1244064435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac0015ccd0 con 0x7fabec003a00 2023-11-28T11:09:00.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.069+0000 7fabf5ffb700 1 -- 172.21.15.156:0/1244064435 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc007310 con 0x7fabec003a00 2023-11-28T11:09:00.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.069+0000 7fac075fd700 1 -- 172.21.15.156:0/1244064435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac000b3880 con 0x7fabec003a00 2023-11-28T11:09:00.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.070+0000 7fac075fd700 1 -- 172.21.15.156:0/1244064435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac000b4950 con 0x7fabec003a00 2023-11-28T11:09:00.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.071+0000 7fabf5ffb700 1 -- 172.21.15.156:0/1244064435 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fabfc010070 con 0x7fabec003a00 2023-11-28T11:09:00.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.071+0000 7fabf5ffb700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fabe40682f0 0x7fabe406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.071+0000 7fabf5ffb700 1 -- 172.21.15.156:0/1244064435 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fabfc086930 con 0x7fabec003a00 2023-11-28T11:09:00.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.071+0000 7fac04b98700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fabe40682f0 0x7fabe406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.072+0000 7fac04b98700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fabe40682f0 0x7fabe406a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fac000af1c0 tx=0x7fabf0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.076+0000 7fabf5ffb700 1 -- 172.21.15.156:0/1244064435 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fabfc053190 con 0x7fabec003a00 2023-11-28T11:09:00.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.274+0000 7fac075fd700 1 -- 172.21.15.156:0/1244064435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fac000720a0 con 0x7fabec003a00 2023-11-28T11:09:00.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.274+0000 7fabf5ffb700 1 -- 172.21.15.156:0/1244064435 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fabfc0567b0 con 0x7fabec003a00 2023-11-28T11:09:00.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.276+0000 7fac075fd700 1 -- 172.21.15.156:0/1244064435 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fabe40682f0 msgr2=0x7fabe406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.276+0000 7fac075fd700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fabe40682f0 0x7fabe406a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fac000af1c0 tx=0x7fabf0008040 comp rx=0 tx=0).stop 2023-11-28T11:09:00.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.276+0000 7fac075fd700 1 -- 172.21.15.156:0/1244064435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 msgr2=0x7fac000c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.276+0000 7fac075fd700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 0x7fac000c6700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00c830 tx=0x7fabfc001a00 comp rx=0 tx=0).stop 2023-11-28T11:09:00.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.276+0000 7fac075fd700 1 -- 172.21.15.156:0/1244064435 shutdown_connections 2023-11-28T11:09:00.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.276+0000 7fac075fd700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fabe40682f0 0x7fabe406a7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.276+0000 7fac075fd700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac000ad7b0 0x7fac000adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.276+0000 7fac075fd700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabec003a00 0x7fac000c6700 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.276+0000 7fac075fd700 1 --2- 172.21.15.156:0/1244064435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac000b2e30 0x7fac001577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.276+0000 7fac075fd700 1 -- 172.21.15.156:0/1244064435 >> 172.21.15.156:0/1244064435 conn(0x7fac000d0000 msgr2=0x7fac0015e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.276+0000 7fac075fd700 1 -- 172.21.15.156:0/1244064435 shutdown_connections 2023-11-28T11:09:00.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.276+0000 7fac075fd700 1 -- 172.21.15.156:0/1244064435 wait complete. 2023-11-28T11:09:00.279 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T11:09:00.289 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-28T11:09:00.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:00.289 DEBUG:teuthology.orchestra.run.smithi186:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-28T11:09:00.335 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T11:09:00.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:00.336 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T11:09:00.354 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:09:00.355 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:00.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.474+0000 7f7f63ac6700 1 Processor -- start 2023-11-28T11:09:00.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.475+0000 7f7f63ac6700 1 -- start start 2023-11-28T11:09:00.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.475+0000 7f7f63ac6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f5c15cef0 0x7f7f5c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.475+0000 7f7f63ac6700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7f5c0cbdf0 0x7f7f5c06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.475+0000 7f7f63ac6700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5c0cc5f0 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.475+0000 7f7f63ac6700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f5c06bd40 con 0x7f7f5c0cbdf0 2023-11-28T11:09:00.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.475+0000 7f7f63ac6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c15d470 con 0x7f7f5c15cef0 2023-11-28T11:09:00.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.476+0000 7f7f61061700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7f5c0cbdf0 0x7f7f5c06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.476+0000 7f7f61061700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7f5c0cbdf0 0x7f7f5c06b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33226/0 (socket says 172.21.15.156:33226) 2023-11-28T11:09:00.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.476+0000 7f7f61061700 1 -- 172.21.15.156:0/1964981015 learned_addr learned my addr 172.21.15.156:0/1964981015 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:00.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.476+0000 7f7f61862700 1 --2- 172.21.15.156:0/1964981015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f5c15cef0 0x7f7f5c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.476+0000 7f7f61061700 1 -- 172.21.15.156:0/1964981015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f5c15cef0 msgr2=0x7f7f5c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:00.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.476+0000 7f7f61061700 1 --2- 172.21.15.156:0/1964981015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f5c15cef0 0x7f7f5c0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.476+0000 7f7f61061700 1 -- 172.21.15.156:0/1964981015 >> v1:172.21.15.186:6789/0 conn(0x7f7f5c06b8c0 legacy=0x7f7f5c071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:00.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.476+0000 7f7f61061700 1 -- 172.21.15.156:0/1964981015 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c15d750 con 0x7f7f5c0cbdf0 2023-11-28T11:09:00.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.476+0000 7f7f61061700 1 --2- 172.21.15.156:0/1964981015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f5c0cbdf0 0x7f7f5c06b380 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800bb90 tx=0x7f7f5800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde3918ff748274e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.476+0000 7f7f60860700 1 -- 172.21.15.156:0/1964981015 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800dd10 con 0x7f7f5c0cbdf0 2023-11-28T11:09:00.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.476+0000 7f7f60860700 1 -- 172.21.15.156:0/1964981015 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5800de70 con 0x7f7f5c0cbdf0 2023-11-28T11:09:00.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.476+0000 7f7f60860700 1 -- 172.21.15.156:0/1964981015 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f58004230 con 0x7f7f5c0cbdf0 2023-11-28T11:09:00.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.477+0000 7f7f63ac6700 1 -- 172.21.15.156:0/1964981015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f5c0cbdf0 msgr2=0x7f7f5c06b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.477+0000 7f7f63ac6700 1 --2- 172.21.15.156:0/1964981015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f5c0cbdf0 0x7f7f5c06b380 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800bb90 tx=0x7f7f5800bea0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.477+0000 7f7f63ac6700 1 -- 172.21.15.156:0/1964981015 shutdown_connections 2023-11-28T11:09:00.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.477+0000 7f7f63ac6700 1 --2- 172.21.15.156:0/1964981015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f5c15cef0 0x7f7f5c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.477+0000 7f7f63ac6700 1 --2- 172.21.15.156:0/1964981015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f5c0cbdf0 0x7f7f5c06b380 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.477+0000 7f7f63ac6700 1 -- 172.21.15.156:0/1964981015 >> 172.21.15.156:0/1964981015 conn(0x7f7f5c0d0000 msgr2=0x7f7f5c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.477+0000 7f7f63ac6700 1 -- 172.21.15.156:0/1964981015 shutdown_connections 2023-11-28T11:09:00.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.477+0000 7f7f63ac6700 1 -- 172.21.15.156:0/1964981015 wait complete. 2023-11-28T11:09:00.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.478+0000 7f7f63ac6700 1 Processor -- start 2023-11-28T11:09:00.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.479+0000 7f7f63ac6700 1 -- start start 2023-11-28T11:09:00.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.479+0000 7f7f63ac6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f5c06b8c0 0x7f7f5c158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.479+0000 7f7f63ac6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c0cbdf0 0x7f7f5c0b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.479+0000 7f7f63ac6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f5c15cef0 0x7f7f5c0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.479+0000 7f7f63ac6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c1e4fe0 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.479+0000 7f7f63ac6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c1e52c0 con 0x7f7f5c0cbdf0 2023-11-28T11:09:00.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.479+0000 7f7f63ac6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c1e55a0 con 0x7f7f5c15cef0 2023-11-28T11:09:00.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f61862700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f5c06b8c0 0x7f7f5c158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f61862700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f5c06b8c0 0x7f7f5c158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33232/0 (socket says 172.21.15.156:33232) 2023-11-28T11:09:00.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f61862700 1 -- 172.21.15.156:0/837830432 learned_addr learned my addr 172.21.15.156:0/837830432 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:00.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f61862700 1 -- 172.21.15.156:0/837830432 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f5c15cef0 msgr2=0x7f7f5c0b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f61862700 1 --2- 172.21.15.156:0/837830432 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f5c15cef0 0x7f7f5c0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f61862700 1 -- 172.21.15.156:0/837830432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c0cbdf0 msgr2=0x7f7f5c0b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f61862700 1 --2- 172.21.15.156:0/837830432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c0cbdf0 0x7f7f5c0b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f61862700 1 -- 172.21.15.156:0/837830432 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5800b7f0 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f61862700 1 --2- 172.21.15.156:0/837830432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f5c06b8c0 0x7f7f5c158c90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00c7f0 tx=0x7f7f4c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f527fc700 1 -- 172.21.15.156:0/837830432 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c011800 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f527fc700 1 -- 172.21.15.156:0/837830432 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c011960 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f527fc700 1 -- 172.21.15.156:0/837830432 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c011cf0 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f63ac6700 1 -- 172.21.15.156:0/837830432 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5c15dd10 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.480+0000 7f7f63ac6700 1 -- 172.21.15.156:0/837830432 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5c15da30 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.481+0000 7f7f63ac6700 1 -- 172.21.15.156:0/837830432 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5c15dff0 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.481+0000 7f7f527fc700 1 -- 172.21.15.156:0/837830432 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7f4c021070 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.482+0000 7f7f527fc700 1 --2- 172.21.15.156:0/837830432 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7f400682f0 0x7f7f4006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.482+0000 7f7f61061700 1 --2- 172.21.15.156:0/837830432 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7f400682f0 0x7f7f4006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.482+0000 7f7f527fc700 1 -- 172.21.15.156:0/837830432 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7f4c085a40 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.482+0000 7f7f61061700 1 --2- 172.21.15.156:0/837830432 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7f400682f0 0x7f7f4006a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800bee0 tx=0x7f7f58012000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.486+0000 7f7f527fc700 1 -- 172.21.15.156:0/837830432 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7f4c0522a0 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.718+0000 7f7f63ac6700 1 -- 172.21.15.156:0/837830432 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f5c0721d0 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.719+0000 7f7f527fc700 1 -- 172.21.15.156:0/837830432 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f7f4c0558c0 con 0x7f7f5c06b8c0 2023-11-28T11:09:00.720 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:00.720 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1221397150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1221397150},{"type":"v1","addr":"172.21.15.186:6834","nonce":1221397150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/22494485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":22494485},{"type":"v1","addr":"172.21.15.156:6835","nonce":22494485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3382922708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3382922708},{"type":"v1","addr":"172.21.15.156:6837","nonce":3382922708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:17.352335+0000","modified":"2023-11-28T11:08:20.752191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/4134790300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":4134790300},{"type":"v1","addr":"172.21.15.186:6835","nonce":4134790300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:00.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.721+0000 7f7f63ac6700 1 -- 172.21.15.156:0/837830432 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7f400682f0 msgr2=0x7f7f4006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.721+0000 7f7f63ac6700 1 --2- 172.21.15.156:0/837830432 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7f400682f0 0x7f7f4006a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800bee0 tx=0x7f7f58012000 comp rx=0 tx=0).stop 2023-11-28T11:09:00.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.721+0000 7f7f63ac6700 1 -- 172.21.15.156:0/837830432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f5c06b8c0 msgr2=0x7f7f5c158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.721+0000 7f7f63ac6700 1 --2- 172.21.15.156:0/837830432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f5c06b8c0 0x7f7f5c158c90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00c7f0 tx=0x7f7f4c00cbb0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.721+0000 7f7f63ac6700 1 -- 172.21.15.156:0/837830432 shutdown_connections 2023-11-28T11:09:00.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.721+0000 7f7f63ac6700 1 --2- 172.21.15.156:0/837830432 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7f400682f0 0x7f7f4006a7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.721+0000 7f7f63ac6700 1 --2- 172.21.15.156:0/837830432 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f5c15cef0 0x7f7f5c0b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.721+0000 7f7f63ac6700 1 --2- 172.21.15.156:0/837830432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f5c06b8c0 0x7f7f5c158c90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.721+0000 7f7f63ac6700 1 --2- 172.21.15.156:0/837830432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c0cbdf0 0x7f7f5c0b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.721+0000 7f7f63ac6700 1 -- 172.21.15.156:0/837830432 >> 172.21.15.156:0/837830432 conn(0x7f7f5c0d0000 msgr2=0x7f7f5c06ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.721+0000 7f7f63ac6700 1 -- 172.21.15.156:0/837830432 shutdown_connections 2023-11-28T11:09:00.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.721+0000 7f7f63ac6700 1 -- 172.21.15.156:0/837830432 wait complete. 2023-11-28T11:09:00.724 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 8 2023-11-28T11:09:00.735 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:00.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.854+0000 7f90144de700 1 Processor -- start 2023-11-28T11:09:00.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.854+0000 7f90144de700 1 -- start start 2023-11-28T11:09:00.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.855+0000 7f90144de700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f900c0cd8c0 0x7f900c140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.855+0000 7f90144de700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c0cdc90 0x7f900c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.855+0000 7f90144de700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900c13ff00 con 0x7f900c0b2e70 2023-11-28T11:09:00.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.855+0000 7f90144de700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c1404c0 con 0x7f900c0cd8c0 2023-11-28T11:09:00.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.855+0000 7f90144de700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c15aa80 con 0x7f900c0cdc90 2023-11-28T11:09:00.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.856+0000 7f901227a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f900c0cd8c0 0x7f900c140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.856+0000 7f9012a7b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f900c0b2e70 0x7f900c06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48124/0 (socket says 172.21.15.156:48124) 2023-11-28T11:09:00.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.856+0000 7f9012a7b700 1 -- 172.21.15.156:0/2242745396 learned_addr learned my addr 172.21.15.156:0/2242745396 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:00.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.856+0000 7f9011a79700 1 --2- 172.21.15.156:0/2242745396 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c0cdc90 0x7f900c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.856+0000 7f901227a700 1 -- 172.21.15.156:0/2242745396 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c0cdc90 msgr2=0x7f900c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:00.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.856+0000 7f901227a700 1 --2- 172.21.15.156:0/2242745396 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c0cdc90 0x7f900c0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.856+0000 7f901227a700 1 -- 172.21.15.156:0/2242745396 >> v1:172.21.15.156:6789/0 conn(0x7f900c0b2e70 legacy=0x7f900c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.856+0000 7f901227a700 1 -- 172.21.15.156:0/2242745396 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0b3310 con 0x7f900c0cd8c0 2023-11-28T11:09:00.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.856+0000 7f901227a700 1 --2- 172.21.15.156:0/2242745396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900c0cd8c0 0x7f900c140950 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9008009820 tx=0x7f9008009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b692b7d1db4ea9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.857+0000 7f9011278700 1 -- 172.21.15.156:0/2242745396 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008005e30 con 0x7f900c0cd8c0 2023-11-28T11:09:00.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.857+0000 7f9011278700 1 -- 172.21.15.156:0/2242745396 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9008006030 con 0x7f900c0cd8c0 2023-11-28T11:09:00.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.857+0000 7f9011278700 1 -- 172.21.15.156:0/2242745396 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90080063c0 con 0x7f900c0cd8c0 2023-11-28T11:09:00.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.858+0000 7f90144de700 1 -- 172.21.15.156:0/2242745396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900c0cd8c0 msgr2=0x7f900c140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.858+0000 7f90144de700 1 --2- 172.21.15.156:0/2242745396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900c0cd8c0 0x7f900c140950 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9008009820 tx=0x7f9008009b30 comp rx=0 tx=0).stop 2023-11-28T11:09:00.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.858+0000 7f90144de700 1 -- 172.21.15.156:0/2242745396 shutdown_connections 2023-11-28T11:09:00.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.858+0000 7f90144de700 1 --2- 172.21.15.156:0/2242745396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900c0cd8c0 0x7f900c140950 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.858+0000 7f90144de700 1 --2- 172.21.15.156:0/2242745396 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c0cdc90 0x7f900c0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.858+0000 7f90144de700 1 -- 172.21.15.156:0/2242745396 >> 172.21.15.156:0/2242745396 conn(0x7f900c06aed0 msgr2=0x7f900c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.858+0000 7f90144de700 1 -- 172.21.15.156:0/2242745396 shutdown_connections 2023-11-28T11:09:00.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.858+0000 7f90144de700 1 -- 172.21.15.156:0/2242745396 wait complete. 2023-11-28T11:09:00.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.859+0000 7f90144de700 1 Processor -- start 2023-11-28T11:09:00.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.859+0000 7f90144de700 1 -- start start 2023-11-28T11:09:00.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f90144de700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f900c0b2e70 0x7f900c158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f90144de700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900c0cd8c0 0x7f900c143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f90144de700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c0cdc90 0x7f900c1437a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f90144de700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0c5dc0 con 0x7f900c0cd8c0 2023-11-28T11:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f90144de700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0c77f0 con 0x7f900c0b2e70 2023-11-28T11:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f90144de700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0c5490 con 0x7f900c0cdc90 2023-11-28T11:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f9011a79700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900c0cd8c0 0x7f900c143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f9011a79700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900c0cd8c0 0x7f900c143260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33250/0 (socket says 172.21.15.156:33250) 2023-11-28T11:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f9011a79700 1 -- 172.21.15.156:0/2650743960 learned_addr learned my addr 172.21.15.156:0/2650743960 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f901227a700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f900c0b2e70 0x7f900c158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f9012a7b700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c0cdc90 0x7f900c1437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f9011a79700 1 -- 172.21.15.156:0/2650743960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c0cdc90 msgr2=0x7f900c1437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f9011a79700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c0cdc90 0x7f900c1437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f9011a79700 1 -- 172.21.15.156:0/2650743960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f900c0b2e70 msgr2=0x7f900c158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.860+0000 7f9011a79700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f900c0b2e70 0x7f900c158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.861+0000 7f9011a79700 1 -- 172.21.15.156:0/2650743960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9008009430 con 0x7f900c0cd8c0 2023-11-28T11:09:00.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.861+0000 7f9011a79700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900c0cd8c0 0x7f900c143260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00e840 tx=0x7f8ffc00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.861+0000 7f9002ffd700 1 -- 172.21.15.156:0/2650743960 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc009590 con 0x7f900c0cd8c0 2023-11-28T11:09:00.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.861+0000 7f90144de700 1 -- 172.21.15.156:0/2650743960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c140150 con 0x7f900c0cd8c0 2023-11-28T11:09:00.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.861+0000 7f9002ffd700 1 -- 172.21.15.156:0/2650743960 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc017d80 con 0x7f900c0cd8c0 2023-11-28T11:09:00.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.861+0000 7f9002ffd700 1 -- 172.21.15.156:0/2650743960 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc01f660 con 0x7f900c0cd8c0 2023-11-28T11:09:00.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.861+0000 7f90144de700 1 -- 172.21.15.156:0/2650743960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c0b35f0 con 0x7f900c0cd8c0 2023-11-28T11:09:00.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.862+0000 7f90144de700 1 -- 172.21.15.156:0/2650743960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c0ce100 con 0x7f900c0cd8c0 2023-11-28T11:09:00.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.863+0000 7f9002ffd700 1 -- 172.21.15.156:0/2650743960 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ffc026030 con 0x7f900c0cd8c0 2023-11-28T11:09:00.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.863+0000 7f9002ffd700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8ff00682f0 0x7f8ff006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.863+0000 7f9002ffd700 1 -- 172.21.15.156:0/2650743960 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8ffc08dd00 con 0x7f900c0cd8c0 2023-11-28T11:09:00.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.863+0000 7f901227a700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8ff00682f0 0x7f8ff006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.864+0000 7f901227a700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8ff00682f0 0x7f8ff006a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9008000c00 tx=0x7f9008008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:00.868+0000 7f9002ffd700 1 -- 172.21.15.156:0/2650743960 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ffc091030 con 0x7f900c0cd8c0 2023-11-28T11:09:01.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.052+0000 7f90144de700 1 -- 172.21.15.156:0/2650743960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f900c068690 con 0x7f900c0cd8c0 2023-11-28T11:09:01.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.053+0000 7f9002ffd700 1 -- 172.21.15.156:0/2650743960 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18050 (secure 0 0 0) 0x7f8ffc024090 con 0x7f900c0cd8c0 2023-11-28T11:09:01.055 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:01.055 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":24,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:00.855446+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.056+0000 7f90144de700 1 -- 172.21.15.156:0/2650743960 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8ff00682f0 msgr2=0x7f8ff006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.056+0000 7f90144de700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8ff00682f0 0x7f8ff006a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9008000c00 tx=0x7f9008008070 comp rx=0 tx=0).stop 2023-11-28T11:09:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.056+0000 7f90144de700 1 -- 172.21.15.156:0/2650743960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900c0cd8c0 msgr2=0x7f900c143260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.056+0000 7f90144de700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900c0cd8c0 0x7f900c143260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00e840 tx=0x7f8ffc00eb50 comp rx=0 tx=0).stop 2023-11-28T11:09:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.056+0000 7f90144de700 1 -- 172.21.15.156:0/2650743960 shutdown_connections 2023-11-28T11:09:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.056+0000 7f90144de700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8ff00682f0 0x7f8ff006a7a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.056+0000 7f90144de700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c0cdc90 0x7f900c1437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.056+0000 7f90144de700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f900c0b2e70 0x7f900c158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.056+0000 7f90144de700 1 --2- 172.21.15.156:0/2650743960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900c0cd8c0 0x7f900c143260 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.056+0000 7f90144de700 1 -- 172.21.15.156:0/2650743960 >> 172.21.15.156:0/2650743960 conn(0x7f900c06aed0 msgr2=0x7f900c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.056+0000 7f90144de700 1 -- 172.21.15.156:0/2650743960 shutdown_connections 2023-11-28T11:09:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.056+0000 7f90144de700 1 -- 172.21.15.156:0/2650743960 wait complete. 2023-11-28T11:09:01.070 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:15.021591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:16.332241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:09:01.070 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:01.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.192+0000 7fc244958700 1 Processor -- start 2023-11-28T11:09:01.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.192+0000 7fc244958700 1 -- start start 2023-11-28T11:09:01.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.193+0000 7fc244958700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc240071040 0x7fc240071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.193+0000 7fc244958700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc240160540 0x7fc240160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.193+0000 7fc244958700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc240071950 con 0x7fc24006dee0 2023-11-28T11:09:01.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.193+0000 7fc244958700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24015d060 con 0x7fc240160540 2023-11-28T11:09:01.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.193+0000 7fc244958700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24015d340 con 0x7fc240071040 2023-11-28T11:09:01.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.193+0000 7fc23ed9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc240160540 0x7fc240160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.193+0000 7fc23ed9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc240160540 0x7fc240160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33260/0 (socket says 172.21.15.156:33260) 2023-11-28T11:09:01.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.193+0000 7fc23ed9d700 1 -- 172.21.15.156:0/65999604 learned_addr learned my addr 172.21.15.156:0/65999604 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:01.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.193+0000 7fc23e59c700 1 --2- 172.21.15.156:0/65999604 >> v2:172.21.15.186:3300/0 conn(0x7fc240071040 0x7fc240071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.193+0000 7fc23ed9d700 1 -- 172.21.15.156:0/65999604 >> v2:172.21.15.186:3300/0 conn(0x7fc240071040 msgr2=0x7fc240071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc23ed9d700 1 --2- 172.21.15.156:0/65999604 >> v2:172.21.15.186:3300/0 conn(0x7fc240071040 0x7fc240071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc23ed9d700 1 -- 172.21.15.156:0/65999604 >> v1:172.21.15.186:6789/0 conn(0x7fc24006dee0 legacy=0x7fc24006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc23ed9d700 1 -- 172.21.15.156:0/65999604 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24015d620 con 0x7fc240160540 2023-11-28T11:09:01.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc23e59c700 1 --2- 172.21.15.156:0/65999604 >> v2:172.21.15.186:3300/0 conn(0x7fc240071040 0x7fc240071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:01.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc23ed9d700 1 --2- 172.21.15.156:0/65999604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc240160540 0x7fc240160980 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc2400d0f50 tx=0x7fc23800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca722fb0f399de7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc23d59a700 1 -- 172.21.15.156:0/65999604 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23800e070 con 0x7fc240160540 2023-11-28T11:09:01.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc23d59a700 1 -- 172.21.15.156:0/65999604 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc238003a20 con 0x7fc240160540 2023-11-28T11:09:01.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc23d59a700 1 -- 172.21.15.156:0/65999604 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238003dd0 con 0x7fc240160540 2023-11-28T11:09:01.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc244958700 1 -- 172.21.15.156:0/65999604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc240160540 msgr2=0x7fc240160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc244958700 1 --2- 172.21.15.156:0/65999604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc240160540 0x7fc240160980 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc2400d0f50 tx=0x7fc23800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:01.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc244958700 1 -- 172.21.15.156:0/65999604 shutdown_connections 2023-11-28T11:09:01.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc244958700 1 --2- 172.21.15.156:0/65999604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc240160540 0x7fc240160980 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc244958700 1 --2- 172.21.15.156:0/65999604 >> v2:172.21.15.186:3300/0 conn(0x7fc240071040 0x7fc240071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc244958700 1 -- 172.21.15.156:0/65999604 >> 172.21.15.156:0/65999604 conn(0x7fc2400d0000 msgr2=0x7fc2400d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc244958700 1 -- 172.21.15.156:0/65999604 shutdown_connections 2023-11-28T11:09:01.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.194+0000 7fc244958700 1 -- 172.21.15.156:0/65999604 wait complete. 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.196+0000 7fc244958700 1 Processor -- start 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.196+0000 7fc244958700 1 -- start start 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.196+0000 7fc244958700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc24006dee0 0x7fc2400c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc244958700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc240071040 0x7fc24014a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc244958700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc240160540 0x7fc240159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc244958700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400c6980 con 0x7fc240071040 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc244958700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400c5fb0 con 0x7fc24006dee0 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc244958700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400b1310 con 0x7fc240160540 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc23dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc240071040 0x7fc24014a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc23dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc240071040 0x7fc24014a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33264/0 (socket says 172.21.15.156:33264) 2023-11-28T11:09:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc23dd9b700 1 -- 172.21.15.156:0/2769307480 learned_addr learned my addr 172.21.15.156:0/2769307480 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc23e59c700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc24006dee0 0x7fc2400c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc23ed9d700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc240160540 0x7fc240159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc23dd9b700 1 -- 172.21.15.156:0/2769307480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc240160540 msgr2=0x7fc240159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc23dd9b700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc240160540 0x7fc240159ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc23dd9b700 1 -- 172.21.15.156:0/2769307480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc24006dee0 msgr2=0x7fc2400c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc23dd9b700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc24006dee0 0x7fc2400c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc23dd9b700 1 -- 172.21.15.156:0/2769307480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc238009b30 con 0x7fc240071040 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc23dd9b700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc240071040 0x7fc24014a310 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc23000b4a0 tx=0x7fc23000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc22effd700 1 -- 172.21.15.156:0/2769307480 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230011830 con 0x7fc240071040 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc22effd700 1 -- 172.21.15.156:0/2769307480 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc230011990 con 0x7fc240071040 2023-11-28T11:09:01.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc244958700 1 -- 172.21.15.156:0/2769307480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24015ccd0 con 0x7fc240071040 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc22effd700 1 -- 172.21.15.156:0/2769307480 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230011d40 con 0x7fc240071040 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.197+0000 7fc244958700 1 -- 172.21.15.156:0/2769307480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24015d900 con 0x7fc240071040 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.198+0000 7fc244958700 1 -- 172.21.15.156:0/2769307480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24006a270 con 0x7fc240071040 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.199+0000 7fc22effd700 1 -- 172.21.15.156:0/2769307480 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc23000f050 con 0x7fc240071040 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.199+0000 7fc22effd700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc2200682f0 0x7fc22006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.199+0000 7fc23e59c700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc2200682f0 0x7fc22006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.199+0000 7fc22effd700 1 -- 172.21.15.156:0/2769307480 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc2300858c0 con 0x7fc240071040 2023-11-28T11:09:01.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.200+0000 7fc23e59c700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc2200682f0 0x7fc22006a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc234006fd0 tx=0x7fc234008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.203+0000 7fc22effd700 1 -- 172.21.15.156:0/2769307480 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc2300520e0 con 0x7fc240071040 2023-11-28T11:09:01.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.385+0000 7fc244958700 1 -- 172.21.15.156:0/2769307480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2400720a0 con 0x7fc240071040 2023-11-28T11:09:01.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.385+0000 7fc22effd700 1 -- 172.21.15.156:0/2769307480 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc230055700 con 0x7fc240071040 2023-11-28T11:09:01.387 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:09:01.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.387+0000 7fc244958700 1 -- 172.21.15.156:0/2769307480 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc2200682f0 msgr2=0x7fc22006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.387+0000 7fc244958700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc2200682f0 0x7fc22006a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc234006fd0 tx=0x7fc234008040 comp rx=0 tx=0).stop 2023-11-28T11:09:01.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.387+0000 7fc244958700 1 -- 172.21.15.156:0/2769307480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc240071040 msgr2=0x7fc24014a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.387+0000 7fc244958700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc240071040 0x7fc24014a310 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc23000b4a0 tx=0x7fc23000b860 comp rx=0 tx=0).stop 2023-11-28T11:09:01.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.388+0000 7fc244958700 1 -- 172.21.15.156:0/2769307480 shutdown_connections 2023-11-28T11:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.388+0000 7fc244958700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc2200682f0 0x7fc22006a7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.388+0000 7fc244958700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc240160540 0x7fc240159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.388+0000 7fc244958700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc24006dee0 0x7fc2400c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.388+0000 7fc244958700 1 --2- 172.21.15.156:0/2769307480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc240071040 0x7fc24014a310 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.388+0000 7fc244958700 1 -- 172.21.15.156:0/2769307480 >> 172.21.15.156:0/2769307480 conn(0x7fc2400d0000 msgr2=0x7fc2400aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.388+0000 7fc244958700 1 -- 172.21.15.156:0/2769307480 shutdown_connections 2023-11-28T11:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.388+0000 7fc244958700 1 -- 172.21.15.156:0/2769307480 wait complete. 2023-11-28T11:09:01.401 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:01.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.523+0000 7f6d4b500700 1 Processor -- start 2023-11-28T11:09:01.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.523+0000 7f6d4b500700 1 -- start start 2023-11-28T11:09:01.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.524+0000 7f6d4b500700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d4406e500 0x7f6d4415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.524+0000 7f6d4b500700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6d440ae9d0 0x7f6d4406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.524+0000 7f6d4b500700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4406ee10 con 0x7f6d4415e540 2023-11-28T11:09:01.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.524+0000 7f6d4b500700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4415bf10 con 0x7f6d440ae9d0 2023-11-28T11:09:01.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.524+0000 7f6d4b500700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d440aee40 con 0x7f6d4406e500 2023-11-28T11:09:01.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.525+0000 7f6d4929c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d4406e500 0x7f6d4415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.525+0000 7f6d48a9b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6d440ae9d0 0x7f6d4406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.525+0000 7f6d49a9d700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f6d4415e540 0x7f6d4415e960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:58510/0 (socket says 172.21.15.156:58510) 2023-11-28T11:09:01.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.525+0000 7f6d49a9d700 1 -- 172.21.15.156:0/184194820 learned_addr learned my addr 172.21.15.156:0/184194820 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:01.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.525+0000 7f6d4929c700 1 -- 172.21.15.156:0/184194820 >> v2:172.21.15.186:3300/0 conn(0x7f6d440ae9d0 msgr2=0x7f6d4406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.525+0000 7f6d4929c700 1 --2- 172.21.15.156:0/184194820 >> v2:172.21.15.186:3300/0 conn(0x7f6d440ae9d0 0x7f6d4406e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.525+0000 7f6d4929c700 1 -- 172.21.15.156:0/184194820 >> v1:172.21.15.186:6789/0 conn(0x7f6d4415e540 legacy=0x7f6d4415e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.525+0000 7f6d4929c700 1 -- 172.21.15.156:0/184194820 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4406c930 con 0x7f6d4406e500 2023-11-28T11:09:01.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.525+0000 7f6d48a9b700 1 --2- 172.21.15.156:0/184194820 >> v2:172.21.15.186:3300/0 conn(0x7f6d440ae9d0 0x7f6d4406e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:01.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.526+0000 7f6d4929c700 1 --2- 172.21.15.156:0/184194820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d4406e500 0x7f6d4415b9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009840 tx=0x7f6d34009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4e2b979ce76d898 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.526+0000 7f6d3bfff700 1 -- 172.21.15.156:0/184194820 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34011830 con 0x7f6d4406e500 2023-11-28T11:09:01.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.526+0000 7f6d3bfff700 1 -- 172.21.15.156:0/184194820 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d34011990 con 0x7f6d4406e500 2023-11-28T11:09:01.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.526+0000 7f6d3bfff700 1 -- 172.21.15.156:0/184194820 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34011d20 con 0x7f6d4406e500 2023-11-28T11:09:01.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.526+0000 7f6d4b500700 1 -- 172.21.15.156:0/184194820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d4406e500 msgr2=0x7f6d4415b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.526+0000 7f6d4b500700 1 --2- 172.21.15.156:0/184194820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d4406e500 0x7f6d4415b9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009840 tx=0x7f6d34009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:01.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.526+0000 7f6d4b500700 1 -- 172.21.15.156:0/184194820 shutdown_connections 2023-11-28T11:09:01.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.526+0000 7f6d4b500700 1 --2- 172.21.15.156:0/184194820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d4406e500 0x7f6d4415b9d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.526+0000 7f6d4b500700 1 --2- 172.21.15.156:0/184194820 >> v2:172.21.15.186:3300/0 conn(0x7f6d440ae9d0 0x7f6d4406e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.526+0000 7f6d4b500700 1 -- 172.21.15.156:0/184194820 >> 172.21.15.156:0/184194820 conn(0x7f6d440b12b0 msgr2=0x7f6d440b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.526+0000 7f6d4b500700 1 -- 172.21.15.156:0/184194820 shutdown_connections 2023-11-28T11:09:01.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.526+0000 7f6d4b500700 1 -- 172.21.15.156:0/184194820 wait complete. 2023-11-28T11:09:01.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.528+0000 7f6d4b500700 1 Processor -- start 2023-11-28T11:09:01.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.528+0000 7f6d4b500700 1 -- start start 2023-11-28T11:09:01.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.528+0000 7f6d4b500700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d4406e500 0x7f6d4414e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.528+0000 7f6d4b500700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440ae9d0 0x7f6d441e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.528+0000 7f6d4b500700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d4415e540 0x7f6d440b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.528+0000 7f6d4b500700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d44156190 con 0x7f6d4415e540 2023-11-28T11:09:01.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.528+0000 7f6d4b500700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d44156ff0 con 0x7f6d4406e500 2023-11-28T11:09:01.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.528+0000 7f6d4b500700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d441563e0 con 0x7f6d440ae9d0 2023-11-28T11:09:01.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d49a9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d4415e540 0x7f6d440b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d49a9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d4415e540 0x7f6d440b3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33280/0 (socket says 172.21.15.156:33280) 2023-11-28T11:09:01.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d49a9d700 1 -- 172.21.15.156:0/1576857823 learned_addr learned my addr 172.21.15.156:0/1576857823 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:01.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d4929c700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d4406e500 0x7f6d4414e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d48a9b700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440ae9d0 0x7f6d441e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d49a9d700 1 -- 172.21.15.156:0/1576857823 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440ae9d0 msgr2=0x7f6d441e4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d49a9d700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440ae9d0 0x7f6d441e4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d49a9d700 1 -- 172.21.15.156:0/1576857823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d4406e500 msgr2=0x7f6d4414e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d49a9d700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d4406e500 0x7f6d4414e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d49a9d700 1 -- 172.21.15.156:0/1576857823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d400094a0 con 0x7f6d4415e540 2023-11-28T11:09:01.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d49a9d700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d4415e540 0x7f6d440b3b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00cbc0 tx=0x7f6d3c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d39ffb700 1 -- 172.21.15.156:0/1576857823 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c00a9e0 con 0x7f6d4415e540 2023-11-28T11:09:01.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d39ffb700 1 -- 172.21.15.156:0/1576857823 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3c00ab40 con 0x7f6d4415e540 2023-11-28T11:09:01.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d4b500700 1 -- 172.21.15.156:0/1576857823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d340094a0 con 0x7f6d4415e540 2023-11-28T11:09:01.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.529+0000 7f6d39ffb700 1 -- 172.21.15.156:0/1576857823 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c012400 con 0x7f6d4415e540 2023-11-28T11:09:01.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.530+0000 7f6d4b500700 1 -- 172.21.15.156:0/1576857823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d440664c0 con 0x7f6d4415e540 2023-11-28T11:09:01.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.531+0000 7f6d4b500700 1 -- 172.21.15.156:0/1576857823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4406cc10 con 0x7f6d4415e540 2023-11-28T11:09:01.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.531+0000 7f6d39ffb700 1 -- 172.21.15.156:0/1576857823 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6d3c019030 con 0x7f6d4415e540 2023-11-28T11:09:01.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.531+0000 7f6d39ffb700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6d280682f0 0x7f6d2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.531+0000 7f6d39ffb700 1 -- 172.21.15.156:0/1576857823 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6d3c085660 con 0x7f6d4415e540 2023-11-28T11:09:01.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.531+0000 7f6d4929c700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6d280682f0 0x7f6d2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.531+0000 7f6d4929c700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6d280682f0 0x7f6d2806a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d34000c00 tx=0x7f6d3401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.535+0000 7f6d39ffb700 1 -- 172.21.15.156:0/1576857823 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6d3c04b5e0 con 0x7f6d4415e540 2023-11-28T11:09:01.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.717+0000 7f6d4b500700 1 -- 172.21.15.156:0/1576857823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6d4406cef0 con 0x7f6d4415e540 2023-11-28T11:09:01.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.717+0000 7f6d39ffb700 1 -- 172.21.15.156:0/1576857823 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6d3c051f30 con 0x7f6d4415e540 2023-11-28T11:09:01.719 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:09:01.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.720+0000 7f6d4b500700 1 -- 172.21.15.156:0/1576857823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6d280682f0 msgr2=0x7f6d2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.720+0000 7f6d4b500700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6d280682f0 0x7f6d2806a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d34000c00 tx=0x7f6d3401c040 comp rx=0 tx=0).stop 2023-11-28T11:09:01.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.720+0000 7f6d4b500700 1 -- 172.21.15.156:0/1576857823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d4415e540 msgr2=0x7f6d440b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.720+0000 7f6d4b500700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d4415e540 0x7f6d440b3b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00cbc0 tx=0x7f6d3c00ced0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.720+0000 7f6d4b500700 1 -- 172.21.15.156:0/1576857823 shutdown_connections 2023-11-28T11:09:01.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.720+0000 7f6d4b500700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d4415e540 0x7f6d440b3b90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.720+0000 7f6d4b500700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d4406e500 0x7f6d4414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.720+0000 7f6d4b500700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6d280682f0 0x7f6d2806a7a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.720+0000 7f6d4b500700 1 --2- 172.21.15.156:0/1576857823 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440ae9d0 0x7f6d441e4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.720+0000 7f6d4b500700 1 -- 172.21.15.156:0/1576857823 >> 172.21.15.156:0/1576857823 conn(0x7f6d440b12b0 msgr2=0x7f6d4406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.720+0000 7f6d4b500700 1 -- 172.21.15.156:0/1576857823 shutdown_connections 2023-11-28T11:09:01.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.720+0000 7f6d4b500700 1 -- 172.21.15.156:0/1576857823 wait complete. 2023-11-28T11:09:01.733 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:01.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.856+0000 7f47de041700 1 Processor -- start 2023-11-28T11:09:01.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.856+0000 7f47de041700 1 -- start start 2023-11-28T11:09:01.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47de041700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f47d806dee0 0x7f47d806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47de041700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f47d8160540 0x7f47d8160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47de041700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d815d060 con 0x7f47d8071040 2023-11-28T11:09:01.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47de041700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d815d340 con 0x7f47d8160540 2023-11-28T11:09:01.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47de041700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d815d620 con 0x7f47d806dee0 2023-11-28T11:09:01.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47d7fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f47d8160540 0x7f47d8160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47d77fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f47d8071040 0x7f47d8071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48162/0 (socket says 172.21.15.156:48162) 2023-11-28T11:09:01.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47d77fe700 1 -- 172.21.15.156:0/3414374541 learned_addr learned my addr 172.21.15.156:0/3414374541 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:01.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47d6ffd700 1 --2- 172.21.15.156:0/3414374541 >> v2:172.21.15.186:3300/0 conn(0x7f47d806dee0 0x7f47d806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47d7fff700 1 -- 172.21.15.156:0/3414374541 >> v2:172.21.15.186:3300/0 conn(0x7f47d806dee0 msgr2=0x7f47d806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47d7fff700 1 --2- 172.21.15.156:0/3414374541 >> v2:172.21.15.186:3300/0 conn(0x7f47d806dee0 0x7f47d806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47d7fff700 1 -- 172.21.15.156:0/3414374541 >> v1:172.21.15.156:6789/0 conn(0x7f47d8071040 legacy=0x7f47d8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.857+0000 7f47d67fc700 1 -- 172.21.15.156:0/3414374541 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993802184 0 0) 0x7f47d815d060 con 0x7f47d8071040 2023-11-28T11:09:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.858+0000 7f47d7fff700 1 -- 172.21.15.156:0/3414374541 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d815d900 con 0x7f47d8160540 2023-11-28T11:09:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.858+0000 7f47d7fff700 1 --2- 172.21.15.156:0/3414374541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d8160540 0x7f47d8160980 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f47d806fb00 tx=0x7f47d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fc78a116b703d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.858+0000 7f47d67fc700 1 -- 172.21.15.156:0/3414374541 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d000e070 con 0x7f47d8160540 2023-11-28T11:09:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.858+0000 7f47d67fc700 1 -- 172.21.15.156:0/3414374541 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d000bbd0 con 0x7f47d8160540 2023-11-28T11:09:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.858+0000 7f47d67fc700 1 -- 172.21.15.156:0/3414374541 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0012430 con 0x7f47d8160540 2023-11-28T11:09:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.859+0000 7f47de041700 1 -- 172.21.15.156:0/3414374541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d8160540 msgr2=0x7f47d8160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.859+0000 7f47de041700 1 --2- 172.21.15.156:0/3414374541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d8160540 0x7f47d8160980 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f47d806fb00 tx=0x7f47d000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.859+0000 7f47de041700 1 -- 172.21.15.156:0/3414374541 shutdown_connections 2023-11-28T11:09:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.859+0000 7f47de041700 1 --2- 172.21.15.156:0/3414374541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d8160540 0x7f47d8160980 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.859+0000 7f47de041700 1 --2- 172.21.15.156:0/3414374541 >> v2:172.21.15.186:3300/0 conn(0x7f47d806dee0 0x7f47d806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.859+0000 7f47de041700 1 -- 172.21.15.156:0/3414374541 >> 172.21.15.156:0/3414374541 conn(0x7f47d80d0000 msgr2=0x7f47d80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.859+0000 7f47de041700 1 -- 172.21.15.156:0/3414374541 shutdown_connections 2023-11-28T11:09:01.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.859+0000 7f47de041700 1 -- 172.21.15.156:0/3414374541 wait complete. 2023-11-28T11:09:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.860+0000 7f47de041700 1 Processor -- start 2023-11-28T11:09:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.860+0000 7f47de041700 1 -- start start 2023-11-28T11:09:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.861+0000 7f47de041700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d806dee0 0x7f47d81537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.861+0000 7f47de041700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47d8071040 0x7f47d80c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.861+0000 7f47de041700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47d8160540 0x7f47d81f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.861+0000 7f47de041700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d81f2ff0 con 0x7f47d806dee0 2023-11-28T11:09:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.861+0000 7f47de041700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d81f32d0 con 0x7f47d8071040 2023-11-28T11:09:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.861+0000 7f47de041700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d81f35b0 con 0x7f47d8160540 2023-11-28T11:09:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47d77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d806dee0 0x7f47d81537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47d77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d806dee0 0x7f47d81537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33290/0 (socket says 172.21.15.156:33290) 2023-11-28T11:09:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47d77fe700 1 -- 172.21.15.156:0/3892837794 learned_addr learned my addr 172.21.15.156:0/3892837794 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:01.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47d7fff700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47d8160540 0x7f47d81f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47d77fe700 1 -- 172.21.15.156:0/3892837794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47d8160540 msgr2=0x7f47d81f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47d77fe700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47d8160540 0x7f47d81f2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47d77fe700 1 -- 172.21.15.156:0/3892837794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47d8071040 msgr2=0x7f47d80c58f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T11:09:01.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47d77fe700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47d8071040 0x7f47d80c58f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47d77fe700 1 -- 172.21.15.156:0/3892837794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d0009a70 con 0x7f47d806dee0 2023-11-28T11:09:01.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47d77fe700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d806dee0 0x7f47d81537f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00c550 tx=0x7f47cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47bffff700 1 -- 172.21.15.156:0/3892837794 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc004c80 con 0x7f47d806dee0 2023-11-28T11:09:01.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47bffff700 1 -- 172.21.15.156:0/3892837794 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc004e00 con 0x7f47d806dee0 2023-11-28T11:09:01.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47de041700 1 -- 172.21.15.156:0/3892837794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d815ccd0 con 0x7f47d806dee0 2023-11-28T11:09:01.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47bffff700 1 -- 172.21.15.156:0/3892837794 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc007310 con 0x7f47d806dee0 2023-11-28T11:09:01.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.862+0000 7f47de041700 1 -- 172.21.15.156:0/3892837794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d806e870 con 0x7f47d806dee0 2023-11-28T11:09:01.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.863+0000 7f47de041700 1 -- 172.21.15.156:0/3892837794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d806a270 con 0x7f47d806dee0 2023-11-28T11:09:01.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.865+0000 7f47bffff700 1 -- 172.21.15.156:0/3892837794 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f47cc01a020 con 0x7f47d806dee0 2023-11-28T11:09:01.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.866+0000 7f47bffff700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f47c40682f0 0x7f47c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.866+0000 7f47bffff700 1 -- 172.21.15.156:0/3892837794 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f47cc085ee0 con 0x7f47d806dee0 2023-11-28T11:09:01.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.866+0000 7f47d6ffd700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f47c40682f0 0x7f47c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.866+0000 7f47d6ffd700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f47c40682f0 0x7f47c406a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f47d80cead0 tx=0x7f47c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:01.869+0000 7f47bffff700 1 -- 172.21.15.156:0/3892837794 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f47cc052720 con 0x7f47d806dee0 2023-11-28T11:09:02.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.054+0000 7f47de041700 1 -- 172.21.15.156:0/3892837794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47d80c68b0 con 0x7f47d806dee0 2023-11-28T11:09:02.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.054+0000 7f47bffff700 1 -- 172.21.15.156:0/3892837794 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f47cc055d40 con 0x7f47d806dee0 2023-11-28T11:09:02.055 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:09:02.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.056+0000 7f47de041700 1 -- 172.21.15.156:0/3892837794 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f47c40682f0 msgr2=0x7f47c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.056+0000 7f47de041700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f47c40682f0 0x7f47c406a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f47d80cead0 tx=0x7f47c8008040 comp rx=0 tx=0).stop 2023-11-28T11:09:02.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.056+0000 7f47de041700 1 -- 172.21.15.156:0/3892837794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d806dee0 msgr2=0x7f47d81537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.056+0000 7f47de041700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d806dee0 0x7f47d81537f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00c550 tx=0x7f47cc00c910 comp rx=0 tx=0).stop 2023-11-28T11:09:02.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.056+0000 7f47de041700 1 -- 172.21.15.156:0/3892837794 shutdown_connections 2023-11-28T11:09:02.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.056+0000 7f47de041700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f47c40682f0 0x7f47c406a7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.056+0000 7f47de041700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47d8160540 0x7f47d81f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.056+0000 7f47de041700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d806dee0 0x7f47d81537f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.056+0000 7f47de041700 1 --2- 172.21.15.156:0/3892837794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47d8071040 0x7f47d80c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.056+0000 7f47de041700 1 -- 172.21.15.156:0/3892837794 >> 172.21.15.156:0/3892837794 conn(0x7f47d80d0000 msgr2=0x7f47d815e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:02.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.056+0000 7f47de041700 1 -- 172.21.15.156:0/3892837794 shutdown_connections 2023-11-28T11:09:02.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.056+0000 7f47de041700 1 -- 172.21.15.156:0/3892837794 wait complete. 2023-11-28T11:09:02.069 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:02.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.191+0000 7f43adfec700 1 Processor -- start 2023-11-28T11:09:02.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.192+0000 7f43adfec700 1 -- start start 2023-11-28T11:09:02.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.192+0000 7f43adfec700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f43a8140950 0x7f43a80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.192+0000 7f43adfec700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43a80b2e70 0x7f43a806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.192+0000 7f43adfec700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a813ff00 con 0x7f43a80cd8c0 2023-11-28T11:09:02.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.192+0000 7f43adfec700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a81404c0 con 0x7f43a8140950 2023-11-28T11:09:02.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.192+0000 7f43adfec700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8140dc0 con 0x7f43a80b2e70 2023-11-28T11:09:02.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a6ffd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f43a8140950 0x7f43a80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a7fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43a80b2e70 0x7f43a806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a6ffd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f43a8140950 0x7f43a80b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:44314/0 (socket says 172.21.15.156:44314) 2023-11-28T11:09:02.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a6ffd700 1 -- 172.21.15.156:0/4043632083 learned_addr learned my addr 172.21.15.156:0/4043632083 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:02.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a77fe700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f43a80cd8c0 0x7f43a80cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:58536/0 (socket says 172.21.15.156:58536) 2023-11-28T11:09:02.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a7fff700 1 -- 172.21.15.156:0/4043632083 >> v2:172.21.15.186:3300/0 conn(0x7f43a8140950 msgr2=0x7f43a80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:02.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a7fff700 1 --2- 172.21.15.156:0/4043632083 >> v2:172.21.15.186:3300/0 conn(0x7f43a8140950 0x7f43a80b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a7fff700 1 -- 172.21.15.156:0/4043632083 >> v1:172.21.15.186:6789/0 conn(0x7f43a80cd8c0 legacy=0x7f43a80cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a67fc700 1 -- 172.21.15.156:0/4043632083 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184266620 0 0) 0x7f43a813ff00 con 0x7f43a80cd8c0 2023-11-28T11:09:02.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a7fff700 1 -- 172.21.15.156:0/4043632083 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a815aa80 con 0x7f43a80b2e70 2023-11-28T11:09:02.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a6ffd700 1 --2- 172.21.15.156:0/4043632083 >> v2:172.21.15.186:3300/0 conn(0x7f43a8140950 0x7f43a80b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:02.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a7fff700 1 --2- 172.21.15.156:0/4043632083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43a80b2e70 0x7f43a806f850 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009d90 tx=0x7f43a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a115c854b5d1d859 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:02.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a67fc700 1 -- 172.21.15.156:0/4043632083 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a000e070 con 0x7f43a80b2e70 2023-11-28T11:09:02.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a67fc700 1 -- 172.21.15.156:0/4043632083 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a000bb20 con 0x7f43a80b2e70 2023-11-28T11:09:02.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.193+0000 7f43a67fc700 1 -- 172.21.15.156:0/4043632083 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0012430 con 0x7f43a80b2e70 2023-11-28T11:09:02.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.194+0000 7f43adfec700 1 -- 172.21.15.156:0/4043632083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43a80b2e70 msgr2=0x7f43a806f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.194+0000 7f43adfec700 1 --2- 172.21.15.156:0/4043632083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43a80b2e70 0x7f43a806f850 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009d90 tx=0x7f43a000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:02.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.194+0000 7f43adfec700 1 -- 172.21.15.156:0/4043632083 shutdown_connections 2023-11-28T11:09:02.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.194+0000 7f43adfec700 1 --2- 172.21.15.156:0/4043632083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43a80b2e70 0x7f43a806f850 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.194+0000 7f43adfec700 1 --2- 172.21.15.156:0/4043632083 >> v2:172.21.15.186:3300/0 conn(0x7f43a8140950 0x7f43a80b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.194+0000 7f43adfec700 1 -- 172.21.15.156:0/4043632083 >> 172.21.15.156:0/4043632083 conn(0x7f43a806aed0 msgr2=0x7f43a806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:02.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.194+0000 7f43adfec700 1 -- 172.21.15.156:0/4043632083 shutdown_connections 2023-11-28T11:09:02.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.194+0000 7f43adfec700 1 -- 172.21.15.156:0/4043632083 wait complete. 2023-11-28T11:09:02.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.196+0000 7f43adfec700 1 Processor -- start 2023-11-28T11:09:02.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.196+0000 7f43adfec700 1 -- start start 2023-11-28T11:09:02.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43adfec700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43a80b2e70 0x7f43a80c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43adfec700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43a80cd8c0 0x7f43a80c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43adfec700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a8140950 0x7f43a8157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43adfec700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80c6710 con 0x7f43a8140950 2023-11-28T11:09:02.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43adfec700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80c7070 con 0x7f43a80b2e70 2023-11-28T11:09:02.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43adfec700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80c5a90 con 0x7f43a80cd8c0 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43a7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a8140950 0x7f43a8157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43a7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a8140950 0x7f43a8157120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33308/0 (socket says 172.21.15.156:33308) 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43a7fff700 1 -- 172.21.15.156:0/1913337188 learned_addr learned my addr 172.21.15.156:0/1913337188 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43a77fe700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43a80b2e70 0x7f43a80c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43a6ffd700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43a80cd8c0 0x7f43a80c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43a7fff700 1 -- 172.21.15.156:0/1913337188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43a80cd8c0 msgr2=0x7f43a80c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43a7fff700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43a80cd8c0 0x7f43a80c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43a7fff700 1 -- 172.21.15.156:0/1913337188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43a80b2e70 msgr2=0x7f43a80c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43a7fff700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43a80b2e70 0x7f43a80c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43a7fff700 1 -- 172.21.15.156:0/1913337188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4398009550 con 0x7f43a8140950 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f43a7fff700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a8140950 0x7f43a8157120 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f43a0014040 tx=0x7f43a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.197+0000 7f438ffff700 1 -- 172.21.15.156:0/1913337188 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0012430 con 0x7f43a8140950 2023-11-28T11:09:02.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.198+0000 7f438ffff700 1 -- 172.21.15.156:0/1913337188 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a0020070 con 0x7f43a8140950 2023-11-28T11:09:02.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.198+0000 7f43adfec700 1 -- 172.21.15.156:0/1913337188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a0009a70 con 0x7f43a8140950 2023-11-28T11:09:02.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.198+0000 7f438ffff700 1 -- 172.21.15.156:0/1913337188 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0004280 con 0x7f43a8140950 2023-11-28T11:09:02.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.198+0000 7f43adfec700 1 -- 172.21.15.156:0/1913337188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a80b35f0 con 0x7f43a8140950 2023-11-28T11:09:02.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.199+0000 7f43adfec700 1 -- 172.21.15.156:0/1913337188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a80b3310 con 0x7f43a8140950 2023-11-28T11:09:02.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.199+0000 7f438ffff700 1 -- 172.21.15.156:0/1913337188 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f43a002e070 con 0x7f43a8140950 2023-11-28T11:09:02.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.199+0000 7f438ffff700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4394068320 0x7f439406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.200+0000 7f438ffff700 1 -- 172.21.15.156:0/1913337188 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f43a0022070 con 0x7f43a8140950 2023-11-28T11:09:02.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.200+0000 7f43a77fe700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4394068320 0x7f439406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.200+0000 7f43a77fe700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4394068320 0x7f439406a7d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f439c000fb0 tx=0x7f439c007d70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:02.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.204+0000 7f438ffff700 1 -- 172.21.15.156:0/1913337188 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f43a00540a0 con 0x7f43a8140950 2023-11-28T11:09:02.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.386+0000 7f43adfec700 1 -- 172.21.15.156:0/1913337188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43a80c6850 con 0x7f43a8140950 2023-11-28T11:09:02.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.386+0000 7f438ffff700 1 -- 172.21.15.156:0/1913337188 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43a00576c0 con 0x7f43a8140950 2023-11-28T11:09:02.387 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.388+0000 7f43adfec700 1 -- 172.21.15.156:0/1913337188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4394068320 msgr2=0x7f439406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.388+0000 7f43adfec700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4394068320 0x7f439406a7d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f439c000fb0 tx=0x7f439c007d70 comp rx=0 tx=0).stop 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.388+0000 7f43adfec700 1 -- 172.21.15.156:0/1913337188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a8140950 msgr2=0x7f43a8157120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.388+0000 7f43adfec700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a8140950 0x7f43a8157120 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f43a0014040 tx=0x7f43a0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.388+0000 7f43adfec700 1 -- 172.21.15.156:0/1913337188 shutdown_connections 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.388+0000 7f43adfec700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a8140950 0x7f43a8157120 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.388+0000 7f43adfec700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43a80b2e70 0x7f43a80c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.388+0000 7f43adfec700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4394068320 0x7f439406a7d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.388+0000 7f43adfec700 1 --2- 172.21.15.156:0/1913337188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43a80cd8c0 0x7f43a80c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.388+0000 7f43adfec700 1 -- 172.21.15.156:0/1913337188 >> 172.21.15.156:0/1913337188 conn(0x7f43a806aed0 msgr2=0x7f43a80d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:02.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.388+0000 7f43adfec700 1 -- 172.21.15.156:0/1913337188 shutdown_connections 2023-11-28T11:09:02.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.388+0000 7f43adfec700 1 -- 172.21.15.156:0/1913337188 wait complete. 2023-11-28T11:09:02.402 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:02.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.522+0000 7ff743b5c700 1 Processor -- start 2023-11-28T11:09:02.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.523+0000 7ff743b5c700 1 -- start start 2023-11-28T11:09:02.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff743b5c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff73c0cb0e0 0x7ff73c065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff743b5c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73c066580 con 0x7ff73c06ec90 2023-11-28T11:09:02.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff743b5c700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73c0cded0 con 0x7ff73c0cec70 2023-11-28T11:09:02.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff743b5c700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73c0ce1b0 con 0x7ff73c0cb0e0 2023-11-28T11:09:02.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff7418f8700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff73c06ec90 0x7ff73c0ca9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48190/0 (socket says 172.21.15.156:48190) 2023-11-28T11:09:02.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff7418f8700 1 -- 172.21.15.156:0/2060252811 learned_addr learned my addr 172.21.15.156:0/2060252811 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:02.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff7410f7700 1 --2- 172.21.15.156:0/2060252811 >> v2:172.21.15.156:3300/0 conn(0x7ff73c0cb0e0 0x7ff73c065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff7408f6700 1 -- 172.21.15.156:0/2060252811 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058865783 0 0) 0x7ff73c066580 con 0x7ff73c06ec90 2023-11-28T11:09:02.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff7408f6700 1 -- 172.21.15.156:0/2060252811 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7280035b0 con 0x7ff73c06ec90 2023-11-28T11:09:02.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff7410f7700 1 -- 172.21.15.156:0/2060252811 >> v1:172.21.15.186:6789/0 conn(0x7ff73c0cec70 legacy=0x7ff73c06d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:02.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff7410f7700 1 -- 172.21.15.156:0/2060252811 >> v1:172.21.15.156:6789/0 conn(0x7ff73c06ec90 legacy=0x7ff73c0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff7410f7700 1 -- 172.21.15.156:0/2060252811 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c0ce490 con 0x7ff73c0cb0e0 2023-11-28T11:09:02.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.524+0000 7ff7410f7700 1 --2- 172.21.15.156:0/2060252811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cb0e0 0x7ff73c065ef0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009840 tx=0x7ff72c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be2c0705806f957 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:02.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.525+0000 7ff7408f6700 1 -- 172.21.15.156:0/2060252811 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c004230 con 0x7ff73c0cb0e0 2023-11-28T11:09:02.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.525+0000 7ff7408f6700 1 -- 172.21.15.156:0/2060252811 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c004390 con 0x7ff73c0cb0e0 2023-11-28T11:09:02.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.525+0000 7ff7408f6700 1 -- 172.21.15.156:0/2060252811 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c0036c0 con 0x7ff73c0cb0e0 2023-11-28T11:09:02.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.525+0000 7ff743b5c700 1 -- 172.21.15.156:0/2060252811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cb0e0 msgr2=0x7ff73c065ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.525+0000 7ff743b5c700 1 --2- 172.21.15.156:0/2060252811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cb0e0 0x7ff73c065ef0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009840 tx=0x7ff72c009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:02.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.525+0000 7ff743b5c700 1 -- 172.21.15.156:0/2060252811 shutdown_connections 2023-11-28T11:09:02.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.525+0000 7ff743b5c700 1 --2- 172.21.15.156:0/2060252811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cb0e0 0x7ff73c065ef0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.525+0000 7ff743b5c700 1 -- 172.21.15.156:0/2060252811 >> 172.21.15.156:0/2060252811 conn(0x7ff73c0b12b0 msgr2=0x7ff73c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:02.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.525+0000 7ff743b5c700 1 -- 172.21.15.156:0/2060252811 shutdown_connections 2023-11-28T11:09:02.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.525+0000 7ff743b5c700 1 -- 172.21.15.156:0/2060252811 wait complete. 2023-11-28T11:09:02.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.526+0000 7ff743b5c700 1 Processor -- start 2023-11-28T11:09:02.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.526+0000 7ff743b5c700 1 -- start start 2023-11-28T11:09:02.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff743b5c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff73c06ec90 0x7ff73c14ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff743b5c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff73c0cb0e0 0x7ff73c0adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff743b5c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cec70 0x7ff73c158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff743b5c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c0ad7b0 con 0x7ff73c0cec70 2023-11-28T11:09:02.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff743b5c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c0acba0 con 0x7ff73c0cb0e0 2023-11-28T11:09:02.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff743b5c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c0ace80 con 0x7ff73c06ec90 2023-11-28T11:09:02.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff7420f9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cec70 0x7ff73c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff7420f9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cec70 0x7ff73c158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33322/0 (socket says 172.21.15.156:33322) 2023-11-28T11:09:02.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff7420f9700 1 -- 172.21.15.156:0/3275440776 learned_addr learned my addr 172.21.15.156:0/3275440776 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:02.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff7410f7700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff73c0cb0e0 0x7ff73c0adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff7418f8700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff73c06ec90 0x7ff73c14ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff7420f9700 1 -- 172.21.15.156:0/3275440776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff73c06ec90 msgr2=0x7ff73c14ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff7420f9700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff73c06ec90 0x7ff73c14ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff7420f9700 1 -- 172.21.15.156:0/3275440776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff73c0cb0e0 msgr2=0x7ff73c0adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.527+0000 7ff7420f9700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff73c0cb0e0 0x7ff73c0adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.528+0000 7ff7420f9700 1 -- 172.21.15.156:0/3275440776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c0094a0 con 0x7ff73c0cec70 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.528+0000 7ff7418f8700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff73c06ec90 0x7ff73c14ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.528+0000 7ff7420f9700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cec70 0x7ff73c158700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff73400cc10 tx=0x7ff73400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.528+0000 7ff7327fc700 1 -- 172.21.15.156:0/3275440776 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73400ab10 con 0x7ff73c0cec70 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.528+0000 7ff7327fc700 1 -- 172.21.15.156:0/3275440776 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73400ac70 con 0x7ff73c0cec70 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.528+0000 7ff743b5c700 1 -- 172.21.15.156:0/3275440776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c0cb6c0 con 0x7ff73c0cec70 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.528+0000 7ff7327fc700 1 -- 172.21.15.156:0/3275440776 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734012420 con 0x7ff73c0cec70 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.528+0000 7ff743b5c700 1 -- 172.21.15.156:0/3275440776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c0ce770 con 0x7ff73c0cec70 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.529+0000 7ff743b5c700 1 -- 172.21.15.156:0/3275440776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73c06f060 con 0x7ff73c0cec70 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.529+0000 7ff7327fc700 1 -- 172.21.15.156:0/3275440776 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff734010050 con 0x7ff73c0cec70 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.530+0000 7ff7327fc700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff7200682f0 0x7ff72006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.530+0000 7ff7327fc700 1 -- 172.21.15.156:0/3275440776 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff734085f30 con 0x7ff73c0cec70 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.530+0000 7ff7418f8700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff7200682f0 0x7ff72006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.530+0000 7ff7418f8700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff7200682f0 0x7ff72006a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff738005910 tx=0x7ff738009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:02.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.534+0000 7ff7327fc700 1 -- 172.21.15.156:0/3275440776 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff734052750 con 0x7ff73c0cec70 2023-11-28T11:09:02.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.758+0000 7ff743b5c700 1 -- 172.21.15.156:0/3275440776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff73c0cf110 con 0x7ff73c0cec70 2023-11-28T11:09:02.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.759+0000 7ff7327fc700 1 -- 172.21.15.156:0/3275440776 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7ff734055d70 con 0x7ff73c0cec70 2023-11-28T11:09:02.760 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:02.760 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1221397150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1221397150},{"type":"v1","addr":"172.21.15.186:6834","nonce":1221397150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/22494485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":22494485},{"type":"v1","addr":"172.21.15.156:6835","nonce":22494485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3382922708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3382922708},{"type":"v1","addr":"172.21.15.156:6837","nonce":3382922708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:17.352335+0000","modified":"2023-11-28T11:08:20.752191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/4134790300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":4134790300},{"type":"v1","addr":"172.21.15.186:6835","nonce":4134790300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.761+0000 7ff743b5c700 1 -- 172.21.15.156:0/3275440776 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff7200682f0 msgr2=0x7ff72006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.761+0000 7ff743b5c700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff7200682f0 0x7ff72006a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff738005910 tx=0x7ff738009040 comp rx=0 tx=0).stop 2023-11-28T11:09:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.761+0000 7ff743b5c700 1 -- 172.21.15.156:0/3275440776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cec70 msgr2=0x7ff73c158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.761+0000 7ff743b5c700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cec70 0x7ff73c158700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff73400cc10 tx=0x7ff73400cf20 comp rx=0 tx=0).stop 2023-11-28T11:09:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.761+0000 7ff743b5c700 1 -- 172.21.15.156:0/3275440776 shutdown_connections 2023-11-28T11:09:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.761+0000 7ff743b5c700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cec70 0x7ff73c158700 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.761+0000 7ff743b5c700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff7200682f0 0x7ff72006a7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.761+0000 7ff743b5c700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff73c06ec90 0x7ff73c14ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.761+0000 7ff743b5c700 1 --2- 172.21.15.156:0/3275440776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff73c0cb0e0 0x7ff73c0adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.761+0000 7ff743b5c700 1 -- 172.21.15.156:0/3275440776 >> 172.21.15.156:0/3275440776 conn(0x7ff73c0b12b0 msgr2=0x7ff73c0aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.761+0000 7ff743b5c700 1 -- 172.21.15.156:0/3275440776 shutdown_connections 2023-11-28T11:09:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.761+0000 7ff743b5c700 1 -- 172.21.15.156:0/3275440776 wait complete. 2023-11-28T11:09:02.764 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 8 2023-11-28T11:09:02.775 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:02.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.895+0000 7f22e92e6700 1 Processor -- start 2023-11-28T11:09:02.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.895+0000 7f22e92e6700 1 -- start start 2023-11-28T11:09:02.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.896+0000 7f22e92e6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e40cd8c0 0x7f22e4140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.896+0000 7f22e92e6700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f22e40cdc90 0x7f22e40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.896+0000 7f22e92e6700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f22e40b2e70 0x7f22e406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.896+0000 7f22e92e6700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e413ff00 con 0x7f22e40b2e70 2023-11-28T11:09:02.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.896+0000 7f22e92e6700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e41404c0 con 0x7f22e40cdc90 2023-11-28T11:09:02.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.896+0000 7f22e92e6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e415aa80 con 0x7f22e40cd8c0 2023-11-28T11:09:02.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.896+0000 7f22e37fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f22e40b2e70 0x7f22e406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.896+0000 7f22e37fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f22e40b2e70 0x7f22e406f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33328/0 (socket says 172.21.15.156:33328) 2023-11-28T11:09:02.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.896+0000 7f22e37fe700 1 -- 172.21.15.156:0/1820124294 learned_addr learned my addr 172.21.15.156:0/1820124294 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:02.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.896+0000 7f22e27fc700 1 --2- 172.21.15.156:0/1820124294 >> v2:172.21.15.186:3300/0 conn(0x7f22e40cdc90 0x7f22e40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.896+0000 7f22e2ffd700 1 --2- 172.21.15.156:0/1820124294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e40cd8c0 0x7f22e4140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e37fe700 1 -- 172.21.15.156:0/1820124294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e40cd8c0 msgr2=0x7f22e4140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:02.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e37fe700 1 --2- 172.21.15.156:0/1820124294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e40cd8c0 0x7f22e4140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e37fe700 1 -- 172.21.15.156:0/1820124294 >> v2:172.21.15.186:3300/0 conn(0x7f22e40cdc90 msgr2=0x7f22e40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:02.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e37fe700 1 --2- 172.21.15.156:0/1820124294 >> v2:172.21.15.186:3300/0 conn(0x7f22e40cdc90 0x7f22e40b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e37fe700 1 -- 172.21.15.156:0/1820124294 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40b3310 con 0x7f22e40b2e70 2023-11-28T11:09:02.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e37fe700 1 --2- 172.21.15.156:0/1820124294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22e40b2e70 0x7f22e406f850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f22e4067c60 tx=0x7f22dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af22795f4415b4f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:02.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e1ffb700 1 -- 172.21.15.156:0/1820124294 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00e070 con 0x7f22e40b2e70 2023-11-28T11:09:02.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e1ffb700 1 -- 172.21.15.156:0/1820124294 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc00b9c0 con 0x7f22e40b2e70 2023-11-28T11:09:02.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e1ffb700 1 -- 172.21.15.156:0/1820124294 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00bd70 con 0x7f22e40b2e70 2023-11-28T11:09:02.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e92e6700 1 -- 172.21.15.156:0/1820124294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22e40b2e70 msgr2=0x7f22e406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e92e6700 1 --2- 172.21.15.156:0/1820124294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22e40b2e70 0x7f22e406f850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f22e4067c60 tx=0x7f22dc00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:02.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e92e6700 1 -- 172.21.15.156:0/1820124294 shutdown_connections 2023-11-28T11:09:02.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e92e6700 1 --2- 172.21.15.156:0/1820124294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22e40b2e70 0x7f22e406f850 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e92e6700 1 --2- 172.21.15.156:0/1820124294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e40cd8c0 0x7f22e4140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e92e6700 1 --2- 172.21.15.156:0/1820124294 >> v2:172.21.15.186:3300/0 conn(0x7f22e40cdc90 0x7f22e40b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.897+0000 7f22e92e6700 1 -- 172.21.15.156:0/1820124294 >> 172.21.15.156:0/1820124294 conn(0x7f22e406aed0 msgr2=0x7f22e406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:02.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.898+0000 7f22e92e6700 1 -- 172.21.15.156:0/1820124294 shutdown_connections 2023-11-28T11:09:02.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.898+0000 7f22e92e6700 1 -- 172.21.15.156:0/1820124294 wait complete. 2023-11-28T11:09:02.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.899+0000 7f22e92e6700 1 Processor -- start 2023-11-28T11:09:02.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.899+0000 7f22e92e6700 1 -- start start 2023-11-28T11:09:02.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.899+0000 7f22e92e6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e40b2e70 0x7f22e413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.899+0000 7f22e92e6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22e40cd8c0 0x7f22e413d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e92e6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e40cdc90 0x7f22e40ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e92e6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40cf0b0 con 0x7f22e40cd8c0 2023-11-28T11:09:02.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e92e6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40cfc30 con 0x7f22e40b2e70 2023-11-28T11:09:02.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e92e6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40cf300 con 0x7f22e40cdc90 2023-11-28T11:09:02.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e27fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22e40cd8c0 0x7f22e413d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e27fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22e40cd8c0 0x7f22e413d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33332/0 (socket says 172.21.15.156:33332) 2023-11-28T11:09:02.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e27fc700 1 -- 172.21.15.156:0/52404779 learned_addr learned my addr 172.21.15.156:0/52404779 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:02.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e37fe700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e40cdc90 0x7f22e40ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e2ffd700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e40b2e70 0x7f22e413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e27fc700 1 -- 172.21.15.156:0/52404779 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e40cdc90 msgr2=0x7f22e40ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e27fc700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e40cdc90 0x7f22e40ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e27fc700 1 -- 172.21.15.156:0/52404779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e40b2e70 msgr2=0x7f22e413ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e27fc700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e40b2e70 0x7f22e413ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e27fc700 1 -- 172.21.15.156:0/52404779 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc009b30 con 0x7f22e40cd8c0 2023-11-28T11:09:02.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e27fc700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22e40cd8c0 0x7f22e413d230 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f22d400e7e0 tx=0x7f22d400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:02.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22cb7fe700 1 -- 172.21.15.156:0/52404779 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4013610 con 0x7f22e40cd8c0 2023-11-28T11:09:02.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22cb7fe700 1 -- 172.21.15.156:0/52404779 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d4013770 con 0x7f22e40cd8c0 2023-11-28T11:09:02.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e92e6700 1 -- 172.21.15.156:0/52404779 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e4140150 con 0x7f22e40cd8c0 2023-11-28T11:09:02.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22cb7fe700 1 -- 172.21.15.156:0/52404779 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4013ae0 con 0x7f22e40cd8c0 2023-11-28T11:09:02.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.900+0000 7f22e92e6700 1 -- 172.21.15.156:0/52404779 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e40b35f0 con 0x7f22e40cd8c0 2023-11-28T11:09:02.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.901+0000 7f22e92e6700 1 -- 172.21.15.156:0/52404779 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e40ce100 con 0x7f22e40cd8c0 2023-11-28T11:09:02.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.902+0000 7f22cb7fe700 1 -- 172.21.15.156:0/52404779 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f22d4023070 con 0x7f22e40cd8c0 2023-11-28T11:09:02.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.902+0000 7f22cb7fe700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f22d00682f0 0x7f22d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.902+0000 7f22e2ffd700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f22d00682f0 0x7f22d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.902+0000 7f22cb7fe700 1 -- 172.21.15.156:0/52404779 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f22d40858e0 con 0x7f22e40cd8c0 2023-11-28T11:09:02.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.902+0000 7f22e2ffd700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f22d00682f0 0x7f22d006a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f22e40d1600 tx=0x7f22d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:02.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:02.906+0000 7f22cb7fe700 1 -- 172.21.15.156:0/52404779 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f22d4052100 con 0x7f22e40cd8c0 2023-11-28T11:09:03.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.091+0000 7f22e92e6700 1 -- 172.21.15.156:0/52404779 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22e40d08c0 con 0x7f22e40cd8c0 2023-11-28T11:09:03.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.092+0000 7f22cb7fe700 1 -- 172.21.15.156:0/52404779 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18050 (secure 0 0 0) 0x7f22d4055720 con 0x7f22e40cd8c0 2023-11-28T11:09:03.093 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:03.093 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":25,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:01.859199+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:03.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.094+0000 7f22e92e6700 1 -- 172.21.15.156:0/52404779 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f22d00682f0 msgr2=0x7f22d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.094+0000 7f22e92e6700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f22d00682f0 0x7f22d006a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f22e40d1600 tx=0x7f22d8008040 comp rx=0 tx=0).stop 2023-11-28T11:09:03.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.094+0000 7f22e92e6700 1 -- 172.21.15.156:0/52404779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22e40cd8c0 msgr2=0x7f22e413d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.094+0000 7f22e92e6700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22e40cd8c0 0x7f22e413d230 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f22d400e7e0 tx=0x7f22d400eba0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.094+0000 7f22e92e6700 1 -- 172.21.15.156:0/52404779 shutdown_connections 2023-11-28T11:09:03.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.094+0000 7f22e92e6700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f22d00682f0 0x7f22d006a7a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.094+0000 7f22e92e6700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e40cdc90 0x7f22e40ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.094+0000 7f22e92e6700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e40b2e70 0x7f22e413ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.094+0000 7f22e92e6700 1 --2- 172.21.15.156:0/52404779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22e40cd8c0 0x7f22e413d230 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.094+0000 7f22e92e6700 1 -- 172.21.15.156:0/52404779 >> 172.21.15.156:0/52404779 conn(0x7f22e406aed0 msgr2=0x7f22e4158f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:03.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.094+0000 7f22e92e6700 1 -- 172.21.15.156:0/52404779 shutdown_connections 2023-11-28T11:09:03.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.094+0000 7f22e92e6700 1 -- 172.21.15.156:0/52404779 wait complete. 2023-11-28T11:09:03.108 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:09:03.108 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:03.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.228+0000 7f2e241cc700 1 Processor -- start 2023-11-28T11:09:03.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.228+0000 7f2e241cc700 1 -- start start 2023-11-28T11:09:03.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e241cc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2e1c06e500 0x7f2e1c15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e241cc700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2e1c06a5e0 0x7f2e1c06a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e241cc700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c0aee40 con 0x7f2e1c0ae9d0 2023-11-28T11:09:03.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e241cc700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1c15bf10 con 0x7f2e1c06e500 2023-11-28T11:09:03.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e241cc700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1c06c930 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e21f68700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2e1c06e500 0x7f2e1c15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e21f68700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2e1c06e500 0x7f2e1c15b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33338/0 (socket says 172.21.15.156:33338) 2023-11-28T11:09:03.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e21f68700 1 -- 172.21.15.156:0/3689456645 learned_addr learned my addr 172.21.15.156:0/3689456645 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:03.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e22769700 1 --2- 172.21.15.156:0/3689456645 >> v2:172.21.15.186:3300/0 conn(0x7f2e1c06a5e0 0x7f2e1c06a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e21f68700 1 -- 172.21.15.156:0/3689456645 >> v2:172.21.15.186:3300/0 conn(0x7f2e1c06a5e0 msgr2=0x7f2e1c06a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:03.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e21f68700 1 --2- 172.21.15.156:0/3689456645 >> v2:172.21.15.186:3300/0 conn(0x7f2e1c06a5e0 0x7f2e1c06a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e21f68700 1 -- 172.21.15.156:0/3689456645 >> v1:172.21.15.186:6789/0 conn(0x7f2e1c0ae9d0 legacy=0x7f2e1c06e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:03.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e21f68700 1 -- 172.21.15.156:0/3689456645 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c06cc10 con 0x7f2e1c06e500 2023-11-28T11:09:03.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e21f68700 1 --2- 172.21.15.156:0/3689456645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1c06e500 0x7f2e1c15b9d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c072160 tx=0x7f2e18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c705e9f88e7327f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e20f66700 1 -- 172.21.15.156:0/3689456645 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1800b870 con 0x7f2e1c06e500 2023-11-28T11:09:03.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e20f66700 1 -- 172.21.15.156:0/3689456645 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1800b9d0 con 0x7f2e1c06e500 2023-11-28T11:09:03.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.229+0000 7f2e20f66700 1 -- 172.21.15.156:0/3689456645 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1800bd60 con 0x7f2e1c06e500 2023-11-28T11:09:03.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.230+0000 7f2e241cc700 1 -- 172.21.15.156:0/3689456645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1c06e500 msgr2=0x7f2e1c15b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.230+0000 7f2e241cc700 1 --2- 172.21.15.156:0/3689456645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1c06e500 0x7f2e1c15b9d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c072160 tx=0x7f2e18009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.230+0000 7f2e241cc700 1 -- 172.21.15.156:0/3689456645 shutdown_connections 2023-11-28T11:09:03.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.230+0000 7f2e241cc700 1 --2- 172.21.15.156:0/3689456645 >> v2:172.21.15.186:3300/0 conn(0x7f2e1c06a5e0 0x7f2e1c06a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.230+0000 7f2e241cc700 1 --2- 172.21.15.156:0/3689456645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1c06e500 0x7f2e1c15b9d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.230+0000 7f2e241cc700 1 -- 172.21.15.156:0/3689456645 >> 172.21.15.156:0/3689456645 conn(0x7f2e1c0b12b0 msgr2=0x7f2e1c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:03.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.230+0000 7f2e241cc700 1 -- 172.21.15.156:0/3689456645 shutdown_connections 2023-11-28T11:09:03.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.230+0000 7f2e241cc700 1 -- 172.21.15.156:0/3689456645 wait complete. 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.231+0000 7f2e241cc700 1 Processor -- start 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.231+0000 7f2e241cc700 1 -- start start 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e241cc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1c06a5e0 0x7f2e1c159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e241cc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e1c06e500 0x7f2e1c0c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e241cc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e1c0ae9d0 0x7f2e1c1f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e241cc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0c8f60 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e241cc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0c7670 con 0x7f2e1c0ae9d0 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e241cc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0c8630 con 0x7f2e1c06e500 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e21f68700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1c06a5e0 0x7f2e1c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e21f68700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1c06a5e0 0x7f2e1c159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33342/0 (socket says 172.21.15.156:33342) 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e21f68700 1 -- 172.21.15.156:0/2034547572 learned_addr learned my addr 172.21.15.156:0/2034547572 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e21767700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e1c06e500 0x7f2e1c0c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e22769700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e1c0ae9d0 0x7f2e1c1f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e21f68700 1 -- 172.21.15.156:0/2034547572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e1c06e500 msgr2=0x7f2e1c0c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e21f68700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e1c06e500 0x7f2e1c0c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e21f68700 1 -- 172.21.15.156:0/2034547572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e1c0ae9d0 msgr2=0x7f2e1c1f08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e21f68700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e1c0ae9d0 0x7f2e1c1f08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e21f68700 1 -- 172.21.15.156:0/2034547572 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18009550 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.232+0000 7f2e21767700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e1c06e500 0x7f2e1c0c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:03.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.233+0000 7f2e21f68700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1c06a5e0 0x7f2e1c159010 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2e18000c00 tx=0x7f2e18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.233+0000 7f2e127fc700 1 -- 172.21.15.156:0/2034547572 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18004540 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.233+0000 7f2e127fc700 1 -- 172.21.15.156:0/2034547572 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e180036a0 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.233+0000 7f2e241cc700 1 -- 172.21.15.156:0/2034547572 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c06cef0 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.233+0000 7f2e127fc700 1 -- 172.21.15.156:0/2034547572 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18003a30 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.233+0000 7f2e241cc700 1 -- 172.21.15.156:0/2034547572 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c0664c0 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.234+0000 7f2e241cc700 1 -- 172.21.15.156:0/2034547572 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1c15ac80 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.234+0000 7f2e127fc700 1 -- 172.21.15.156:0/2034547572 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2e1800f050 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.234+0000 7f2e127fc700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e000682f0 0x7f2e0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.235+0000 7f2e127fc700 1 -- 172.21.15.156:0/2034547572 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2e180876c0 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.235+0000 7f2e21767700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e000682f0 0x7f2e0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.235+0000 7f2e21767700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e000682f0 0x7f2e0006a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c15c350 tx=0x7f2e0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.239+0000 7f2e127fc700 1 -- 172.21.15.156:0/2034547572 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2e18053df0 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.464+0000 7f2e241cc700 1 -- 172.21.15.156:0/2034547572 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e1c0c9240 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.464+0000 7f2e127fc700 1 -- 172.21.15.156:0/2034547572 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f2e18057410 con 0x7f2e1c06a5e0 2023-11-28T11:09:03.466 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:03.466 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1221397150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1221397150},{"type":"v1","addr":"172.21.15.186:6834","nonce":1221397150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/22494485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":22494485},{"type":"v1","addr":"172.21.15.156:6835","nonce":22494485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3382922708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3382922708},{"type":"v1","addr":"172.21.15.156:6837","nonce":3382922708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:17.352335+0000","modified":"2023-11-28T11:08:20.752191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/4134790300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":4134790300},{"type":"v1","addr":"172.21.15.186:6835","nonce":4134790300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:03.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.466+0000 7f2e241cc700 1 -- 172.21.15.156:0/2034547572 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e000682f0 msgr2=0x7f2e0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.466+0000 7f2e241cc700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e000682f0 0x7f2e0006a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c15c350 tx=0x7f2e0c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:03.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.466+0000 7f2e241cc700 1 -- 172.21.15.156:0/2034547572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1c06a5e0 msgr2=0x7f2e1c159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.466+0000 7f2e241cc700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1c06a5e0 0x7f2e1c159010 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2e18000c00 tx=0x7f2e18004750 comp rx=0 tx=0).stop 2023-11-28T11:09:03.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.466+0000 7f2e241cc700 1 -- 172.21.15.156:0/2034547572 shutdown_connections 2023-11-28T11:09:03.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.466+0000 7f2e241cc700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e1c0ae9d0 0x7f2e1c1f08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.466+0000 7f2e241cc700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1c06a5e0 0x7f2e1c159010 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.466+0000 7f2e241cc700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e000682f0 0x7f2e0006a7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.466+0000 7f2e241cc700 1 --2- 172.21.15.156:0/2034547572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e1c06e500 0x7f2e1c0c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.466+0000 7f2e241cc700 1 -- 172.21.15.156:0/2034547572 >> 172.21.15.156:0/2034547572 conn(0x7f2e1c0b12b0 msgr2=0x7f2e1c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:03.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.466+0000 7f2e241cc700 1 -- 172.21.15.156:0/2034547572 shutdown_connections 2023-11-28T11:09:03.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.466+0000 7f2e241cc700 1 -- 172.21.15.156:0/2034547572 wait complete. 2023-11-28T11:09:03.470 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 8 2023-11-28T11:09:03.480 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:03.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.600+0000 7fe93053b700 1 Processor -- start 2023-11-28T11:09:03.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.600+0000 7fe93053b700 1 -- start start 2023-11-28T11:09:03.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe93053b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280d0a00 0x7fe92815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe93053b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe9280b2930 0x7fe9280b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe93053b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92806b420 con 0x7fe9280ce040 2023-11-28T11:09:03.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe93053b700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9280b32f0 con 0x7fe9280b2930 2023-11-28T11:09:03.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe93053b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92806ff60 con 0x7fe9280d0a00 2023-11-28T11:09:03.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92e2d7700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe9280ce040 0x7fe92806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48228/0 (socket says 172.21.15.156:48228) 2023-11-28T11:09:03.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92e2d7700 1 -- 172.21.15.156:0/2021822018 learned_addr learned my addr 172.21.15.156:0/2021822018 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:03.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92dad6700 1 --2- 172.21.15.156:0/2021822018 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280d0a00 0x7fe92815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92ead8700 1 --2- 172.21.15.156:0/2021822018 >> v2:172.21.15.186:3300/0 conn(0x7fe9280b2930 0x7fe9280b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35255829 0 0) 0x7fe92806b420 con 0x7fe9280ce040 2023-11-28T11:09:03.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9140035b0 con 0x7fe9280ce040 2023-11-28T11:09:03.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 869860773 0 0) 0x7fe9140035b0 con 0x7fe9280ce040 2023-11-28T11:09:03.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280d0a00 msgr2=0x7fe92815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:03.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92d2d5700 1 --2- 172.21.15.156:0/2021822018 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280d0a00 0x7fe92815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 >> v2:172.21.15.186:3300/0 conn(0x7fe9280b2930 msgr2=0x7fe9280b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:03.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92d2d5700 1 --2- 172.21.15.156:0/2021822018 >> v2:172.21.15.186:3300/0 conn(0x7fe9280b2930 0x7fe9280b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92806b700 con 0x7fe9280ce040 2023-11-28T11:09:03.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fe9240017f0 con 0x7fe9280ce040 2023-11-28T11:09:03.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 >> v1:172.21.15.156:6789/0 conn(0x7fe9280ce040 legacy=0x7fe92806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92d2d5700 1 --2- 172.21.15.156:0/2021822018 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280d0a00 0x7fe914003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.601+0000 7fe92d2d5700 1 --2- 172.21.15.156:0/2021822018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe914003f80 0x7fe914004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92d2d5700 1 --2- 172.21.15.156:0/2021822018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe914004d90 0x7fe914005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280b32f0 con 0x7fe914003f80 2023-11-28T11:09:03.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92806ff60 con 0x7fe914004d90 2023-11-28T11:09:03.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91400ba10 con 0x7fe9280d0a00 2023-11-28T11:09:03.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92dad6700 1 --2- 172.21.15.156:0/2021822018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe914003f80 0x7fe914004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92e2d7700 1 --2- 172.21.15.156:0/2021822018 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280d0a00 0x7fe914003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92ead8700 1 --2- 172.21.15.156:0/2021822018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe914004d90 0x7fe914005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92dad6700 1 -- 172.21.15.156:0/2021822018 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280d0a00 msgr2=0x7fe914003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:03.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92dad6700 1 --2- 172.21.15.156:0/2021822018 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280d0a00 0x7fe914003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92dad6700 1 -- 172.21.15.156:0/2021822018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe914004d90 msgr2=0x7fe914005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:03.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92dad6700 1 --2- 172.21.15.156:0/2021822018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe914004d90 0x7fe914005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92dad6700 1 -- 172.21.15.156:0/2021822018 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9140035b0 con 0x7fe914003f80 2023-11-28T11:09:03.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92dad6700 1 --2- 172.21.15.156:0/2021822018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe914003f80 0x7fe914004850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe918009b60 tx=0x7fe914005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ea230e859fdc01 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91800e6c0 con 0x7fe914003f80 2023-11-28T11:09:03.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91800e820 con 0x7fe914003f80 2023-11-28T11:09:03.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.602+0000 7fe92d2d5700 1 -- 172.21.15.156:0/2021822018 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91800eb90 con 0x7fe914003f80 2023-11-28T11:09:03.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.603+0000 7fe93053b700 1 -- 172.21.15.156:0/2021822018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe914003f80 msgr2=0x7fe914004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.603+0000 7fe93053b700 1 --2- 172.21.15.156:0/2021822018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe914003f80 0x7fe914004850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe918009b60 tx=0x7fe914005880 comp rx=0 tx=0).stop 2023-11-28T11:09:03.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.603+0000 7fe92ead8700 1 -- 172.21.15.156:0/2021822018 reap_dead start 2023-11-28T11:09:03.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.603+0000 7fe93053b700 1 -- 172.21.15.156:0/2021822018 shutdown_connections 2023-11-28T11:09:03.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.603+0000 7fe93053b700 1 -- 172.21.15.156:0/2021822018 >> 172.21.15.156:0/2021822018 conn(0x7fe92806aed0 msgr2=0x7fe92806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:03.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.603+0000 7fe93053b700 1 -- 172.21.15.156:0/2021822018 shutdown_connections 2023-11-28T11:09:03.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.603+0000 7fe93053b700 1 -- 172.21.15.156:0/2021822018 wait complete. 2023-11-28T11:09:03.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.604+0000 7fe93053b700 1 Processor -- start 2023-11-28T11:09:03.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.604+0000 7fe93053b700 1 -- start start 2023-11-28T11:09:03.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe93053b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe914003f80 0x7fe9281f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe93053b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9280c45b0 0x7fe9280b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe93053b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280c7000 0x7fe9280c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe93053b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280b6e60 con 0x7fe9280c45b0 2023-11-28T11:09:03.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe93053b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9281f1db0 con 0x7fe914003f80 2023-11-28T11:09:03.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe93053b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92814cbb0 con 0x7fe9280c7000 2023-11-28T11:09:03.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe92dad6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9280c45b0 0x7fe9280b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe92dad6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9280c45b0 0x7fe9280b6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33364/0 (socket says 172.21.15.156:33364) 2023-11-28T11:09:03.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe92dad6700 1 -- 172.21.15.156:0/3765457444 learned_addr learned my addr 172.21.15.156:0/3765457444 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:03.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe92ead8700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280c7000 0x7fe9280c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe92e2d7700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe914003f80 0x7fe9281f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe92dad6700 1 -- 172.21.15.156:0/3765457444 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280c7000 msgr2=0x7fe9280c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe92dad6700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280c7000 0x7fe9280c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe92dad6700 1 -- 172.21.15.156:0/3765457444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe914003f80 msgr2=0x7fe9281f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe92dad6700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe914003f80 0x7fe9281f1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe92dad6700 1 -- 172.21.15.156:0/3765457444 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92806bcc0 con 0x7fe9280c45b0 2023-11-28T11:09:03.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe92e2d7700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe914003f80 0x7fe9281f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:03.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.605+0000 7fe92dad6700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9280c45b0 0x7fe9280b6610 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe91800e4b0 tx=0x7fe9180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.606+0000 7fe91effd700 1 -- 172.21.15.156:0/3765457444 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918021070 con 0x7fe9280c45b0 2023-11-28T11:09:03.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.606+0000 7fe91effd700 1 -- 172.21.15.156:0/3765457444 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe918004510 con 0x7fe9280c45b0 2023-11-28T11:09:03.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.606+0000 7fe91effd700 1 -- 172.21.15.156:0/3765457444 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91800e6c0 con 0x7fe9280c45b0 2023-11-28T11:09:03.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.606+0000 7fe93053b700 1 -- 172.21.15.156:0/3765457444 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92806b9e0 con 0x7fe9280c45b0 2023-11-28T11:09:03.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.606+0000 7fe93053b700 1 -- 172.21.15.156:0/3765457444 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92806bfa0 con 0x7fe9280c45b0 2023-11-28T11:09:03.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.607+0000 7fe93053b700 1 -- 172.21.15.156:0/3765457444 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe928068690 con 0x7fe9280c45b0 2023-11-28T11:09:03.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.607+0000 7fe91effd700 1 -- 172.21.15.156:0/3765457444 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe91802e030 con 0x7fe9280c45b0 2023-11-28T11:09:03.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.607+0000 7fe91effd700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe90c0682f0 0x7fe90c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.607+0000 7fe91effd700 1 -- 172.21.15.156:0/3765457444 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe918086ec0 con 0x7fe9280c45b0 2023-11-28T11:09:03.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.607+0000 7fe92e2d7700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe90c0682f0 0x7fe90c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.608+0000 7fe92e2d7700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe90c0682f0 0x7fe90c06a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe924000c00 tx=0x7fe924009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.611+0000 7fe91effd700 1 -- 172.21.15.156:0/3765457444 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe9180536e0 con 0x7fe9280c45b0 2023-11-28T11:09:03.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.836+0000 7fe93053b700 1 -- 172.21.15.156:0/3765457444 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe92815f670 con 0x7fe9280c45b0 2023-11-28T11:09:03.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.837+0000 7fe91effd700 1 -- 172.21.15.156:0/3765457444 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7fe918056d00 con 0x7fe9280c45b0 2023-11-28T11:09:03.839 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:03.839 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1221397150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1221397150},{"type":"v1","addr":"172.21.15.186:6834","nonce":1221397150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/22494485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":22494485},{"type":"v1","addr":"172.21.15.156:6835","nonce":22494485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3382922708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3382922708},{"type":"v1","addr":"172.21.15.156:6837","nonce":3382922708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:17.352335+0000","modified":"2023-11-28T11:08:20.752191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/4134790300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":4134790300},{"type":"v1","addr":"172.21.15.186:6835","nonce":4134790300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:03.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.838+0000 7fe93053b700 1 -- 172.21.15.156:0/3765457444 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe90c0682f0 msgr2=0x7fe90c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.838+0000 7fe93053b700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe90c0682f0 0x7fe90c06a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe924000c00 tx=0x7fe924009040 comp rx=0 tx=0).stop 2023-11-28T11:09:03.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.839+0000 7fe93053b700 1 -- 172.21.15.156:0/3765457444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9280c45b0 msgr2=0x7fe9280b6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.839+0000 7fe93053b700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9280c45b0 0x7fe9280b6610 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe91800e4b0 tx=0x7fe9180047c0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.839+0000 7fe93053b700 1 -- 172.21.15.156:0/3765457444 shutdown_connections 2023-11-28T11:09:03.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.839+0000 7fe93053b700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe90c0682f0 0x7fe90c06a7a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.839+0000 7fe93053b700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9280c7000 0x7fe9280c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.839+0000 7fe93053b700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe914003f80 0x7fe9281f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.839+0000 7fe93053b700 1 --2- 172.21.15.156:0/3765457444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9280c45b0 0x7fe9280b6610 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.839+0000 7fe93053b700 1 -- 172.21.15.156:0/3765457444 >> 172.21.15.156:0/3765457444 conn(0x7fe92806aed0 msgr2=0x7fe9280cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:03.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.839+0000 7fe93053b700 1 -- 172.21.15.156:0/3765457444 shutdown_connections 2023-11-28T11:09:03.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.839+0000 7fe93053b700 1 -- 172.21.15.156:0/3765457444 wait complete. 2023-11-28T11:09:03.841 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 8 2023-11-28T11:09:03.852 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:03.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.972+0000 7f4cf2aa2700 1 Processor -- start 2023-11-28T11:09:03.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.972+0000 7f4cf2aa2700 1 -- start start 2023-11-28T11:09:03.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.972+0000 7f4cf2aa2700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4cec15ace0 0x7f4cec0b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.972+0000 7f4cf2aa2700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cec0cf1b0 0x7f4cec0d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.972+0000 7f4cf2aa2700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cec0d0240 con 0x7f4cec0cd5a0 2023-11-28T11:09:03.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.972+0000 7f4cf2aa2700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cec0cdd60 con 0x7f4cec15ace0 2023-11-28T11:09:03.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.972+0000 7f4cf2aa2700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec0ce040 con 0x7f4cec0cf1b0 2023-11-28T11:09:03.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.973+0000 7f4cf103f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cec0cf1b0 0x7f4cec0d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.973+0000 7f4cf083e700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4cec15ace0 0x7f4cec0b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.973+0000 7f4cf103f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cec0cf1b0 0x7f4cec0d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37238/0 (socket says 172.21.15.156:37238) 2023-11-28T11:09:03.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.973+0000 7f4cebfff700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f4cec0cd5a0 0x7f4cec0ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:58610/0 (socket says 172.21.15.156:58610) 2023-11-28T11:09:03.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.973+0000 7f4cf103f700 1 -- 172.21.15.156:0/1529171615 learned_addr learned my addr 172.21.15.156:0/1529171615 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:03.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.973+0000 7f4cf103f700 1 -- 172.21.15.156:0/1529171615 >> v2:172.21.15.186:3300/0 conn(0x7f4cec15ace0 msgr2=0x7f4cec0b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:03.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.973+0000 7f4cf103f700 1 --2- 172.21.15.156:0/1529171615 >> v2:172.21.15.186:3300/0 conn(0x7f4cec15ace0 0x7f4cec0b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.973+0000 7f4cf103f700 1 -- 172.21.15.156:0/1529171615 >> v1:172.21.15.186:6789/0 conn(0x7f4cec0cd5a0 legacy=0x7f4cec0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.973+0000 7f4cf103f700 1 -- 172.21.15.156:0/1529171615 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec0d04e0 con 0x7f4cec0cf1b0 2023-11-28T11:09:03.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.973+0000 7f4cf083e700 1 --2- 172.21.15.156:0/1529171615 >> v2:172.21.15.186:3300/0 conn(0x7f4cec15ace0 0x7f4cec0b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:03.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.973+0000 7f4cf103f700 1 --2- 172.21.15.156:0/1529171615 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cec0cf1b0 0x7f4cec0d0a00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009e50 tx=0x7f4ce400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=364693884de639ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.973+0000 7f4ceb7fe700 1 -- 172.21.15.156:0/1529171615 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce400e070 con 0x7f4cec0cf1b0 2023-11-28T11:09:03.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.974+0000 7f4ceb7fe700 1 -- 172.21.15.156:0/1529171615 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce400b910 con 0x7f4cec0cf1b0 2023-11-28T11:09:03.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.974+0000 7f4ceb7fe700 1 -- 172.21.15.156:0/1529171615 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce400bcc0 con 0x7f4cec0cf1b0 2023-11-28T11:09:03.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.974+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1529171615 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cec0cf1b0 msgr2=0x7f4cec0d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.974+0000 7f4cf2aa2700 1 --2- 172.21.15.156:0/1529171615 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cec0cf1b0 0x7f4cec0d0a00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009e50 tx=0x7f4ce400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:03.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.974+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1529171615 shutdown_connections 2023-11-28T11:09:03.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.974+0000 7f4cf2aa2700 1 --2- 172.21.15.156:0/1529171615 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cec0cf1b0 0x7f4cec0d0a00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.974+0000 7f4cf2aa2700 1 --2- 172.21.15.156:0/1529171615 >> v2:172.21.15.186:3300/0 conn(0x7f4cec15ace0 0x7f4cec0b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.974+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1529171615 >> 172.21.15.156:0/1529171615 conn(0x7f4cec06a270 msgr2=0x7f4cec06a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:03.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.974+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1529171615 shutdown_connections 2023-11-28T11:09:03.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.974+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1529171615 wait complete. 2023-11-28T11:09:03.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.975+0000 7f4cf2aa2700 1 Processor -- start 2023-11-28T11:09:03.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.975+0000 7f4cf2aa2700 1 -- start start 2023-11-28T11:09:03.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.976+0000 7f4cf2aa2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cec0cd5a0 0x7f4cec1f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.976+0000 7f4cf2aa2700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cec0cf1b0 0x7f4cec06cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.976+0000 7f4cf2aa2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cec15ace0 0x7f4cec0b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.976+0000 7f4cf2aa2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec0b6be0 con 0x7f4cec0cd5a0 2023-11-28T11:09:03.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.976+0000 7f4cf2aa2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec1f3cc0 con 0x7f4cec15ace0 2023-11-28T11:09:03.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.976+0000 7f4cf2aa2700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec1f3a20 con 0x7f4cec0cf1b0 2023-11-28T11:09:03.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.976+0000 7f4cf083e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cec0cd5a0 0x7f4cec1f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.976+0000 7f4cf083e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cec0cd5a0 0x7f4cec1f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33376/0 (socket says 172.21.15.156:33376) 2023-11-28T11:09:03.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.976+0000 7f4cf083e700 1 -- 172.21.15.156:0/1142280413 learned_addr learned my addr 172.21.15.156:0/1142280413 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:03.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4cebfff700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cec0cf1b0 0x7f4cec06cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4cf103f700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cec15ace0 0x7f4cec0b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4cf083e700 1 -- 172.21.15.156:0/1142280413 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cec0cf1b0 msgr2=0x7f4cec06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4cf083e700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cec0cf1b0 0x7f4cec06cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4cf083e700 1 -- 172.21.15.156:0/1142280413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cec15ace0 msgr2=0x7f4cec0b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4cf083e700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cec15ace0 0x7f4cec0b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4cf083e700 1 -- 172.21.15.156:0/1142280413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce00094a0 con 0x7f4cec0cd5a0 2023-11-28T11:09:03.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4cf103f700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cec15ace0 0x7f4cec0b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:03.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4cf083e700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cec0cd5a0 0x7f4cec1f2820 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000ebe0 tx=0x7f4ce0009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4ce97fa700 1 -- 172.21.15.156:0/1142280413 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce000c8b0 con 0x7f4cec0cd5a0 2023-11-28T11:09:03.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4ce97fa700 1 -- 172.21.15.156:0/1142280413 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce000ca10 con 0x7f4cec0cd5a0 2023-11-28T11:09:03.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4ce97fa700 1 -- 172.21.15.156:0/1142280413 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce000cda0 con 0x7f4cec0cd5a0 2023-11-28T11:09:03.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1142280413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce4009b30 con 0x7f4cec0cd5a0 2023-11-28T11:09:03.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.977+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1142280413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cec0cda40 con 0x7f4cec0cd5a0 2023-11-28T11:09:03.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.978+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1142280413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cec0ce320 con 0x7f4cec0cd5a0 2023-11-28T11:09:03.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.978+0000 7f4ce97fa700 1 -- 172.21.15.156:0/1142280413 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4ce0019030 con 0x7f4cec0cd5a0 2023-11-28T11:09:03.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.979+0000 7f4ce97fa700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4cd00682f0 0x7f4cd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.979+0000 7f4ce97fa700 1 -- 172.21.15.156:0/1142280413 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4ce0085cb0 con 0x7f4cec0cd5a0 2023-11-28T11:09:03.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.979+0000 7f4cebfff700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4cd00682f0 0x7f4cd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.979+0000 7f4cebfff700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4cd00682f0 0x7f4cd006a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4cec06cd80 tx=0x7f4cd8007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:03.983+0000 7f4ce97fa700 1 -- 172.21.15.156:0/1142280413 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4ce00524d0 con 0x7f4cec0cd5a0 2023-11-28T11:09:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.166+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1142280413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cec0b3f30 con 0x7f4cec0cd5a0 2023-11-28T11:09:04.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.168+0000 7f4ce97fa700 1 -- 172.21.15.156:0/1142280413 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18050 (secure 0 0 0) 0x7f4ce0021090 con 0x7f4cec0cd5a0 2023-11-28T11:09:04.169 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:04.169 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":25,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:01.859199+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:15.021591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:16.332241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:04.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.169+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1142280413 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4cd00682f0 msgr2=0x7f4cd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.170+0000 7f4cf2aa2700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4cd00682f0 0x7f4cd006a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4cec06cd80 tx=0x7f4cd8007d10 comp rx=0 tx=0).stop 2023-11-28T11:09:04.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.170+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1142280413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cec0cd5a0 msgr2=0x7f4cec1f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.170+0000 7f4cf2aa2700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cec0cd5a0 0x7f4cec1f2820 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000ebe0 tx=0x7f4ce0009e20 comp rx=0 tx=0).stop 2023-11-28T11:09:04.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.170+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1142280413 shutdown_connections 2023-11-28T11:09:04.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.170+0000 7f4cf2aa2700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cec15ace0 0x7f4cec0b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.170+0000 7f4cf2aa2700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cec0cd5a0 0x7f4cec1f2820 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.170+0000 7f4cf2aa2700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4cd00682f0 0x7f4cd006a7a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.170+0000 7f4cf2aa2700 1 --2- 172.21.15.156:0/1142280413 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cec0cf1b0 0x7f4cec06cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.170+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1142280413 >> 172.21.15.156:0/1142280413 conn(0x7f4cec06a270 msgr2=0x7f4cec0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:04.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.170+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1142280413 shutdown_connections 2023-11-28T11:09:04.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.170+0000 7f4cf2aa2700 1 -- 172.21.15.156:0/1142280413 wait complete. 2023-11-28T11:09:04.184 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:09:04.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.303+0000 7f0f5bb9e700 1 Processor -- start 2023-11-28T11:09:04.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.303+0000 7f0f5bb9e700 1 -- start start 2023-11-28T11:09:04.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.304+0000 7f0f5bb9e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f5406dee0 0x7f0f5406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.304+0000 7f0f5bb9e700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0f54160540 0x7f0f54160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.304+0000 7f0f5bb9e700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5415d060 con 0x7f0f54071040 2023-11-28T11:09:04.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.304+0000 7f0f5bb9e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5415d340 con 0x7f0f5406dee0 2023-11-28T11:09:04.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.304+0000 7f0f5bb9e700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5415d620 con 0x7f0f54160540 2023-11-28T11:09:04.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.304+0000 7f0f59139700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f5406dee0 0x7f0f5406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.304+0000 7f0f59139700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f5406dee0 0x7f0f5406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33386/0 (socket says 172.21.15.156:33386) 2023-11-28T11:09:04.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.304+0000 7f0f59139700 1 -- 172.21.15.156:0/1342990387 learned_addr learned my addr 172.21.15.156:0/1342990387 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:04.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.304+0000 7f0f5a13b700 1 --2- 172.21.15.156:0/1342990387 >> v2:172.21.15.186:3300/0 conn(0x7f0f54160540 0x7f0f54160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f59139700 1 -- 172.21.15.156:0/1342990387 >> v2:172.21.15.186:3300/0 conn(0x7f0f54160540 msgr2=0x7f0f54160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:04.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f59139700 1 --2- 172.21.15.156:0/1342990387 >> v2:172.21.15.186:3300/0 conn(0x7f0f54160540 0x7f0f54160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f59139700 1 -- 172.21.15.156:0/1342990387 >> v1:172.21.15.186:6789/0 conn(0x7f0f54071040 legacy=0x7f0f54071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f59139700 1 -- 172.21.15.156:0/1342990387 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5415d900 con 0x7f0f5406dee0 2023-11-28T11:09:04.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f59139700 1 --2- 172.21.15.156:0/1342990387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f5406dee0 0x7f0f5406e330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0f540c3a10 tx=0x7f0f44009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0ee6dba674d33f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:04.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f58938700 1 -- 172.21.15.156:0/1342990387 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4400b870 con 0x7f0f5406dee0 2023-11-28T11:09:04.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f58938700 1 -- 172.21.15.156:0/1342990387 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4400b9d0 con 0x7f0f5406dee0 2023-11-28T11:09:04.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f58938700 1 -- 172.21.15.156:0/1342990387 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4400bd60 con 0x7f0f5406dee0 2023-11-28T11:09:04.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/1342990387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f5406dee0 msgr2=0x7f0f5406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f5bb9e700 1 --2- 172.21.15.156:0/1342990387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f5406dee0 0x7f0f5406e330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0f540c3a10 tx=0x7f0f44009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/1342990387 shutdown_connections 2023-11-28T11:09:04.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f5bb9e700 1 --2- 172.21.15.156:0/1342990387 >> v2:172.21.15.186:3300/0 conn(0x7f0f54160540 0x7f0f54160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.305+0000 7f0f5bb9e700 1 --2- 172.21.15.156:0/1342990387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f5406dee0 0x7f0f5406e330 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.306+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/1342990387 >> 172.21.15.156:0/1342990387 conn(0x7f0f540d0000 msgr2=0x7f0f540d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:04.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.306+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/1342990387 shutdown_connections 2023-11-28T11:09:04.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.306+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/1342990387 wait complete. 2023-11-28T11:09:04.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.307+0000 7f0f5bb9e700 1 Processor -- start 2023-11-28T11:09:04.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.307+0000 7f0f5bb9e700 1 -- start start 2023-11-28T11:09:04.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.307+0000 7f0f5bb9e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f5406dee0 0x7f0f5414a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.307+0000 7f0f5bb9e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f54071040 0x7f0f540b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.307+0000 7f0f5bb9e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f54160540 0x7f0f540b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5bb9e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f541f7190 con 0x7f0f54160540 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5bb9e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f541f6610 con 0x7f0f5406dee0 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5bb9e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f541f68f0 con 0x7f0f54071040 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5a13b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f54160540 0x7f0f540b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5a13b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f54160540 0x7f0f540b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33394/0 (socket says 172.21.15.156:33394) 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5a13b700 1 -- 172.21.15.156:0/635527928 learned_addr learned my addr 172.21.15.156:0/635527928 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f59139700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f54071040 0x7f0f540b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5993a700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f5406dee0 0x7f0f5414a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5a13b700 1 -- 172.21.15.156:0/635527928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f54071040 msgr2=0x7f0f540b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5a13b700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f54071040 0x7f0f540b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5a13b700 1 -- 172.21.15.156:0/635527928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f5406dee0 msgr2=0x7f0f5414a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5a13b700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f5406dee0 0x7f0f5414a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5a13b700 1 -- 172.21.15.156:0/635527928 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f44009550 con 0x7f0f54160540 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f59139700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f54071040 0x7f0f540b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5993a700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f5406dee0 0x7f0f5414a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5a13b700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f54160540 0x7f0f540b0150 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c007e60 tx=0x7f0f4c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f4a7fc700 1 -- 172.21.15.156:0/635527928 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00ede0 con 0x7f0f54160540 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f4a7fc700 1 -- 172.21.15.156:0/635527928 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c014420 con 0x7f0f54160540 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/635527928 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5415ccd0 con 0x7f0f54160540 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f4a7fc700 1 -- 172.21.15.156:0/635527928 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0147d0 con 0x7f0f54160540 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.308+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/635527928 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5406e870 con 0x7f0f54160540 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.309+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/635527928 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5406a270 con 0x7f0f54160540 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.310+0000 7f0f4a7fc700 1 -- 172.21.15.156:0/635527928 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0f4c012050 con 0x7f0f54160540 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.310+0000 7f0f4a7fc700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f380682f0 0x7f0f3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.310+0000 7f0f4a7fc700 1 -- 172.21.15.156:0/635527928 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0f4c086300 con 0x7f0f54160540 2023-11-28T11:09:04.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.310+0000 7f0f5993a700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f380682f0 0x7f0f3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.310+0000 7f0f5993a700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f380682f0 0x7f0f3806a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0f50000f60 tx=0x7f0f50009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:04.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.314+0000 7f0f4a7fc700 1 -- 172.21.15.156:0/635527928 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0f4c052b20 con 0x7f0f54160540 2023-11-28T11:09:04.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.537+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/635527928 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0f540c5050 con 0x7f0f54160540 2023-11-28T11:09:04.880 INFO:tasks.ceph.mon.a.smithi156.stderr:2023-11-28T11:09:04.878+0000 7f6a4eb7c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:09:04.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.889+0000 7f0f4a7fc700 1 -- 172.21.15.156:0/635527928 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0f540c5050 con 0x7f0f54160540 2023-11-28T11:09:04.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.892+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/635527928 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f380682f0 msgr2=0x7f0f3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.892+0000 7f0f5bb9e700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f380682f0 0x7f0f3806a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0f50000f60 tx=0x7f0f50009040 comp rx=0 tx=0).stop 2023-11-28T11:09:04.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.892+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/635527928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f54160540 msgr2=0x7f0f540b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.892+0000 7f0f5bb9e700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f54160540 0x7f0f540b0150 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c007e60 tx=0x7f0f4c00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.892+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/635527928 shutdown_connections 2023-11-28T11:09:04.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.892+0000 7f0f5bb9e700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f54160540 0x7f0f540b0150 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.892+0000 7f0f5bb9e700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f5406dee0 0x7f0f5414a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.892+0000 7f0f5bb9e700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f380682f0 0x7f0f3806a7a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.892+0000 7f0f5bb9e700 1 --2- 172.21.15.156:0/635527928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f54071040 0x7f0f540b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.892+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/635527928 >> 172.21.15.156:0/635527928 conn(0x7f0f540d0000 msgr2=0x7f0f5415f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:04.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.892+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/635527928 shutdown_connections 2023-11-28T11:09:04.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:04.892+0000 7f0f5bb9e700 1 -- 172.21.15.156:0/635527928 wait complete. 2023-11-28T11:09:04.894 INFO:teuthology.orchestra.run.smithi156.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:09:04.906 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:09:04.930 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2023-11-28T11:09:05.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.031+0000 7fdf1adec700 1 Processor -- start 2023-11-28T11:09:05.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.032+0000 7fdf1adec700 1 -- start start 2023-11-28T11:09:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.032+0000 7fdf1adec700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf1406dee0 0x7fdf1406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.032+0000 7fdf1adec700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1415d060 con 0x7fdf14160540 2023-11-28T11:09:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.032+0000 7fdf1adec700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1415d340 con 0x7fdf14071040 2023-11-28T11:09:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.032+0000 7fdf1adec700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1415d620 con 0x7fdf1406dee0 2023-11-28T11:09:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.032+0000 7fdf19389700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fdf14160540 0x7fdf141609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48280/0 (socket says 172.21.15.156:48280) 2023-11-28T11:09:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.032+0000 7fdf19389700 1 -- 172.21.15.156:0/1546767368 learned_addr learned my addr 172.21.15.156:0/1546767368 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442935924 0 0) 0x7fdf1415d060 con 0x7fdf14160540 2023-11-28T11:09:05.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf13fff700 1 --2- 172.21.15.156:0/1546767368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf1406dee0 0x7fdf1406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdefc0035b0 con 0x7fdf14160540 2023-11-28T11:09:05.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127796874 0 0) 0x7fdf1415d340 con 0x7fdf14071040 2023-11-28T11:09:05.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf1415d060 con 0x7fdf14071040 2023-11-28T11:09:05.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1122520953 0 0) 0x7fdefc0035b0 con 0x7fdf14160540 2023-11-28T11:09:05.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf1406dee0 msgr2=0x7fdf1406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:05.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 --2- 172.21.15.156:0/1546767368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf1406dee0 0x7fdf1406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 >> v1:172.21.15.186:6789/0 conn(0x7fdf14071040 legacy=0x7fdf14071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1415d900 con 0x7fdf14160540 2023-11-28T11:09:05.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fdf04001670 con 0x7fdf14160540 2023-11-28T11:09:05.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 >> v1:172.21.15.156:6789/0 conn(0x7fdf14160540 legacy=0x7fdf141609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 --2- 172.21.15.156:0/1546767368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdefc003bc0 0x7fdefc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 --2- 172.21.15.156:0/1546767368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf1406dee0 0x7fdefc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 --2- 172.21.15.156:0/1546767368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdefc004d90 0x7fdefc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1415d620 con 0x7fdefc004d90 2023-11-28T11:09:05.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc00bbf0 con 0x7fdefc003bc0 2023-11-28T11:09:05.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc00bdb0 con 0x7fdf1406dee0 2023-11-28T11:09:05.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf19389700 1 --2- 172.21.15.156:0/1546767368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdefc004d90 0x7fdefc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf18b88700 1 --2- 172.21.15.156:0/1546767368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdefc003bc0 0x7fdefc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf13fff700 1 --2- 172.21.15.156:0/1546767368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf1406dee0 0x7fdefc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf19389700 1 -- 172.21.15.156:0/1546767368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf1406dee0 msgr2=0x7fdefc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:05.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf19389700 1 --2- 172.21.15.156:0/1546767368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf1406dee0 0x7fdefc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf19389700 1 -- 172.21.15.156:0/1546767368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdefc003bc0 msgr2=0x7fdefc003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:05.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf19389700 1 --2- 172.21.15.156:0/1546767368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdefc003bc0 0x7fdefc003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.033+0000 7fdf19389700 1 -- 172.21.15.156:0/1546767368 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc0035b0 con 0x7fdefc004d90 2023-11-28T11:09:05.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.034+0000 7fdf19389700 1 --2- 172.21.15.156:0/1546767368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdefc004d90 0x7fdefc005200 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c110 tx=0x7fdefc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2970b92129521458 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:05.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.034+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0400ccb0 con 0x7fdefc004d90 2023-11-28T11:09:05.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.034+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0400ce10 con 0x7fdefc004d90 2023-11-28T11:09:05.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.034+0000 7fdf137fe700 1 -- 172.21.15.156:0/1546767368 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04005ba0 con 0x7fdefc004d90 2023-11-28T11:09:05.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.034+0000 7fdf1adec700 1 -- 172.21.15.156:0/1546767368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdefc004d90 msgr2=0x7fdefc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.034+0000 7fdf1adec700 1 --2- 172.21.15.156:0/1546767368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdefc004d90 0x7fdefc005200 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c110 tx=0x7fdefc005880 comp rx=0 tx=0).stop 2023-11-28T11:09:05.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.034+0000 7fdf19389700 1 -- 172.21.15.156:0/1546767368 reap_dead start 2023-11-28T11:09:05.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.034+0000 7fdf1adec700 1 -- 172.21.15.156:0/1546767368 shutdown_connections 2023-11-28T11:09:05.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.034+0000 7fdf1adec700 1 -- 172.21.15.156:0/1546767368 >> 172.21.15.156:0/1546767368 conn(0x7fdf140d0000 msgr2=0x7fdf140d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:05.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.034+0000 7fdf1adec700 1 -- 172.21.15.156:0/1546767368 shutdown_connections 2023-11-28T11:09:05.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.034+0000 7fdf1adec700 1 -- 172.21.15.156:0/1546767368 wait complete. 2023-11-28T11:09:05.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.036+0000 7fdf1adec700 1 Processor -- start 2023-11-28T11:09:05.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.036+0000 7fdf1adec700 1 -- start start 2023-11-28T11:09:05.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf1adec700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdefc004d90 0x7fdf141566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf1adec700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf140c4e90 0x7fdf140c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf1adec700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf14157470 0x7fdf141578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf1adec700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf14157e20 con 0x7fdf140c4e90 2023-11-28T11:09:05.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf1adec700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140c5930 con 0x7fdf14157470 2023-11-28T11:09:05.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf1adec700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140c5c10 con 0x7fdefc004d90 2023-11-28T11:09:05.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf13fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf140c4e90 0x7fdf140c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf18b88700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdefc004d90 0x7fdf141566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf18b88700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdefc004d90 0x7fdf141566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37274/0 (socket says 172.21.15.156:37274) 2023-11-28T11:09:05.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf18b88700 1 -- 172.21.15.156:0/315132458 learned_addr learned my addr 172.21.15.156:0/315132458 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:05.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf19389700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf14157470 0x7fdf141578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf13fff700 1 -- 172.21.15.156:0/315132458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdefc004d90 msgr2=0x7fdf141566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf13fff700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdefc004d90 0x7fdf141566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf13fff700 1 -- 172.21.15.156:0/315132458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf14157470 msgr2=0x7fdf141578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf13fff700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf14157470 0x7fdf141578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.037+0000 7fdf13fff700 1 -- 172.21.15.156:0/315132458 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1406a270 con 0x7fdf140c4e90 2023-11-28T11:09:05.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.038+0000 7fdf13fff700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf140c4e90 0x7fdf140c5300 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdf08011960 tx=0x7fdf08011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:05.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.038+0000 7fdf117fa700 1 -- 172.21.15.156:0/315132458 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08004300 con 0x7fdf140c4e90 2023-11-28T11:09:05.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.038+0000 7fdf117fa700 1 -- 172.21.15.156:0/315132458 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf08004460 con 0x7fdf140c4e90 2023-11-28T11:09:05.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.038+0000 7fdf117fa700 1 -- 172.21.15.156:0/315132458 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf080038c0 con 0x7fdf140c4e90 2023-11-28T11:09:05.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.038+0000 7fdf1adec700 1 -- 172.21.15.156:0/315132458 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1415ccd0 con 0x7fdf140c4e90 2023-11-28T11:09:05.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.038+0000 7fdf1adec700 1 -- 172.21.15.156:0/315132458 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf140c61d0 con 0x7fdf140c4e90 2023-11-28T11:09:05.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.039+0000 7fdf1adec700 1 -- 172.21.15.156:0/315132458 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf140c6820 con 0x7fdf140c4e90 2023-11-28T11:09:05.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.039+0000 7fdf117fa700 1 -- 172.21.15.156:0/315132458 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdf08025070 con 0x7fdf140c4e90 2023-11-28T11:09:05.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.039+0000 7fdf117fa700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdf000682f0 0x7fdf0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.040+0000 7fdf117fa700 1 -- 172.21.15.156:0/315132458 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fdf08085f80 con 0x7fdf140c4e90 2023-11-28T11:09:05.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.040+0000 7fdf18b88700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdf000682f0 0x7fdf0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.040+0000 7fdf18b88700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdf000682f0 0x7fdf0006a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c001460 tx=0x7fdf0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:05.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.044+0000 7fdf117fa700 1 -- 172.21.15.156:0/315132458 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdf08052750 con 0x7fdf140c4e90 2023-11-28T11:09:05.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.270+0000 7fdf1adec700 1 -- 172.21.15.156:0/315132458 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdf1406a400 con 0x7fdf140c4e90 2023-11-28T11:09:05.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.899+0000 7fdf117fa700 1 -- 172.21.15.156:0/315132458 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdf1406a400 con 0x7fdf140c4e90 2023-11-28T11:09:05.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.901+0000 7fdf1adec700 1 -- 172.21.15.156:0/315132458 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdf000682f0 msgr2=0x7fdf0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.901+0000 7fdf1adec700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdf000682f0 0x7fdf0006a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c001460 tx=0x7fdf0c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:05.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.901+0000 7fdf1adec700 1 -- 172.21.15.156:0/315132458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf140c4e90 msgr2=0x7fdf140c5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.901+0000 7fdf1adec700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf140c4e90 0x7fdf140c5300 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdf08011960 tx=0x7fdf08011c70 comp rx=0 tx=0).stop 2023-11-28T11:09:05.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.902+0000 7fdf1adec700 1 -- 172.21.15.156:0/315132458 shutdown_connections 2023-11-28T11:09:05.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.902+0000 7fdf1adec700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf14157470 0x7fdf141578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.902+0000 7fdf1adec700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdf000682f0 0x7fdf0006a7a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.902+0000 7fdf1adec700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdefc004d90 0x7fdf141566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.902+0000 7fdf1adec700 1 --2- 172.21.15.156:0/315132458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf140c4e90 0x7fdf140c5300 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.902+0000 7fdf1adec700 1 -- 172.21.15.156:0/315132458 >> 172.21.15.156:0/315132458 conn(0x7fdf140d0000 msgr2=0x7fdf14071410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:05.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.902+0000 7fdf1adec700 1 -- 172.21.15.156:0/315132458 shutdown_connections 2023-11-28T11:09:05.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:05.902+0000 7fdf1adec700 1 -- 172.21.15.156:0/315132458 wait complete. 2023-11-28T11:09:05.917 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:09:06.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.044+0000 7fd006226700 1 Processor -- start 2023-11-28T11:09:06.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.044+0000 7fd006226700 1 -- start start 2023-11-28T11:09:06.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.045+0000 7fd006226700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd0000d0610 0x7fd0000cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.045+0000 7fd006226700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd00006a270 0x7fd00006a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.045+0000 7fd006226700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00006de50 con 0x7fd000071c00 2023-11-28T11:09:06.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.045+0000 7fd006226700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0000d0a80 con 0x7fd0000d0610 2023-11-28T11:09:06.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.045+0000 7fd006226700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00006ac30 con 0x7fd00006a270 2023-11-28T11:09:06.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.045+0000 7fcffeffd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd0000d0610 0x7fd0000cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.045+0000 7fcffeffd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd0000d0610 0x7fd0000cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:44432/0 (socket says 172.21.15.156:44432) 2023-11-28T11:09:06.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.045+0000 7fcffffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd00006a270 0x7fd00006a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.045+0000 7fcffeffd700 1 -- 172.21.15.156:0/3317047966 learned_addr learned my addr 172.21.15.156:0/3317047966 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:06.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.045+0000 7fcfff7fe700 1 --1- 172.21.15.156:0/3317047966 >> v1:172.21.15.186:6789/0 conn(0x7fd000071c00 0x7fd000070d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:58656/0 (socket says 172.21.15.156:58656) 2023-11-28T11:09:06.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.046+0000 7fcffeffd700 1 -- 172.21.15.156:0/3317047966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd00006a270 msgr2=0x7fd00006a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.046+0000 7fcffeffd700 1 --2- 172.21.15.156:0/3317047966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd00006a270 0x7fd00006a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.046+0000 7fcffeffd700 1 -- 172.21.15.156:0/3317047966 >> v1:172.21.15.186:6789/0 conn(0x7fd000071c00 legacy=0x7fd000070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.046+0000 7fcffeffd700 1 -- 172.21.15.156:0/3317047966 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd000071fd0 con 0x7fd0000d0610 2023-11-28T11:09:06.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.046+0000 7fcffe7fc700 1 -- 172.21.15.156:0/3317047966 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774814983 0 0) 0x7fcfe8001110 con 0x7fd000071c00 2023-11-28T11:09:06.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.046+0000 7fcffffff700 1 --2- 172.21.15.156:0/3317047966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd00006a270 0x7fd00006a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:06.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.046+0000 7fcffeffd700 1 --2- 172.21.15.156:0/3317047966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0000d0610 0x7fd0000cd290 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcff00098f0 tx=0x7fcff0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a0edd55e38fbbb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.046+0000 7fcffe7fc700 1 -- 172.21.15.156:0/3317047966 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0011830 con 0x7fd0000d0610 2023-11-28T11:09:06.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.046+0000 7fcffe7fc700 1 -- 172.21.15.156:0/3317047966 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff0011990 con 0x7fd0000d0610 2023-11-28T11:09:06.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.046+0000 7fcffe7fc700 1 -- 172.21.15.156:0/3317047966 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0011d20 con 0x7fd0000d0610 2023-11-28T11:09:06.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.047+0000 7fd006226700 1 -- 172.21.15.156:0/3317047966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0000d0610 msgr2=0x7fd0000cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.047+0000 7fd006226700 1 --2- 172.21.15.156:0/3317047966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0000d0610 0x7fd0000cd290 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcff00098f0 tx=0x7fcff0009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:06.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.047+0000 7fd006226700 1 -- 172.21.15.156:0/3317047966 shutdown_connections 2023-11-28T11:09:06.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.047+0000 7fd006226700 1 --2- 172.21.15.156:0/3317047966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd00006a270 0x7fd00006a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.047+0000 7fd006226700 1 --2- 172.21.15.156:0/3317047966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0000d0610 0x7fd0000cd290 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.047+0000 7fd006226700 1 -- 172.21.15.156:0/3317047966 >> 172.21.15.156:0/3317047966 conn(0x7fd00006b380 msgr2=0x7fd00006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.047+0000 7fd006226700 1 -- 172.21.15.156:0/3317047966 shutdown_connections 2023-11-28T11:09:06.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.047+0000 7fd006226700 1 -- 172.21.15.156:0/3317047966 wait complete. 2023-11-28T11:09:06.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.048+0000 7fd006226700 1 Processor -- start 2023-11-28T11:09:06.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.049+0000 7fd006226700 1 -- start start 2023-11-28T11:09:06.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.049+0000 7fd006226700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd00006a270 0x7fd0001495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.049+0000 7fd006226700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd000071c00 0x7fd00014c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.049+0000 7fd006226700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0000d0610 0x7fd00014b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.049+0000 7fd006226700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd000149af0 con 0x7fd00006a270 2023-11-28T11:09:06.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.049+0000 7fd006226700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00014cd50 con 0x7fd0000d0610 2023-11-28T11:09:06.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.049+0000 7fd006226700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00014a310 con 0x7fd000071c00 2023-11-28T11:09:06.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcfff7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd00006a270 0x7fd0001495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcfff7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd00006a270 0x7fd0001495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33422/0 (socket says 172.21.15.156:33422) 2023-11-28T11:09:06.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcfff7fe700 1 -- 172.21.15.156:0/1233941909 learned_addr learned my addr 172.21.15.156:0/1233941909 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:06.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcffffff700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0000d0610 0x7fd00014b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcffeffd700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd000071c00 0x7fd00014c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcfff7fe700 1 -- 172.21.15.156:0/1233941909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd000071c00 msgr2=0x7fd00014c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcfff7fe700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd000071c00 0x7fd00014c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcfff7fe700 1 -- 172.21.15.156:0/1233941909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0000d0610 msgr2=0x7fd00014b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcfff7fe700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0000d0610 0x7fd00014b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcfff7fe700 1 -- 172.21.15.156:0/1233941909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff8009a70 con 0x7fd00006a270 2023-11-28T11:09:06.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcffffff700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0000d0610 0x7fd00014b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:06.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcfff7fe700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd00006a270 0x7fd0001495b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800c760 tx=0x7fcfe800cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcff7fff700 1 -- 172.21.15.156:0/1233941909 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8004c80 con 0x7fd00006a270 2023-11-28T11:09:06.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcff7fff700 1 -- 172.21.15.156:0/1233941909 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe8004e00 con 0x7fd00006a270 2023-11-28T11:09:06.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fd006226700 1 -- 172.21.15.156:0/1233941909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff0009550 con 0x7fd00006a270 2023-11-28T11:09:06.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fcff7fff700 1 -- 172.21.15.156:0/1233941909 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8007310 con 0x7fd00006a270 2023-11-28T11:09:06.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.050+0000 7fd006226700 1 -- 172.21.15.156:0/1233941909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00006d890 con 0x7fd00006a270 2023-11-28T11:09:06.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.051+0000 7fd006226700 1 -- 172.21.15.156:0/1233941909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0000cb440 con 0x7fd00006a270 2023-11-28T11:09:06.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.052+0000 7fcff7fff700 1 -- 172.21.15.156:0/1233941909 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcfe801a020 con 0x7fd00006a270 2023-11-28T11:09:06.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.052+0000 7fcff7fff700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcfe00682f0 0x7fcfe006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.052+0000 7fcff7fff700 1 -- 172.21.15.156:0/1233941909 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fcfe8085820 con 0x7fd00006a270 2023-11-28T11:09:06.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.052+0000 7fcffeffd700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcfe00682f0 0x7fcfe006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.052+0000 7fcffeffd700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcfe00682f0 0x7fcfe006a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009c40 tx=0x7fcff0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.056+0000 7fcff7fff700 1 -- 172.21.15.156:0/1233941909 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcfe8051ff0 con 0x7fd00006a270 2023-11-28T11:09:06.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.238+0000 7fd006226700 1 -- 172.21.15.156:0/1233941909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd00006db70 con 0x7fd00006a270 2023-11-28T11:09:06.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.900+0000 7fcff7fff700 1 -- 172.21.15.156:0/1233941909 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcfe8055610 con 0x7fd00006a270 2023-11-28T11:09:06.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.904+0000 7fd006226700 1 -- 172.21.15.156:0/1233941909 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcfe00682f0 msgr2=0x7fcfe006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.904+0000 7fd006226700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcfe00682f0 0x7fcfe006a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009c40 tx=0x7fcff0008030 comp rx=0 tx=0).stop 2023-11-28T11:09:06.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.904+0000 7fd006226700 1 -- 172.21.15.156:0/1233941909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd00006a270 msgr2=0x7fd0001495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.904+0000 7fd006226700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd00006a270 0x7fd0001495b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800c760 tx=0x7fcfe800cb20 comp rx=0 tx=0).stop 2023-11-28T11:09:06.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.904+0000 7fd006226700 1 -- 172.21.15.156:0/1233941909 shutdown_connections 2023-11-28T11:09:06.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.904+0000 7fd006226700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0000d0610 0x7fd00014b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.904+0000 7fd006226700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd00006a270 0x7fd0001495b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.904+0000 7fd006226700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcfe00682f0 0x7fcfe006a7a0 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009c40 tx=0x7fcff0008030 comp rx=0 tx=0).stop 2023-11-28T11:09:06.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.904+0000 7fd006226700 1 --2- 172.21.15.156:0/1233941909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd000071c00 0x7fd00014c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.904+0000 7fd006226700 1 -- 172.21.15.156:0/1233941909 >> 172.21.15.156:0/1233941909 conn(0x7fd00006b380 msgr2=0x7fd00015a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.905+0000 7fd006226700 1 -- 172.21.15.156:0/1233941909 shutdown_connections 2023-11-28T11:09:06.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:06.905+0000 7fd006226700 1 -- 172.21.15.156:0/1233941909 wait complete. 2023-11-28T11:09:06.909 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:09:06.921 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:09:07.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.047+0000 7fa7ef5eb700 1 Processor -- start 2023-11-28T11:09:07.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.047+0000 7fa7ef5eb700 1 -- start start 2023-11-28T11:09:07.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ef5eb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa7e80af060 0x7fa7e8070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:07.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ef5eb700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa7e80af430 0x7fa7e806d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:07.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ef5eb700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7e806a270 0x7fa7e806a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:07.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ef5eb700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e806dee0 con 0x7fa7e80af060 2023-11-28T11:09:07.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ef5eb700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e806abd0 con 0x7fa7e80af430 2023-11-28T11:09:07.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ef5eb700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80713c0 con 0x7fa7e806a270 2023-11-28T11:09:07.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ed387700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa7e80af060 0x7fa7e8070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:07.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ed387700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa7e80af060 0x7fa7e8070d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33430/0 (socket says 172.21.15.156:33430) 2023-11-28T11:09:07.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ed387700 1 -- 172.21.15.156:0/1217152680 learned_addr learned my addr 172.21.15.156:0/1217152680 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:07.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7edb88700 1 --2- 172.21.15.156:0/1217152680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7e806a270 0x7fa7e806a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:07.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ecb86700 1 --2- 172.21.15.156:0/1217152680 >> v2:172.21.15.186:3300/0 conn(0x7fa7e80af430 0x7fa7e806d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:07.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ed387700 1 -- 172.21.15.156:0/1217152680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7e806a270 msgr2=0x7fa7e806a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:07.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ed387700 1 --2- 172.21.15.156:0/1217152680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7e806a270 0x7fa7e806a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ed387700 1 -- 172.21.15.156:0/1217152680 >> v2:172.21.15.186:3300/0 conn(0x7fa7e80af430 msgr2=0x7fa7e806d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:07.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ed387700 1 --2- 172.21.15.156:0/1217152680 >> v2:172.21.15.186:3300/0 conn(0x7fa7e80af430 0x7fa7e806d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ed387700 1 -- 172.21.15.156:0/1217152680 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e80716a0 con 0x7fa7e80af060 2023-11-28T11:09:07.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.048+0000 7fa7ed387700 1 --2- 172.21.15.156:0/1217152680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7e80af060 0x7fa7e8070d50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80af800 tx=0x7fa7e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1210b0c543487dc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:07.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.049+0000 7fa7dffff700 1 -- 172.21.15.156:0/1217152680 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e400b870 con 0x7fa7e80af060 2023-11-28T11:09:07.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.049+0000 7fa7dffff700 1 -- 172.21.15.156:0/1217152680 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e400b9d0 con 0x7fa7e80af060 2023-11-28T11:09:07.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.049+0000 7fa7dffff700 1 -- 172.21.15.156:0/1217152680 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e400bd60 con 0x7fa7e80af060 2023-11-28T11:09:07.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.049+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/1217152680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7e80af060 msgr2=0x7fa7e8070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:07.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.049+0000 7fa7ef5eb700 1 --2- 172.21.15.156:0/1217152680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7e80af060 0x7fa7e8070d50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80af800 tx=0x7fa7e4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.049+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/1217152680 shutdown_connections 2023-11-28T11:09:07.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.049+0000 7fa7ef5eb700 1 --2- 172.21.15.156:0/1217152680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7e806a270 0x7fa7e806a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.049+0000 7fa7ef5eb700 1 --2- 172.21.15.156:0/1217152680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7e80af060 0x7fa7e8070d50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.049+0000 7fa7ef5eb700 1 --2- 172.21.15.156:0/1217152680 >> v2:172.21.15.186:3300/0 conn(0x7fa7e80af430 0x7fa7e806d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.049+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/1217152680 >> 172.21.15.156:0/1217152680 conn(0x7fa7e806b380 msgr2=0x7fa7e806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:07.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.049+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/1217152680 shutdown_connections 2023-11-28T11:09:07.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.049+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/1217152680 wait complete. 2023-11-28T11:09:07.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.050+0000 7fa7ef5eb700 1 Processor -- start 2023-11-28T11:09:07.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.050+0000 7fa7ef5eb700 1 -- start start 2023-11-28T11:09:07.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.051+0000 7fa7ef5eb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7e806a270 0x7fa7e814e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:07.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.051+0000 7fa7ef5eb700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7e80af060 0x7fa7e814add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:07.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.051+0000 7fa7ef5eb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7e80af430 0x7fa7e80b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:07.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.051+0000 7fa7ef5eb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e815f900 con 0x7fa7e806a270 2023-11-28T11:09:07.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.051+0000 7fa7ef5eb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e814dd70 con 0x7fa7e80af430 2023-11-28T11:09:07.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.051+0000 7fa7ef5eb700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e814b310 con 0x7fa7e80af060 2023-11-28T11:09:07.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.051+0000 7fa7ed387700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7e806a270 0x7fa7e814e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:07.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.051+0000 7fa7ed387700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7e806a270 0x7fa7e814e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33436/0 (socket says 172.21.15.156:33436) 2023-11-28T11:09:07.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.051+0000 7fa7ed387700 1 -- 172.21.15.156:0/759078927 learned_addr learned my addr 172.21.15.156:0/759078927 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:07.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.051+0000 7fa7ecb86700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7e80af060 0x7fa7e814add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:07.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.051+0000 7fa7edb88700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7e80af430 0x7fa7e80b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:07.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7ed387700 1 -- 172.21.15.156:0/759078927 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7e80af060 msgr2=0x7fa7e814add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:07.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7ed387700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7e80af060 0x7fa7e814add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7ed387700 1 -- 172.21.15.156:0/759078927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7e80af430 msgr2=0x7fa7e80b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:07.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7ed387700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7e80af430 0x7fa7e80b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7ed387700 1 -- 172.21.15.156:0/759078927 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e4009550 con 0x7fa7e806a270 2023-11-28T11:09:07.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7ecb86700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7e80af060 0x7fa7e814add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:07.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7edb88700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7e80af430 0x7fa7e80b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:07.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7ed387700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7e806a270 0x7fa7e814e200 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4014040 tx=0x7fa7e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:07.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7ddffb700 1 -- 172.21.15.156:0/759078927 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4004520 con 0x7fa7e806a270 2023-11-28T11:09:07.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7ddffb700 1 -- 172.21.15.156:0/759078927 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e401e070 con 0x7fa7e806a270 2023-11-28T11:09:07.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7ddffb700 1 -- 172.21.15.156:0/759078927 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40038c0 con 0x7fa7e806a270 2023-11-28T11:09:07.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/759078927 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e8071c60 con 0x7fa7e806a270 2023-11-28T11:09:07.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.052+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/759078927 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e8071980 con 0x7fa7e806a270 2023-11-28T11:09:07.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.053+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/759078927 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e8071f40 con 0x7fa7e806a270 2023-11-28T11:09:07.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.054+0000 7fa7ddffb700 1 -- 172.21.15.156:0/759078927 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa7e400f050 con 0x7fa7e806a270 2023-11-28T11:09:07.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.054+0000 7fa7ddffb700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa7cc0682f0 0x7fa7cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:07.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.054+0000 7fa7ecb86700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa7cc0682f0 0x7fa7cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:07.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.054+0000 7fa7ddffb700 1 -- 172.21.15.156:0/759078927 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5768+0+0 (secure 0 0 0) 0x7fa7e4020070 con 0x7fa7e806a270 2023-11-28T11:09:07.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.054+0000 7fa7ecb86700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa7cc0682f0 0x7fa7cc06a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa7e806cd20 tx=0x7fa7d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:07.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.058+0000 7fa7ddffb700 1 -- 172.21.15.156:0/759078927 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa7e40b85e0 con 0x7fa7e806a270 2023-11-28T11:09:07.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.239+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/759078927 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa7e8068690 con 0x7fa7e806a270 2023-11-28T11:09:07.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.911+0000 7fa7ddffb700 1 -- 172.21.15.156:0/759078927 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa7e4053a90 con 0x7fa7e806a270 2023-11-28T11:09:07.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.915+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/759078927 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa7cc0682f0 msgr2=0x7fa7cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:07.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.915+0000 7fa7ef5eb700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa7cc0682f0 0x7fa7cc06a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa7e806cd20 tx=0x7fa7d8008040 comp rx=0 tx=0).stop 2023-11-28T11:09:07.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.915+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/759078927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7e806a270 msgr2=0x7fa7e814e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:07.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.915+0000 7fa7ef5eb700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7e806a270 0x7fa7e814e200 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4014040 tx=0x7fa7e4004750 comp rx=0 tx=0).stop 2023-11-28T11:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.919+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/759078927 shutdown_connections 2023-11-28T11:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.919+0000 7fa7ef5eb700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7e80af430 0x7fa7e80b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.919+0000 7fa7ef5eb700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7e806a270 0x7fa7e814e200 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.919+0000 7fa7ef5eb700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa7cc0682f0 0x7fa7cc06a7a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.919+0000 7fa7ef5eb700 1 --2- 172.21.15.156:0/759078927 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7e80af060 0x7fa7e814add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:07.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.919+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/759078927 >> 172.21.15.156:0/759078927 conn(0x7fa7e806b380 msgr2=0x7fa7e80ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:07.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.919+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/759078927 shutdown_connections 2023-11-28T11:09:07.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:07.919+0000 7fa7ef5eb700 1 -- 172.21.15.156:0/759078927 wait complete. 2023-11-28T11:09:07.922 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' removed 2023-11-28T11:09:07.935 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:09:07.936 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:09:07.936 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:09:07.936 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:07.936 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:09:07.936 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:09:07.936 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:09:07.937 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:07.937 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:09:07.937 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:09:07.937 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:09:07.937 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:07.937 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:09:07.937 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:09:07.938 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:09:07.938 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:07.946 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:09:07.946 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:09:07.947 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:09:07.947 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:09:07.948 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:09:07.948 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:09:07.949 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:09:07.949 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:09:07.988 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:09:07.989 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:07.990 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:09:07.990 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:07.991 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:09:07.991 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:07.991 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:09:07.992 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:07.992 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:09:08.069 INFO:tasks.ceph.mds.a.smithi156.stdout:starting mds.a at 2023-11-28T11:09:08.070 INFO:tasks.ceph.mds.c.smithi156.stdout:starting mds.c at 2023-11-28T11:09:08.070 INFO:tasks.ceph.mds.a.smithi156.stderr:2023-11-28T11:09:08.068+0000 7f656740e700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-28T11:09:08.075 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2023-11-28T11:09:08.075 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-11-28T11:09:08.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.125+0000 7fbc69694700 1 Processor -- start 2023-11-28T11:09:08.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.125+0000 7fbc69694700 1 -- start start 2023-11-28T11:09:08.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.126+0000 7fbc69694700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc640cd8c0 0x7fbc64140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.126+0000 7fbc69694700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbc640cdc90 0x7fbc640b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.126+0000 7fbc69694700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6413ff00 con 0x7fbc640b2e70 2023-11-28T11:09:08.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.126+0000 7fbc69694700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc641404c0 con 0x7fbc640cdc90 2023-11-28T11:09:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.126+0000 7fbc69694700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6415aa80 con 0x7fbc640cd8c0 2023-11-28T11:09:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc627fc700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbc640cdc90 0x7fbc640b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc62ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc640cd8c0 0x7fbc64140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc637fe700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fbc640b2e70 0x7fbc6406f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:58708/0 (socket says 172.21.15.156:58708) 2023-11-28T11:09:08.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc627fc700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbc640cdc90 0x7fbc640b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:44480/0 (socket says 172.21.15.156:44480) 2023-11-28T11:09:08.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc637fe700 1 -- 172.21.15.156:0/2046189678 learned_addr learned my addr 172.21.15.156:0/2046189678 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:08.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc61ffb700 1 -- 172.21.15.156:0/2046189678 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331713501 0 0) 0x7fbc6413ff00 con 0x7fbc640b2e70 2023-11-28T11:09:08.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc61ffb700 1 -- 172.21.15.156:0/2046189678 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc500035b0 con 0x7fbc640b2e70 2023-11-28T11:09:08.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc62ffd700 1 -- 172.21.15.156:0/2046189678 >> v2:172.21.15.186:3300/0 conn(0x7fbc640cdc90 msgr2=0x7fbc640b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc62ffd700 1 --2- 172.21.15.156:0/2046189678 >> v2:172.21.15.186:3300/0 conn(0x7fbc640cdc90 0x7fbc640b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc62ffd700 1 -- 172.21.15.156:0/2046189678 >> v1:172.21.15.186:6789/0 conn(0x7fbc640b2e70 legacy=0x7fbc6406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc62ffd700 1 -- 172.21.15.156:0/2046189678 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640b3310 con 0x7fbc640cd8c0 2023-11-28T11:09:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc627fc700 1 --2- 172.21.15.156:0/2046189678 >> v2:172.21.15.186:3300/0 conn(0x7fbc640cdc90 0x7fbc640b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc62ffd700 1 --2- 172.21.15.156:0/2046189678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc640cd8c0 0x7fbc64140950 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009de0 tx=0x7fbc5400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1236e26dd9bc1222 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc61ffb700 1 -- 172.21.15.156:0/2046189678 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5400e070 con 0x7fbc640cd8c0 2023-11-28T11:09:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc61ffb700 1 -- 172.21.15.156:0/2046189678 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5400bb20 con 0x7fbc640cd8c0 2023-11-28T11:09:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.127+0000 7fbc61ffb700 1 -- 172.21.15.156:0/2046189678 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc54012430 con 0x7fbc640cd8c0 2023-11-28T11:09:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.128+0000 7fbc69694700 1 -- 172.21.15.156:0/2046189678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc640cd8c0 msgr2=0x7fbc64140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.128+0000 7fbc69694700 1 --2- 172.21.15.156:0/2046189678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc640cd8c0 0x7fbc64140950 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009de0 tx=0x7fbc5400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.128+0000 7fbc69694700 1 -- 172.21.15.156:0/2046189678 shutdown_connections 2023-11-28T11:09:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.128+0000 7fbc69694700 1 --2- 172.21.15.156:0/2046189678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc640cd8c0 0x7fbc64140950 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.128+0000 7fbc69694700 1 --2- 172.21.15.156:0/2046189678 >> v2:172.21.15.186:3300/0 conn(0x7fbc640cdc90 0x7fbc640b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.128+0000 7fbc69694700 1 -- 172.21.15.156:0/2046189678 >> 172.21.15.156:0/2046189678 conn(0x7fbc6406aed0 msgr2=0x7fbc6406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.128+0000 7fbc69694700 1 -- 172.21.15.156:0/2046189678 shutdown_connections 2023-11-28T11:09:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.128+0000 7fbc69694700 1 -- 172.21.15.156:0/2046189678 wait complete. 2023-11-28T11:09:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.129+0000 7fbc69694700 1 Processor -- start 2023-11-28T11:09:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.129+0000 7fbc69694700 1 -- start start 2023-11-28T11:09:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc69694700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc640b2e70 0x7fbc640c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc69694700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc640cd8c0 0x7fbc641550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc69694700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc640cdc90 0x7fbc6406bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc69694700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6406cbc0 con 0x7fbc640cd8c0 2023-11-28T11:09:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc69694700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6406cea0 con 0x7fbc640cdc90 2023-11-28T11:09:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc69694700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6406d180 con 0x7fbc640b2e70 2023-11-28T11:09:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc627fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc640cd8c0 0x7fbc641550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc627fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc640cd8c0 0x7fbc641550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33474/0 (socket says 172.21.15.156:33474) 2023-11-28T11:09:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc627fc700 1 -- 172.21.15.156:0/775860224 learned_addr learned my addr 172.21.15.156:0/775860224 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:08.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc637fe700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc640cdc90 0x7fbc6406bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc62ffd700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc640b2e70 0x7fbc640c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc627fc700 1 -- 172.21.15.156:0/775860224 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc640b2e70 msgr2=0x7fbc640c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc627fc700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc640b2e70 0x7fbc640c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc627fc700 1 -- 172.21.15.156:0/775860224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc640cdc90 msgr2=0x7fbc6406bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.130+0000 7fbc627fc700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc640cdc90 0x7fbc6406bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.131+0000 7fbc627fc700 1 -- 172.21.15.156:0/775860224 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c0094a0 con 0x7fbc640cd8c0 2023-11-28T11:09:08.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.131+0000 7fbc627fc700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc640cd8c0 0x7fbc641550b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00ebb0 tx=0x7fbc4c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.131+0000 7fbc5b7fe700 1 -- 172.21.15.156:0/775860224 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c00c850 con 0x7fbc640cd8c0 2023-11-28T11:09:08.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.131+0000 7fbc69694700 1 -- 172.21.15.156:0/775860224 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc54009a70 con 0x7fbc640cd8c0 2023-11-28T11:09:08.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.131+0000 7fbc5b7fe700 1 -- 172.21.15.156:0/775860224 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c00c9b0 con 0x7fbc640cd8c0 2023-11-28T11:09:08.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.131+0000 7fbc5b7fe700 1 -- 172.21.15.156:0/775860224 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c00cd40 con 0x7fbc640cd8c0 2023-11-28T11:09:08.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.131+0000 7fbc69694700 1 -- 172.21.15.156:0/775860224 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc64140150 con 0x7fbc640cd8c0 2023-11-28T11:09:08.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.132+0000 7fbc69694700 1 -- 172.21.15.156:0/775860224 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc640b35f0 con 0x7fbc640cd8c0 2023-11-28T11:09:08.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.133+0000 7fbc5b7fe700 1 -- 172.21.15.156:0/775860224 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbc4c019030 con 0x7fbc640cd8c0 2023-11-28T11:09:08.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.133+0000 7fbc5b7fe700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbc440682f0 0x7fbc4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.133+0000 7fbc5b7fe700 1 -- 172.21.15.156:0/775860224 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbc4c086020 con 0x7fbc640cd8c0 2023-11-28T11:09:08.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.133+0000 7fbc62ffd700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbc440682f0 0x7fbc4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.134+0000 7fbc62ffd700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbc440682f0 0x7fbc4406a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbc54014040 tx=0x7fbc54011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.136+0000 7fbc5b7fe700 1 -- 172.21.15.156:0/775860224 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbc4c052b80 con 0x7fbc640cd8c0 2023-11-28T11:09:08.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:08.335+0000 7fbc69694700 1 -- 172.21.15.156:0/775860224 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbc640ce100 con 0x7fbc640cd8c0 2023-11-28T11:09:09.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.124+0000 7fbc5b7fe700 1 -- 172.21.15.156:0/775860224 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbc4c017080 con 0x7fbc640cd8c0 2023-11-28T11:09:09.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.127+0000 7fbc69694700 1 -- 172.21.15.156:0/775860224 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbc440682f0 msgr2=0x7fbc4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.127+0000 7fbc69694700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbc440682f0 0x7fbc4406a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbc54014040 tx=0x7fbc54011000 comp rx=0 tx=0).stop 2023-11-28T11:09:09.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.127+0000 7fbc69694700 1 -- 172.21.15.156:0/775860224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc640cd8c0 msgr2=0x7fbc641550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.127+0000 7fbc69694700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc640cd8c0 0x7fbc641550b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00ebb0 tx=0x7fbc4c00eec0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.127+0000 7fbc69694700 1 -- 172.21.15.156:0/775860224 shutdown_connections 2023-11-28T11:09:09.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.127+0000 7fbc69694700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc640cdc90 0x7fbc6406bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.127+0000 7fbc69694700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbc440682f0 0x7fbc4406a7a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.127+0000 7fbc69694700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc640b2e70 0x7fbc640c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.127+0000 7fbc69694700 1 --2- 172.21.15.156:0/775860224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc640cd8c0 0x7fbc641550b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.127+0000 7fbc69694700 1 -- 172.21.15.156:0/775860224 >> 172.21.15.156:0/775860224 conn(0x7fbc6406aed0 msgr2=0x7fbc640b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:09.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.127+0000 7fbc69694700 1 -- 172.21.15.156:0/775860224 shutdown_connections 2023-11-28T11:09:09.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.127+0000 7fbc69694700 1 -- 172.21.15.156:0/775860224 wait complete. 2023-11-28T11:09:09.133 INFO:teuthology.orchestra.run.smithi156.stderr: removed all blocklist entries 2023-11-28T11:09:09.141 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:09:09.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.263+0000 7f90526eb700 1 Processor -- start 2023-11-28T11:09:09.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.263+0000 7f90526eb700 1 -- start start 2023-11-28T11:09:09.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.263+0000 7f90526eb700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904c0ce040 0x7f904c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.263+0000 7f90526eb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f904c0b2930 0x7f904c0b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.263+0000 7f90526eb700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f904c06b420 con 0x7f904c0d0a00 2023-11-28T11:09:09.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.263+0000 7f90526eb700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904c0b32f0 con 0x7f904c0b2930 2023-11-28T11:09:09.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.263+0000 7f90526eb700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904c06ff60 con 0x7f904c0ce040 2023-11-28T11:09:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f904b7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f904c0d0a00 0x7f904c06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48364/0 (socket says 172.21.15.156:48364) 2023-11-28T11:09:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f904b7fe700 1 -- 172.21.15.156:0/3853985258 learned_addr learned my addr 172.21.15.156:0/3853985258 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f9050c88700 1 --2- 172.21.15.156:0/3853985258 >> v2:172.21.15.156:3300/0 conn(0x7f904c0b2930 0x7f904c0b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f904affd700 1 -- 172.21.15.156:0/3853985258 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420788520 0 0) 0x7f904c06b420 con 0x7f904c0d0a00 2023-11-28T11:09:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f904affd700 1 -- 172.21.15.156:0/3853985258 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90340035b0 con 0x7f904c0d0a00 2023-11-28T11:09:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f904bfff700 1 --2- 172.21.15.156:0/3853985258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904c0ce040 0x7f904c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f9050c88700 1 -- 172.21.15.156:0/3853985258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904c0ce040 msgr2=0x7f904c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f9050c88700 1 --2- 172.21.15.156:0/3853985258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904c0ce040 0x7f904c15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f9050c88700 1 -- 172.21.15.156:0/3853985258 >> v1:172.21.15.156:6789/0 conn(0x7f904c0d0a00 legacy=0x7f904c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f9050c88700 1 -- 172.21.15.156:0/3853985258 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c06b700 con 0x7f904c0b2930 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f904bfff700 1 --2- 172.21.15.156:0/3853985258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904c0ce040 0x7f904c15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f9050c88700 1 --2- 172.21.15.156:0/3853985258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f904c0b2930 0x7f904c0b2db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9044009d90 tx=0x7f904400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3f14fadbfb4074 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f904affd700 1 -- 172.21.15.156:0/3853985258 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904400e070 con 0x7f904c0b2930 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f904affd700 1 -- 172.21.15.156:0/3853985258 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904400bb20 con 0x7f904c0b2930 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.264+0000 7f904affd700 1 -- 172.21.15.156:0/3853985258 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044012430 con 0x7f904c0b2930 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.265+0000 7f90526eb700 1 -- 172.21.15.156:0/3853985258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f904c0b2930 msgr2=0x7f904c0b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.265+0000 7f90526eb700 1 --2- 172.21.15.156:0/3853985258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f904c0b2930 0x7f904c0b2db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9044009d90 tx=0x7f904400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:09.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.265+0000 7f90526eb700 1 -- 172.21.15.156:0/3853985258 shutdown_connections 2023-11-28T11:09:09.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.265+0000 7f90526eb700 1 --2- 172.21.15.156:0/3853985258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f904c0b2930 0x7f904c0b2db0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.265+0000 7f90526eb700 1 --2- 172.21.15.156:0/3853985258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904c0ce040 0x7f904c15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.265+0000 7f90526eb700 1 -- 172.21.15.156:0/3853985258 >> 172.21.15.156:0/3853985258 conn(0x7f904c06aed0 msgr2=0x7f904c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:09.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.265+0000 7f90526eb700 1 -- 172.21.15.156:0/3853985258 shutdown_connections 2023-11-28T11:09:09.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.265+0000 7f90526eb700 1 -- 172.21.15.156:0/3853985258 wait complete. 2023-11-28T11:09:09.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.266+0000 7f90526eb700 1 Processor -- start 2023-11-28T11:09:09.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.266+0000 7f90526eb700 1 -- start start 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f90526eb700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904c0b2930 0x7f904c1f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f90526eb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f904c0ce040 0x7f904c0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f90526eb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f904c0d0a00 0x7f904c0c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f90526eb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c1f5a60 con 0x7f904c0ce040 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f90526eb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c1f2e70 con 0x7f904c0d0a00 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f90526eb700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904c1f63c0 con 0x7f904c0b2930 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f904b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f904c0ce040 0x7f904c0c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f904b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f904c0ce040 0x7f904c0c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33492/0 (socket says 172.21.15.156:33492) 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f904b7fe700 1 -- 172.21.15.156:0/2254739806 learned_addr learned my addr 172.21.15.156:0/2254739806 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f904bfff700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904c0b2930 0x7f904c1f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f9050c88700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f904c0d0a00 0x7f904c0c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f904b7fe700 1 -- 172.21.15.156:0/2254739806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904c0b2930 msgr2=0x7f904c1f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f904b7fe700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904c0b2930 0x7f904c1f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f904b7fe700 1 -- 172.21.15.156:0/2254739806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f904c0d0a00 msgr2=0x7f904c0c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f904b7fe700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f904c0d0a00 0x7f904c0c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f904b7fe700 1 -- 172.21.15.156:0/2254739806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9044009a70 con 0x7f904c0ce040 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f9050c88700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f904c0d0a00 0x7f904c0c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f904bfff700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904c0b2930 0x7f904c1f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.267+0000 7f904b7fe700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f904c0ce040 0x7f904c0c5be0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f904000c700 tx=0x7f904000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.268+0000 7f9048ff9700 1 -- 172.21.15.156:0/2254739806 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040005d70 con 0x7f904c0ce040 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.268+0000 7f9048ff9700 1 -- 172.21.15.156:0/2254739806 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90400070d0 con 0x7f904c0ce040 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.268+0000 7f90526eb700 1 -- 172.21.15.156:0/2254739806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904c06bcc0 con 0x7f904c0ce040 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.268+0000 7f9048ff9700 1 -- 172.21.15.156:0/2254739806 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040007460 con 0x7f904c0ce040 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.268+0000 7f90526eb700 1 -- 172.21.15.156:0/2254739806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904c06b9e0 con 0x7f904c0ce040 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.269+0000 7f90526eb700 1 -- 172.21.15.156:0/2254739806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904c06bfa0 con 0x7f904c0ce040 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.269+0000 7f9048ff9700 1 -- 172.21.15.156:0/2254739806 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9040019030 con 0x7f904c0ce040 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.269+0000 7f9048ff9700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90380682f0 0x7f903806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.270+0000 7f904bfff700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90380682f0 0x7f903806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.270+0000 7f9048ff9700 1 -- 172.21.15.156:0/2254739806 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9040084e80 con 0x7f904c0ce040 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.270+0000 7f904bfff700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90380682f0 0x7f903806a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f903c00aa10 tx=0x7f903c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.274+0000 7f9048ff9700 1 -- 172.21.15.156:0/2254739806 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f90400519b0 con 0x7f904c0ce040 2023-11-28T11:09:09.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.525+0000 7f90526eb700 1 -- 172.21.15.156:0/2254739806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f904c1f5ba0 con 0x7f904c0ce040 2023-11-28T11:09:09.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.525+0000 7f9048ff9700 1 -- 172.21.15.156:0/2254739806 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f904c1f5ba0 con 0x7f904c0ce040 2023-11-28T11:09:09.527 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:09.527 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:09:09.527 INFO:teuthology.orchestra.run.smithi156.stdout: "auth_dump": [ 2023-11-28T11:09:09.527 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.527 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.a", 2023-11-28T11:09:09.527 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlKLGIAhAAFJHYsC+LCoq5Qq1eh+MsgA==", 2023-11-28T11:09:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:09:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:09:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.b", 2023-11-28T11:09:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlr6ZnDxAAnJd4CtD5MUHauaV9GVqqVA==", 2023-11-28T11:09:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:09.529 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:09:09.529 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:09:09.529 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.529 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.529 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.529 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.529 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.c", 2023-11-28T11:09:09.529 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlygniCRAAI6YYq5P02LiCGC4/tdGwqg==", 2023-11-28T11:09:09.529 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.529 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:09.529 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:09:09.529 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:09:09.530 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.530 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.530 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.530 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.530 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.d", 2023-11-28T11:09:09.530 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlc4gEFxAA6J82vSOm+mN6hvRs7Ec8SA==", 2023-11-28T11:09:09.530 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.530 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:09.530 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:09:09.530 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:09:09.530 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.531 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.531 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.531 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.531 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.0", 2023-11-28T11:09:09.531 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDsyWVlhbi+ORAAG5EKPXefX8NjPR7TNi6Wgg==", 2023-11-28T11:09:09.531 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.531 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:09.531 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:09.532 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.532 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.532 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.532 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.532 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.1", 2023-11-28T11:09:09.532 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDvyWVlu+nEOBAAmvcXFjC+MNeWkDsZsI30PA==", 2023-11-28T11:09:09.532 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.532 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:09.532 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:09.532 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.532 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.533 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.533 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.533 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.2", 2023-11-28T11:09:09.533 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDyyWVlYMdIOBAABzlpv33IMXmgXqwROZ1n7A==", 2023-11-28T11:09:09.533 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.533 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:09.533 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:09.533 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.533 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.533 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.533 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.533 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.3", 2023-11-28T11:09:09.534 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD1yWVltreINxAAcPoQjsD28WEyEYN7kZZEUg==", 2023-11-28T11:09:09.534 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.534 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:09.534 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:09.534 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.534 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.534 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.534 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.534 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.4", 2023-11-28T11:09:09.534 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD8yWVlliukARAA3YVAwd8OePh6WT7H0Jykog==", 2023-11-28T11:09:09.534 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.535 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:09.535 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:09.535 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.535 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.535 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.535 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.535 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.5", 2023-11-28T11:09:09.535 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD/yWVl6nilARAAfFTZi6p2kdxqOgtWonRRKA==", 2023-11-28T11:09:09.535 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.535 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:09.535 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:09.536 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.536 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.536 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.536 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.536 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.6", 2023-11-28T11:09:09.536 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQACymVleb4IARAAXz8dDUDjem6eqAczZ5omDg==", 2023-11-28T11:09:09.536 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.536 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:09.536 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:09.536 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.536 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.536 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.7", 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAFymVlloyRABAAMa2gdE3weqqPWgiCI28jbQ==", 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.0", 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVl5LhsGxAA7WC9xFoSeBWdUPsmO5QaSg==", 2023-11-28T11:09:09.537 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.538 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:09.538 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T11:09:09.538 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T11:09:09.538 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T11:09:09.538 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.538 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.538 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.538 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.1", 2023-11-28T11:09:09.538 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlXV40IhAAhx1t1uwqXPA2g7GmoSXVmA==", 2023-11-28T11:09:09.539 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.539 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:09.539 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T11:09:09.539 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T11:09:09.539 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T11:09:09.539 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.539 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.539 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.539 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.2", 2023-11-28T11:09:09.539 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlnJDEHhAAEZX8g8t6Gbm6InUWOLqetw==", 2023-11-28T11:09:09.539 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.539 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:09.540 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T11:09:09.540 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T11:09:09.540 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T11:09:09.540 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.540 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.540 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.540 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.3", 2023-11-28T11:09:09.540 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlPtmDJRAAoEvqSTpIIIWm42myVE3jVw==", 2023-11-28T11:09:09.540 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.540 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:09.540 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T11:09:09.540 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T11:09:09.541 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T11:09:09.541 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.541 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.541 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.541 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.admin", 2023-11-28T11:09:09.541 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDqyWVl8mP6HRAAudHt/W3jSGkgKdYHCgM2uQ==", 2023-11-28T11:09:09.541 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.541 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T11:09:09.541 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:09:09.541 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:09:09.541 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.541 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.542 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.542 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.542 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.bootstrap-mds", 2023-11-28T11:09:09.542 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAKymVlAVV5OhAAHiuHyswlpyKhdQ2LQNaG7g==", 2023-11-28T11:09:09.542 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.542 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile bootstrap-mds" 2023-11-28T11:09:09.542 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.542 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.542 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.542 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.bootstrap-mgr", 2023-11-28T11:09:09.542 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAKymVlf115OhAAgVGKb8OHjPrwMV84BMq9+Q==", 2023-11-28T11:09:09.542 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.543 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-28T11:09:09.543 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.543 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.543 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.543 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.bootstrap-osd", 2023-11-28T11:09:09.543 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAKymVlxWR5OhAARHcryUSyS6fg0g206vWnaw==", 2023-11-28T11:09:09.543 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.543 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile bootstrap-osd" 2023-11-28T11:09:09.543 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.543 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.543 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.543 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.bootstrap-rbd", 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAKymVlqWx5OhAAed0cMNHzffMzMVhjeb5Rmg==", 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAKymVl7HN5OhAAmzosVgfWd59+Bp9gljv/uQ==", 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.544 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.545 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.bootstrap-rgw", 2023-11-28T11:09:09.545 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAKymVlRHt5OhAARk4XKqhj0hGn7z1BV65raw==", 2023-11-28T11:09:09.545 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.545 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-28T11:09:09.545 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.545 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.545 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.545 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mgr.x", 2023-11-28T11:09:09.545 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDqyWVly3m5OhAAWirKz8b4mHzNgLftULkWQg==", 2023-11-28T11:09:09.545 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.545 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T11:09:09.546 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile mgr", 2023-11-28T11:09:09.546 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.546 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.546 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:09.546 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:09.546 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mgr.y", 2023-11-28T11:09:09.546 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDqyWVlI+zRMxAANSXxnBj90PtANlg25RwW1A==", 2023-11-28T11:09:09.546 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:09.546 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T11:09:09.546 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile mgr", 2023-11-28T11:09:09.546 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:09.546 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.547 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:09.547 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2023-11-28T11:09:09.547 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:09:09.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.527+0000 7f90526eb700 1 -- 172.21.15.156:0/2254739806 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90380682f0 msgr2=0x7f903806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.527+0000 7f90526eb700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90380682f0 0x7f903806a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f903c00aa10 tx=0x7f903c008040 comp rx=0 tx=0).stop 2023-11-28T11:09:09.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.527+0000 7f90526eb700 1 -- 172.21.15.156:0/2254739806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f904c0ce040 msgr2=0x7f904c0c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.527+0000 7f90526eb700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f904c0ce040 0x7f904c0c5be0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f904000c700 tx=0x7f904000cac0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.527+0000 7f90526eb700 1 -- 172.21.15.156:0/2254739806 shutdown_connections 2023-11-28T11:09:09.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.527+0000 7f90526eb700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f904c0d0a00 0x7f904c0c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.527+0000 7f90526eb700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90380682f0 0x7f903806a7a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.527+0000 7f90526eb700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904c0b2930 0x7f904c1f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.527+0000 7f90526eb700 1 --2- 172.21.15.156:0/2254739806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f904c0ce040 0x7f904c0c5be0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.527+0000 7f90526eb700 1 -- 172.21.15.156:0/2254739806 >> 172.21.15.156:0/2254739806 conn(0x7f904c06aed0 msgr2=0x7f904c15de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:09.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.528+0000 7f90526eb700 1 -- 172.21.15.156:0/2254739806 shutdown_connections 2023-11-28T11:09:09.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.528+0000 7f90526eb700 1 -- 172.21.15.156:0/2254739806 wait complete. 2023-11-28T11:09:09.550 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-28T11:09:09.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.670+0000 7f6bd1e1f700 1 Processor -- start 2023-11-28T11:09:09.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.670+0000 7f6bd1e1f700 1 -- start start 2023-11-28T11:09:09.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bd1e1f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6bcc154e70 0x7f6bcc0cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bd1e1f700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6bcc071750 0x7f6bcc15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bd1e1f700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bcc155240 con 0x7f6bcc0cf360 2023-11-28T11:09:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bd1e1f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bcc15b100 con 0x7f6bcc154e70 2023-11-28T11:09:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bd1e1f700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bcc15b3e0 con 0x7f6bcc071750 2023-11-28T11:09:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bcb7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6bcc154e70 0x7f6bcc0cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bcb7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6bcc154e70 0x7f6bcc0cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33500/0 (socket says 172.21.15.156:33500) 2023-11-28T11:09:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bcb7fe700 1 -- 172.21.15.156:0/1724000958 learned_addr learned my addr 172.21.15.156:0/1724000958 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bcbfff700 1 --2- 172.21.15.156:0/1724000958 >> v2:172.21.15.186:3300/0 conn(0x7f6bcc071750 0x7f6bcc15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bcb7fe700 1 -- 172.21.15.156:0/1724000958 >> v2:172.21.15.186:3300/0 conn(0x7f6bcc071750 msgr2=0x7f6bcc15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bcb7fe700 1 --2- 172.21.15.156:0/1724000958 >> v2:172.21.15.186:3300/0 conn(0x7f6bcc071750 0x7f6bcc15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bcb7fe700 1 -- 172.21.15.156:0/1724000958 >> v1:172.21.15.186:6789/0 conn(0x7f6bcc0cf360 legacy=0x7f6bcc071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bcb7fe700 1 -- 172.21.15.156:0/1724000958 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc15b6c0 con 0x7f6bcc154e70 2023-11-28T11:09:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bcbfff700 1 --2- 172.21.15.156:0/1724000958 >> v2:172.21.15.186:3300/0 conn(0x7f6bcc071750 0x7f6bcc15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.671+0000 7f6bcb7fe700 1 --2- 172.21.15.156:0/1724000958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bcc154e70 0x7f6bcc0cdbd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0cb0a0 tx=0x7f6bb4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb2c51f1c8c8815 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.672+0000 7f6bca7fc700 1 -- 172.21.15.156:0/1724000958 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4004230 con 0x7f6bcc154e70 2023-11-28T11:09:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.672+0000 7f6bca7fc700 1 -- 172.21.15.156:0/1724000958 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb4004390 con 0x7f6bcc154e70 2023-11-28T11:09:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.672+0000 7f6bca7fc700 1 -- 172.21.15.156:0/1724000958 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40036a0 con 0x7f6bcc154e70 2023-11-28T11:09:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.672+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/1724000958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bcc154e70 msgr2=0x7f6bcc0cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.672+0000 7f6bd1e1f700 1 --2- 172.21.15.156:0/1724000958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bcc154e70 0x7f6bcc0cdbd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0cb0a0 tx=0x7f6bb4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.672+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/1724000958 shutdown_connections 2023-11-28T11:09:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.672+0000 7f6bd1e1f700 1 --2- 172.21.15.156:0/1724000958 >> v2:172.21.15.186:3300/0 conn(0x7f6bcc071750 0x7f6bcc15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.672+0000 7f6bd1e1f700 1 --2- 172.21.15.156:0/1724000958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bcc154e70 0x7f6bcc0cdbd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.672+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/1724000958 >> 172.21.15.156:0/1724000958 conn(0x7f6bcc0d0000 msgr2=0x7f6bcc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.672+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/1724000958 shutdown_connections 2023-11-28T11:09:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.672+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/1724000958 wait complete. 2023-11-28T11:09:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.673+0000 7f6bd1e1f700 1 Processor -- start 2023-11-28T11:09:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.674+0000 7f6bd1e1f700 1 -- start start 2023-11-28T11:09:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.674+0000 7f6bd1e1f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bcc071750 0x7f6bcc1f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.674+0000 7f6bd1e1f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6bcc0cf360 0x7f6bcc157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.674+0000 7f6bd1e1f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bcc154e70 0x7f6bcc0b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.674+0000 7f6bd1e1f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc1f4ae0 con 0x7f6bcc071750 2023-11-28T11:09:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.674+0000 7f6bd1e1f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc158290 con 0x7f6bcc154e70 2023-11-28T11:09:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.674+0000 7f6bd1e1f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc0b4e60 con 0x7f6bcc0cf360 2023-11-28T11:09:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.674+0000 7f6bcb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bcc071750 0x7f6bcc1f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.674+0000 7f6bcb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bcc071750 0x7f6bcc1f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33506/0 (socket says 172.21.15.156:33506) 2023-11-28T11:09:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.674+0000 7f6bcb7fe700 1 -- 172.21.15.156:0/196046942 learned_addr learned my addr 172.21.15.156:0/196046942 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bcbfff700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bcc154e70 0x7f6bcc0b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bcaffd700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6bcc0cf360 0x7f6bcc157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bcb7fe700 1 -- 172.21.15.156:0/196046942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6bcc0cf360 msgr2=0x7f6bcc157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bcb7fe700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6bcc0cf360 0x7f6bcc157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bcb7fe700 1 -- 172.21.15.156:0/196046942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bcc154e70 msgr2=0x7f6bcc0b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bcb7fe700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bcc154e70 0x7f6bcc0b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bcb7fe700 1 -- 172.21.15.156:0/196046942 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb4009550 con 0x7f6bcc071750 2023-11-28T11:09:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bcbfff700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bcc154e70 0x7f6bcc0b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bcb7fe700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bcc071750 0x7f6bcc1f45a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc073bb0 tx=0x7f6bb4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bc3fff700 1 -- 172.21.15.156:0/196046942 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4011630 con 0x7f6bcc071750 2023-11-28T11:09:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bc3fff700 1 -- 172.21.15.156:0/196046942 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb4011790 con 0x7f6bcc071750 2023-11-28T11:09:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/196046942 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc0af520 con 0x7f6bcc071750 2023-11-28T11:09:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bc3fff700 1 -- 172.21.15.156:0/196046942 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4011b40 con 0x7f6bcc071750 2023-11-28T11:09:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.675+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/196046942 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc15b9a0 con 0x7f6bcc071750 2023-11-28T11:09:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.676+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/196046942 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bcc15c060 con 0x7f6bcc071750 2023-11-28T11:09:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.676+0000 7f6bc3fff700 1 -- 172.21.15.156:0/196046942 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6bb4017020 con 0x7f6bcc071750 2023-11-28T11:09:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.677+0000 7f6bc3fff700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6bac0682f0 0x7f6bac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.677+0000 7f6bc3fff700 1 -- 172.21.15.156:0/196046942 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6bb4086f60 con 0x7f6bcc071750 2023-11-28T11:09:09.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.677+0000 7f6bcaffd700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6bac0682f0 0x7f6bac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.677+0000 7f6bcaffd700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6bac0682f0 0x7f6bac06a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc1f3b20 tx=0x7f6bbc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.681+0000 7f6bc3fff700 1 -- 172.21.15.156:0/196046942 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6bb40b85e0 con 0x7f6bcc071750 2023-11-28T11:09:09.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.935+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/196046942 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f6bcc15bd80 con 0x7f6bcc071750 2023-11-28T11:09:09.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.938+0000 7f6bc3fff700 1 -- 172.21.15.156:0/196046942 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6bb401f0a0 con 0x7f6bcc071750 2023-11-28T11:09:09.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.940+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/196046942 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6bac0682f0 msgr2=0x7f6bac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.940+0000 7f6bd1e1f700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6bac0682f0 0x7f6bac06a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc1f3b20 tx=0x7f6bbc009040 comp rx=0 tx=0).stop 2023-11-28T11:09:09.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.940+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/196046942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bcc071750 msgr2=0x7f6bcc1f45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.940+0000 7f6bd1e1f700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bcc071750 0x7f6bcc1f45a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc073bb0 tx=0x7f6bb4004750 comp rx=0 tx=0).stop 2023-11-28T11:09:09.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.940+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/196046942 shutdown_connections 2023-11-28T11:09:09.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.940+0000 7f6bd1e1f700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bcc154e70 0x7f6bcc0b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.940+0000 7f6bd1e1f700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bcc071750 0x7f6bcc1f45a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.940+0000 7f6bd1e1f700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6bac0682f0 0x7f6bac06a7a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.940+0000 7f6bd1e1f700 1 --2- 172.21.15.156:0/196046942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6bcc0cf360 0x7f6bcc157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.940+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/196046942 >> 172.21.15.156:0/196046942 conn(0x7f6bcc0d0000 msgr2=0x7f6bcc15c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:09.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.941+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/196046942 shutdown_connections 2023-11-28T11:09:09.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:09.941+0000 7f6bd1e1f700 1 -- 172.21.15.156:0/196046942 wait complete. 2023-11-28T11:09:09.954 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-28T11:09:10.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.074+0000 7f3f292e4700 1 Processor -- start 2023-11-28T11:09:10.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.074+0000 7f3f292e4700 1 -- start start 2023-11-28T11:09:10.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.075+0000 7f3f292e4700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3f24160740 0x7f3f240cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.075+0000 7f3f292e4700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2406bbb0 con 0x7f3f24160b10 2023-11-28T11:09:10.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.075+0000 7f3f292e4700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f240cc030 con 0x7f3f2406b060 2023-11-28T11:09:10.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.075+0000 7f3f292e4700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f240ca720 con 0x7f3f24160740 2023-11-28T11:09:10.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.075+0000 7f3f227fc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3f24160b10 0x7f3f240cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48392/0 (socket says 172.21.15.156:48392) 2023-11-28T11:09:10.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.075+0000 7f3f227fc700 1 -- 172.21.15.156:0/2171008867 learned_addr learned my addr 172.21.15.156:0/2171008867 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:10.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f22ffd700 1 --2- 172.21.15.156:0/2171008867 >> v2:172.21.15.186:3300/0 conn(0x7f3f24160740 0x7f3f240cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812097609 0 0) 0x7f3f2406bbb0 con 0x7f3f24160b10 2023-11-28T11:09:10.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f0c0035b0 con 0x7f3f24160b10 2023-11-28T11:09:10.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 982069410 0 0) 0x7f3f0c0035b0 con 0x7f3f24160b10 2023-11-28T11:09:10.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 >> v2:172.21.15.186:3300/0 conn(0x7f3f24160740 msgr2=0x7f3f240cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 --2- 172.21.15.156:0/2171008867 >> v2:172.21.15.186:3300/0 conn(0x7f3f24160740 0x7f3f240cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 >> v1:172.21.15.186:6789/0 conn(0x7f3f2406b060 legacy=0x7f3f2406b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240c9e10 con 0x7f3f24160b10 2023-11-28T11:09:10.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f3f14001670 con 0x7f3f24160b10 2023-11-28T11:09:10.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 >> v1:172.21.15.156:6789/0 conn(0x7f3f24160b10 legacy=0x7f3f240cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 --2- 172.21.15.156:0/2171008867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f0c003a00 0x7f3f0c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 --2- 172.21.15.156:0/2171008867 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f0c004850 0x7f3f0c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 --2- 172.21.15.156:0/2171008867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f0c005200 0x7f3f0c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c00c050 con 0x7f3f0c005200 2023-11-28T11:09:10.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c00c1c0 con 0x7f3f0c003a00 2023-11-28T11:09:10.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c00c380 con 0x7f3f0c004850 2023-11-28T11:09:10.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f237fe700 1 --2- 172.21.15.156:0/2171008867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f0c005200 0x7f3f0c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f22ffd700 1 --2- 172.21.15.156:0/2171008867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f0c003a00 0x7f3f0c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f227fc700 1 --2- 172.21.15.156:0/2171008867 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f0c004850 0x7f3f0c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f237fe700 1 -- 172.21.15.156:0/2171008867 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f0c004850 msgr2=0x7f3f0c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f237fe700 1 --2- 172.21.15.156:0/2171008867 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f0c004850 0x7f3f0c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f237fe700 1 -- 172.21.15.156:0/2171008867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f0c003a00 msgr2=0x7f3f0c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f237fe700 1 --2- 172.21.15.156:0/2171008867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f0c003a00 0x7f3f0c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.076+0000 7f3f237fe700 1 -- 172.21.15.156:0/2171008867 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0035b0 con 0x7f3f0c005200 2023-11-28T11:09:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.077+0000 7f3f237fe700 1 -- 172.21.15.156:0/2171008867 reap_dead start 2023-11-28T11:09:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.077+0000 7f3f237fe700 1 --2- 172.21.15.156:0/2171008867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f0c005200 0x7f3f0c00ba10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c240 tx=0x7f3f0c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a06435c7a4a170 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.077+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c001df0 con 0x7f3f0c005200 2023-11-28T11:09:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.077+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c00cdc0 con 0x7f3f0c005200 2023-11-28T11:09:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.077+0000 7f3f21ffb700 1 -- 172.21.15.156:0/2171008867 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c003c40 con 0x7f3f0c005200 2023-11-28T11:09:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.077+0000 7f3f292e4700 1 -- 172.21.15.156:0/2171008867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f0c005200 msgr2=0x7f3f0c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.077+0000 7f3f292e4700 1 --2- 172.21.15.156:0/2171008867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f0c005200 0x7f3f0c00ba10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c240 tx=0x7f3f0c005880 comp rx=0 tx=0).stop 2023-11-28T11:09:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.077+0000 7f3f292e4700 1 -- 172.21.15.156:0/2171008867 shutdown_connections 2023-11-28T11:09:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.077+0000 7f3f292e4700 1 --2- 172.21.15.156:0/2171008867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f0c005200 0x7f3f0c00ba10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.077+0000 7f3f292e4700 1 -- 172.21.15.156:0/2171008867 >> 172.21.15.156:0/2171008867 conn(0x7f3f240b12b0 msgr2=0x7f3f240b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.077+0000 7f3f292e4700 1 -- 172.21.15.156:0/2171008867 shutdown_connections 2023-11-28T11:09:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.077+0000 7f3f292e4700 1 -- 172.21.15.156:0/2171008867 wait complete. 2023-11-28T11:09:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.079+0000 7f3f292e4700 1 Processor -- start 2023-11-28T11:09:10.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.079+0000 7f3f292e4700 1 -- start start 2023-11-28T11:09:10.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.079+0000 7f3f292e4700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f0c005200 0x7f3f240c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.079+0000 7f3f292e4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f240ce3a0 0x7f3f240ce810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.079+0000 7f3f292e4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f240c6b30 0x7f3f240c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.079+0000 7f3f292e4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f241f6c70 con 0x7f3f240c6b30 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.079+0000 7f3f292e4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f240ced50 con 0x7f3f240ce3a0 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.079+0000 7f3f292e4700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f241f62d0 con 0x7f3f0c005200 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f237fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f240c6b30 0x7f3f240c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f237fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f240c6b30 0x7f3f240c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33530/0 (socket says 172.21.15.156:33530) 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f237fe700 1 -- 172.21.15.156:0/843889053 learned_addr learned my addr 172.21.15.156:0/843889053 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f22ffd700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f0c005200 0x7f3f240c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f227fc700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f240ce3a0 0x7f3f240ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f237fe700 1 -- 172.21.15.156:0/843889053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f0c005200 msgr2=0x7f3f240c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f237fe700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f0c005200 0x7f3f240c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f237fe700 1 -- 172.21.15.156:0/843889053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f240ce3a0 msgr2=0x7f3f240ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f237fe700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f240ce3a0 0x7f3f240ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f237fe700 1 -- 172.21.15.156:0/843889053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240ca0d0 con 0x7f3f240c6b30 2023-11-28T11:09:10.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f237fe700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f240c6b30 0x7f3f240c6fa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c830 tx=0x7f3f1c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f0b7fe700 1 -- 172.21.15.156:0/843889053 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c005280 con 0x7f3f240c6b30 2023-11-28T11:09:10.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f0b7fe700 1 -- 172.21.15.156:0/843889053 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c005400 con 0x7f3f240c6b30 2023-11-28T11:09:10.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f292e4700 1 -- 172.21.15.156:0/843889053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2406efc0 con 0x7f3f240c6b30 2023-11-28T11:09:10.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f0b7fe700 1 -- 172.21.15.156:0/843889053 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c0057b0 con 0x7f3f240c6b30 2023-11-28T11:09:10.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.080+0000 7f3f292e4700 1 -- 172.21.15.156:0/843889053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f240ca3b0 con 0x7f3f240c6b30 2023-11-28T11:09:10.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.081+0000 7f3f292e4700 1 -- 172.21.15.156:0/843889053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f241f6db0 con 0x7f3f240c6b30 2023-11-28T11:09:10.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.082+0000 7f3f0b7fe700 1 -- 172.21.15.156:0/843889053 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f1c010050 con 0x7f3f240c6b30 2023-11-28T11:09:10.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.082+0000 7f3f0b7fe700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3f10068320 0x7f3f1006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.082+0000 7f3f0b7fe700 1 -- 172.21.15.156:0/843889053 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3f1c086560 con 0x7f3f240c6b30 2023-11-28T11:09:10.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.082+0000 7f3f22ffd700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3f10068320 0x7f3f1006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.083+0000 7f3f22ffd700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3f10068320 0x7f3f1006a7d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009be0 tx=0x7f3f18009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.086+0000 7f3f0b7fe700 1 -- 172.21.15.156:0/843889053 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f241f6db0 con 0x7f3f240c6b30 2023-11-28T11:09:10.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.343+0000 7f3f292e4700 1 -- 172.21.15.156:0/843889053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f3f241f6db0 con 0x7f3f240c6b30 2023-11-28T11:09:10.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.346+0000 7f3f0b7fe700 1 -- 172.21.15.156:0/843889053 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3f241f6db0 con 0x7f3f240c6b30 2023-11-28T11:09:10.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.348+0000 7f3f292e4700 1 -- 172.21.15.156:0/843889053 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3f10068320 msgr2=0x7f3f1006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.348+0000 7f3f292e4700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3f10068320 0x7f3f1006a7d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009be0 tx=0x7f3f18009210 comp rx=0 tx=0).stop 2023-11-28T11:09:10.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.348+0000 7f3f292e4700 1 -- 172.21.15.156:0/843889053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f240c6b30 msgr2=0x7f3f240c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.348+0000 7f3f292e4700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f240c6b30 0x7f3f240c6fa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c830 tx=0x7f3f1c0014a0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.348+0000 7f3f292e4700 1 -- 172.21.15.156:0/843889053 shutdown_connections 2023-11-28T11:09:10.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.348+0000 7f3f292e4700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f240c6b30 0x7f3f240c6fa0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.348+0000 7f3f292e4700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3f10068320 0x7f3f1006a7d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.348+0000 7f3f292e4700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f0c005200 0x7f3f240c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.348+0000 7f3f292e4700 1 --2- 172.21.15.156:0/843889053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f240ce3a0 0x7f3f240ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.348+0000 7f3f292e4700 1 -- 172.21.15.156:0/843889053 >> 172.21.15.156:0/843889053 conn(0x7f3f240b12b0 msgr2=0x7f3f240b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.349+0000 7f3f292e4700 1 -- 172.21.15.156:0/843889053 shutdown_connections 2023-11-28T11:09:10.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.349+0000 7f3f292e4700 1 -- 172.21.15.156:0/843889053 wait complete. 2023-11-28T11:09:10.362 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-28T11:09:10.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.482+0000 7f1686ee8700 1 Processor -- start 2023-11-28T11:09:10.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.483+0000 7f1686ee8700 1 -- start start 2023-11-28T11:09:10.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.483+0000 7f1686ee8700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1680071040 0x7f1680071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.483+0000 7f1686ee8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f168006dee0 0x7f168006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.483+0000 7f1686ee8700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1680071950 con 0x7f168006cdc0 2023-11-28T11:09:10.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.483+0000 7f1686ee8700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1680161550 con 0x7f168006dee0 2023-11-28T11:09:10.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.483+0000 7f1686ee8700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168015d060 con 0x7f1680071040 2023-11-28T11:09:10.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.484+0000 7f167ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f168006dee0 0x7f168006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.484+0000 7f167ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f168006dee0 0x7f168006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33536/0 (socket says 172.21.15.156:33536) 2023-11-28T11:09:10.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.484+0000 7f167ffff700 1 -- 172.21.15.156:0/953318615 learned_addr learned my addr 172.21.15.156:0/953318615 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:10.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.484+0000 7f1684c84700 1 --2- 172.21.15.156:0/953318615 >> v2:172.21.15.186:3300/0 conn(0x7f1680071040 0x7f1680071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.484+0000 7f167ffff700 1 -- 172.21.15.156:0/953318615 >> v2:172.21.15.186:3300/0 conn(0x7f1680071040 msgr2=0x7f1680071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.484+0000 7f167ffff700 1 --2- 172.21.15.156:0/953318615 >> v2:172.21.15.186:3300/0 conn(0x7f1680071040 0x7f1680071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.484+0000 7f167ffff700 1 -- 172.21.15.156:0/953318615 >> v1:172.21.15.186:6789/0 conn(0x7f168006cdc0 legacy=0x7f168006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.484+0000 7f167ffff700 1 -- 172.21.15.156:0/953318615 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168015d340 con 0x7f168006dee0 2023-11-28T11:09:10.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.484+0000 7f167ffff700 1 --2- 172.21.15.156:0/953318615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f168006dee0 0x7f168006e330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f16800d0f50 tx=0x7f166c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5073b88e798fa3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.484+0000 7f167f7fe700 1 -- 172.21.15.156:0/953318615 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c011830 con 0x7f168006dee0 2023-11-28T11:09:10.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.484+0000 7f167f7fe700 1 -- 172.21.15.156:0/953318615 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166c011990 con 0x7f168006dee0 2023-11-28T11:09:10.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.484+0000 7f167f7fe700 1 -- 172.21.15.156:0/953318615 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c011d20 con 0x7f168006dee0 2023-11-28T11:09:10.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.485+0000 7f1686ee8700 1 -- 172.21.15.156:0/953318615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f168006dee0 msgr2=0x7f168006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.485+0000 7f1686ee8700 1 --2- 172.21.15.156:0/953318615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f168006dee0 0x7f168006e330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f16800d0f50 tx=0x7f166c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.485+0000 7f1686ee8700 1 -- 172.21.15.156:0/953318615 shutdown_connections 2023-11-28T11:09:10.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.485+0000 7f1686ee8700 1 --2- 172.21.15.156:0/953318615 >> v2:172.21.15.186:3300/0 conn(0x7f1680071040 0x7f1680071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.485+0000 7f1686ee8700 1 --2- 172.21.15.156:0/953318615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f168006dee0 0x7f168006e330 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.485+0000 7f1686ee8700 1 -- 172.21.15.156:0/953318615 >> 172.21.15.156:0/953318615 conn(0x7f16800d0000 msgr2=0x7f16800d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.485+0000 7f1686ee8700 1 -- 172.21.15.156:0/953318615 shutdown_connections 2023-11-28T11:09:10.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.485+0000 7f1686ee8700 1 -- 172.21.15.156:0/953318615 wait complete. 2023-11-28T11:09:10.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.486+0000 7f1686ee8700 1 Processor -- start 2023-11-28T11:09:10.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.486+0000 7f1686ee8700 1 -- start start 2023-11-28T11:09:10.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f1686ee8700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f168006cdc0 0x7f16800b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f1686ee8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f168006dee0 0x7f168014a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f1686ee8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1680071040 0x7f16800ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f1686ee8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16800ae4f0 con 0x7f168006dee0 2023-11-28T11:09:10.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f1686ee8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168014a7e0 con 0x7f1680071040 2023-11-28T11:09:10.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f1686ee8700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16800ad8e0 con 0x7f168006cdc0 2023-11-28T11:09:10.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f167ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f168006dee0 0x7f168014a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f167ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f168006dee0 0x7f168014a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33544/0 (socket says 172.21.15.156:33544) 2023-11-28T11:09:10.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f167ffff700 1 -- 172.21.15.156:0/4070849007 learned_addr learned my addr 172.21.15.156:0/4070849007 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:10.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f1684c84700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f168006cdc0 0x7f16800b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f1685485700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1680071040 0x7f16800ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f167ffff700 1 -- 172.21.15.156:0/4070849007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f168006cdc0 msgr2=0x7f16800b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f167ffff700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f168006cdc0 0x7f16800b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f167ffff700 1 -- 172.21.15.156:0/4070849007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1680071040 msgr2=0x7f16800ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f167ffff700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1680071040 0x7f16800ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.487+0000 7f167ffff700 1 -- 172.21.15.156:0/4070849007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c009550 con 0x7f168006dee0 2023-11-28T11:09:10.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.488+0000 7f167ffff700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f168006dee0 0x7f168014a2a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f166c009c10 tx=0x7f166c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.488+0000 7f167d7fa700 1 -- 172.21.15.156:0/4070849007 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c004450 con 0x7f168006dee0 2023-11-28T11:09:10.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.488+0000 7f167d7fa700 1 -- 172.21.15.156:0/4070849007 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166c01e070 con 0x7f168006dee0 2023-11-28T11:09:10.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.488+0000 7f1686ee8700 1 -- 172.21.15.156:0/4070849007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168015d900 con 0x7f168006dee0 2023-11-28T11:09:10.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.488+0000 7f167d7fa700 1 -- 172.21.15.156:0/4070849007 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c0038e0 con 0x7f168006dee0 2023-11-28T11:09:10.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.488+0000 7f1686ee8700 1 -- 172.21.15.156:0/4070849007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168015d620 con 0x7f168006dee0 2023-11-28T11:09:10.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.489+0000 7f1686ee8700 1 -- 172.21.15.156:0/4070849007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168006e870 con 0x7f168006dee0 2023-11-28T11:09:10.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.489+0000 7f167d7fa700 1 -- 172.21.15.156:0/4070849007 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f166c00f050 con 0x7f168006dee0 2023-11-28T11:09:10.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.489+0000 7f167d7fa700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f16640682f0 0x7f166406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.490+0000 7f167d7fa700 1 -- 172.21.15.156:0/4070849007 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f166c0866c0 con 0x7f168006dee0 2023-11-28T11:09:10.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.490+0000 7f1684c84700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f16640682f0 0x7f166406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.490+0000 7f1684c84700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f16640682f0 0x7f166406a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1674009b40 tx=0x7f1674009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.494+0000 7f167d7fa700 1 -- 172.21.15.156:0/4070849007 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f166c053270 con 0x7f168006dee0 2023-11-28T11:09:10.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.753+0000 7f1686ee8700 1 -- 172.21.15.156:0/4070849007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f16800ae630 con 0x7f168006dee0 2023-11-28T11:09:10.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.757+0000 7f167d7fa700 1 -- 172.21.15.156:0/4070849007 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f166c0200a0 con 0x7f168006dee0 2023-11-28T11:09:10.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.759+0000 7f1686ee8700 1 -- 172.21.15.156:0/4070849007 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f16640682f0 msgr2=0x7f166406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.759+0000 7f1686ee8700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f16640682f0 0x7f166406a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1674009b40 tx=0x7f1674009210 comp rx=0 tx=0).stop 2023-11-28T11:09:10.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.759+0000 7f1686ee8700 1 -- 172.21.15.156:0/4070849007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f168006dee0 msgr2=0x7f168014a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.759+0000 7f1686ee8700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f168006dee0 0x7f168014a2a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f166c009c10 tx=0x7f166c004750 comp rx=0 tx=0).stop 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.759+0000 7f1686ee8700 1 -- 172.21.15.156:0/4070849007 shutdown_connections 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.759+0000 7f1686ee8700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1680071040 0x7f16800ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.759+0000 7f1686ee8700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f16640682f0 0x7f166406a7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.759+0000 7f1686ee8700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f168006cdc0 0x7f16800b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.759+0000 7f1686ee8700 1 --2- 172.21.15.156:0/4070849007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f168006dee0 0x7f168014a2a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.759+0000 7f1686ee8700 1 -- 172.21.15.156:0/4070849007 >> 172.21.15.156:0/4070849007 conn(0x7f16800d0000 msgr2=0x7f1680157db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.759+0000 7f1686ee8700 1 -- 172.21.15.156:0/4070849007 shutdown_connections 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.759+0000 7f1686ee8700 1 -- 172.21.15.156:0/4070849007 wait complete. 2023-11-28T11:09:10.772 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-28T11:09:10.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.892+0000 7f74a9a3e700 1 Processor -- start 2023-11-28T11:09:10.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.892+0000 7f74a9a3e700 1 -- start start 2023-11-28T11:09:10.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a9a3e700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f74a40cd7c0 0x7f74a40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a9a3e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a40d1950 con 0x7f74a415c9b0 2023-11-28T11:09:10.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a9a3e700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a40ce140 con 0x7f74a4161340 2023-11-28T11:09:10.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a9a3e700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a40b5800 con 0x7f74a40cd7c0 2023-11-28T11:09:10.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a2ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f74a415c9b0 0x7f74a40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48426/0 (socket says 172.21.15.156:48426) 2023-11-28T11:09:10.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a2ffd700 1 -- 172.21.15.156:0/1361823461 learned_addr learned my addr 172.21.15.156:0/1361823461 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:10.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189342814 0 0) 0x7f74a40d1950 con 0x7f74a415c9b0 2023-11-28T11:09:10.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a37fe700 1 --2- 172.21.15.156:0/1361823461 >> v2:172.21.15.186:3300/0 conn(0x7f74a40cd7c0 0x7f74a40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f748c0035b0 con 0x7f74a415c9b0 2023-11-28T11:09:10.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3780623629 0 0) 0x7f748c0035b0 con 0x7f74a415c9b0 2023-11-28T11:09:10.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 >> v2:172.21.15.186:3300/0 conn(0x7f74a40cd7c0 msgr2=0x7f74a40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 --2- 172.21.15.156:0/1361823461 >> v2:172.21.15.186:3300/0 conn(0x7f74a40cd7c0 0x7f74a40cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 >> v1:172.21.15.186:6789/0 conn(0x7f74a4161340 legacy=0x7f74a40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a40d24f0 con 0x7f74a415c9b0 2023-11-28T11:09:10.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f7494001670 con 0x7f74a415c9b0 2023-11-28T11:09:10.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 >> v1:172.21.15.156:6789/0 conn(0x7f74a415c9b0 legacy=0x7f74a40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 --2- 172.21.15.156:0/1361823461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 0x7f748c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 --2- 172.21.15.156:0/1361823461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f748c004850 0x7f748c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 --2- 172.21.15.156:0/1361823461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f748c005200 0x7f748c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a40b5800 con 0x7f748c003a00 2023-11-28T11:09:10.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c00c110 con 0x7f748c005200 2023-11-28T11:09:10.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.893+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c00c2d0 con 0x7f748c004850 2023-11-28T11:09:10.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a37fe700 1 --2- 172.21.15.156:0/1361823461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f748c005200 0x7f748c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a27fc700 1 --2- 172.21.15.156:0/1361823461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f748c004850 0x7f748c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a2ffd700 1 --2- 172.21.15.156:0/1361823461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 0x7f748c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a2ffd700 1 -- 172.21.15.156:0/1361823461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f748c004850 msgr2=0x7f748c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a2ffd700 1 --2- 172.21.15.156:0/1361823461 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f748c004850 0x7f748c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a2ffd700 1 -- 172.21.15.156:0/1361823461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f748c005200 msgr2=0x7f748c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a2ffd700 1 --2- 172.21.15.156:0/1361823461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f748c005200 0x7f748c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a37fe700 1 -- 172.21.15.156:0/1361823461 reap_dead start 2023-11-28T11:09:10.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a2ffd700 1 -- 172.21.15.156:0/1361823461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0035b0 con 0x7f748c003a00 2023-11-28T11:09:10.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a2ffd700 1 --2- 172.21.15.156:0/1361823461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 0x7f748c003e10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f749400c240 tx=0x7f748c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ac152ce69793b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494007c10 con 0x7f748c003a00 2023-11-28T11:09:10.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749400cd60 con 0x7f748c003a00 2023-11-28T11:09:10.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.894+0000 7f74a1ffb700 1 -- 172.21.15.156:0/1361823461 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494005ba0 con 0x7f748c003a00 2023-11-28T11:09:10.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.895+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1361823461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 msgr2=0x7f748c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.895+0000 7f74a9a3e700 1 --2- 172.21.15.156:0/1361823461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 0x7f748c003e10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f749400c240 tx=0x7f748c005880 comp rx=0 tx=0).stop 2023-11-28T11:09:10.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.895+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1361823461 shutdown_connections 2023-11-28T11:09:10.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.895+0000 7f74a9a3e700 1 --2- 172.21.15.156:0/1361823461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 0x7f748c003e10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.895+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1361823461 >> 172.21.15.156:0/1361823461 conn(0x7f74a4072600 msgr2=0x7f74a40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.895+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1361823461 shutdown_connections 2023-11-28T11:09:10.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.895+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1361823461 wait complete. 2023-11-28T11:09:10.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.896+0000 7f74a9a3e700 1 Processor -- start 2023-11-28T11:09:10.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.896+0000 7f74a9a3e700 1 -- start start 2023-11-28T11:09:10.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a9a3e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 0x7f74a4157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a9a3e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74a40c9880 0x7f74a41e7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a9a3e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74a41e7540 0x7f74a415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a9a3e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a4157d90 con 0x7f748c003a00 2023-11-28T11:09:10.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a9a3e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a40c8ac0 con 0x7f74a40c9880 2023-11-28T11:09:10.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a9a3e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a40c8da0 con 0x7f74a41e7540 2023-11-28T11:09:10.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 0x7f74a4157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 0x7f74a4157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33560/0 (socket says 172.21.15.156:33560) 2023-11-28T11:09:10.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a2ffd700 1 -- 172.21.15.156:0/1476567739 learned_addr learned my addr 172.21.15.156:0/1476567739 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:10.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a27fc700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74a40c9880 0x7f74a41e7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a37fe700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74a41e7540 0x7f74a415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a2ffd700 1 -- 172.21.15.156:0/1476567739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74a41e7540 msgr2=0x7f74a415e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a2ffd700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74a41e7540 0x7f74a415e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a2ffd700 1 -- 172.21.15.156:0/1476567739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74a40c9880 msgr2=0x7f74a41e7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a2ffd700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74a40c9880 0x7f74a41e7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a2ffd700 1 -- 172.21.15.156:0/1476567739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a40d1300 con 0x7f748c003a00 2023-11-28T11:09:10.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.897+0000 7f74a2ffd700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 0x7f74a4157850 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c0e0 tx=0x7f7494001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.898+0000 7f748bfff700 1 -- 172.21.15.156:0/1476567739 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749400ced0 con 0x7f748c003a00 2023-11-28T11:09:10.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.898+0000 7f748bfff700 1 -- 172.21.15.156:0/1476567739 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7494001640 con 0x7f748c003a00 2023-11-28T11:09:10.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.898+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1476567739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a40b5520 con 0x7f748c003a00 2023-11-28T11:09:10.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.898+0000 7f748bfff700 1 -- 172.21.15.156:0/1476567739 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74940072f0 con 0x7f748c003a00 2023-11-28T11:09:10.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.898+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1476567739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a40d15e0 con 0x7f748c003a00 2023-11-28T11:09:10.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.899+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1476567739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a40745c0 con 0x7f748c003a00 2023-11-28T11:09:10.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.899+0000 7f748bfff700 1 -- 172.21.15.156:0/1476567739 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7494024030 con 0x7f748c003a00 2023-11-28T11:09:10.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.900+0000 7f748bfff700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74900682f0 0x7f749006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.900+0000 7f74a27fc700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74900682f0 0x7f749006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.900+0000 7f748bfff700 1 -- 172.21.15.156:0/1476567739 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7494087150 con 0x7f748c003a00 2023-11-28T11:09:10.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.900+0000 7f74a27fc700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74900682f0 0x7f749006a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f74a40c9c70 tx=0x7f7498009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:10.904+0000 7f748bfff700 1 -- 172.21.15.156:0/1476567739 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7494053d00 con 0x7f748c003a00 2023-11-28T11:09:11.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.162+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1476567739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f74a40d08a0 con 0x7f748c003a00 2023-11-28T11:09:11.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.166+0000 7f748bfff700 1 -- 172.21.15.156:0/1476567739 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7494007d70 con 0x7f748c003a00 2023-11-28T11:09:11.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.168+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1476567739 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74900682f0 msgr2=0x7f749006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.168+0000 7f74a9a3e700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74900682f0 0x7f749006a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f74a40c9c70 tx=0x7f7498009040 comp rx=0 tx=0).stop 2023-11-28T11:09:11.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.168+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1476567739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 msgr2=0x7f74a4157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.168+0000 7f74a9a3e700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 0x7f74a4157850 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c0e0 tx=0x7f7494001a00 comp rx=0 tx=0).stop 2023-11-28T11:09:11.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.168+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1476567739 shutdown_connections 2023-11-28T11:09:11.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.168+0000 7f74a9a3e700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74900682f0 0x7f749006a7a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.168+0000 7f74a9a3e700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74a41e7540 0x7f74a415e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.168+0000 7f74a9a3e700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748c003a00 0x7f74a4157850 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.168+0000 7f74a9a3e700 1 --2- 172.21.15.156:0/1476567739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74a40c9880 0x7f74a41e7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.168+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1476567739 >> 172.21.15.156:0/1476567739 conn(0x7f74a4072600 msgr2=0x7f74a406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.168+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1476567739 shutdown_connections 2023-11-28T11:09:11.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.168+0000 7f74a9a3e700 1 -- 172.21.15.156:0/1476567739 wait complete. 2023-11-28T11:09:11.182 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-28T11:09:11.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.303+0000 7f46c8864700 1 Processor -- start 2023-11-28T11:09:11.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.303+0000 7f46c8864700 1 -- start start 2023-11-28T11:09:11.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c8864700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c40cd8c0 0x7f46c4140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c8864700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46c40cdc90 0x7f46c40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c8864700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c413ff00 con 0x7f46c40b2e70 2023-11-28T11:09:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c8864700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c41404c0 con 0x7f46c40cdc90 2023-11-28T11:09:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c8864700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c415aa80 con 0x7f46c40cd8c0 2023-11-28T11:09:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c1d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46c40cdc90 0x7f46c40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c1d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46c40cdc90 0x7f46c40b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33570/0 (socket says 172.21.15.156:33570) 2023-11-28T11:09:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c1d9b700 1 -- 172.21.15.156:0/3395461926 learned_addr learned my addr 172.21.15.156:0/3395461926 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c259c700 1 --2- 172.21.15.156:0/3395461926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c40cd8c0 0x7f46c4140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c1d9b700 1 -- 172.21.15.156:0/3395461926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c40cd8c0 msgr2=0x7f46c4140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c1d9b700 1 --2- 172.21.15.156:0/3395461926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c40cd8c0 0x7f46c4140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c1d9b700 1 -- 172.21.15.156:0/3395461926 >> v1:172.21.15.186:6789/0 conn(0x7f46c40b2e70 legacy=0x7f46c406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c1d9b700 1 -- 172.21.15.156:0/3395461926 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c40b3310 con 0x7f46c40cdc90 2023-11-28T11:09:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c1d9b700 1 --2- 172.21.15.156:0/3395461926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c40cdc90 0x7f46c40b2930 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f46b80098f0 tx=0x7f46b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775d9583e49186d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c159a700 1 -- 172.21.15.156:0/3395461926 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b800b8a0 con 0x7f46c40cdc90 2023-11-28T11:09:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c159a700 1 -- 172.21.15.156:0/3395461926 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b800ba00 con 0x7f46c40cdc90 2023-11-28T11:09:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.304+0000 7f46c159a700 1 -- 172.21.15.156:0/3395461926 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b800bd90 con 0x7f46c40cdc90 2023-11-28T11:09:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.305+0000 7f46c8864700 1 -- 172.21.15.156:0/3395461926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c40cdc90 msgr2=0x7f46c40b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.305+0000 7f46c8864700 1 --2- 172.21.15.156:0/3395461926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c40cdc90 0x7f46c40b2930 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f46b80098f0 tx=0x7f46b8009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.305+0000 7f46c8864700 1 -- 172.21.15.156:0/3395461926 shutdown_connections 2023-11-28T11:09:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.305+0000 7f46c8864700 1 --2- 172.21.15.156:0/3395461926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c40cd8c0 0x7f46c4140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.305+0000 7f46c8864700 1 --2- 172.21.15.156:0/3395461926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c40cdc90 0x7f46c40b2930 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.305+0000 7f46c8864700 1 -- 172.21.15.156:0/3395461926 >> 172.21.15.156:0/3395461926 conn(0x7f46c406aed0 msgr2=0x7f46c406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.305+0000 7f46c8864700 1 -- 172.21.15.156:0/3395461926 shutdown_connections 2023-11-28T11:09:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.305+0000 7f46c8864700 1 -- 172.21.15.156:0/3395461926 wait complete. 2023-11-28T11:09:11.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.306+0000 7f46c8864700 1 Processor -- start 2023-11-28T11:09:11.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.306+0000 7f46c8864700 1 -- start start 2023-11-28T11:09:11.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c8864700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c40b2e70 0x7f46c41f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c8864700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c40cd8c0 0x7f46c41431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c8864700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c40cdc90 0x7f46c4143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c8864700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c4159940 con 0x7f46c40b2e70 2023-11-28T11:09:11.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c8864700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c40c45c0 con 0x7f46c40cdc90 2023-11-28T11:09:11.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c8864700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c4159380 con 0x7f46c40cd8c0 2023-11-28T11:09:11.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c40b2e70 0x7f46c41f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c40b2e70 0x7f46c41f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33576/0 (socket says 172.21.15.156:33576) 2023-11-28T11:09:11.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c259c700 1 -- 172.21.15.156:0/3165177547 learned_addr learned my addr 172.21.15.156:0/3165177547 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:11.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c1d9b700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c40cd8c0 0x7f46c41431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c2d9d700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c40cdc90 0x7f46c4143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c259c700 1 -- 172.21.15.156:0/3165177547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c40cd8c0 msgr2=0x7f46c41431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.307+0000 7f46c259c700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c40cd8c0 0x7f46c41431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.308+0000 7f46c259c700 1 -- 172.21.15.156:0/3165177547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c40cdc90 msgr2=0x7f46c4143700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.308+0000 7f46c259c700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c40cdc90 0x7f46c4143700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.308+0000 7f46c259c700 1 -- 172.21.15.156:0/3165177547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8009550 con 0x7f46c40b2e70 2023-11-28T11:09:11.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.308+0000 7f46c259c700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c40b2e70 0x7f46c41f3430 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f46b400e730 tx=0x7f46b400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.308+0000 7f46b2ffd700 1 -- 172.21.15.156:0/3165177547 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b400c070 con 0x7f46c40b2e70 2023-11-28T11:09:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.308+0000 7f46b2ffd700 1 -- 172.21.15.156:0/3165177547 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b4013610 con 0x7f46c40b2e70 2023-11-28T11:09:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.308+0000 7f46c8864700 1 -- 172.21.15.156:0/3165177547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c4140150 con 0x7f46c40b2e70 2023-11-28T11:09:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.308+0000 7f46b2ffd700 1 -- 172.21.15.156:0/3165177547 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b40139c0 con 0x7f46c40b2e70 2023-11-28T11:09:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.308+0000 7f46c8864700 1 -- 172.21.15.156:0/3165177547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c40b35f0 con 0x7f46c40b2e70 2023-11-28T11:09:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.309+0000 7f46c8864700 1 -- 172.21.15.156:0/3165177547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c40ce100 con 0x7f46c40b2e70 2023-11-28T11:09:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.309+0000 7f46b2ffd700 1 -- 172.21.15.156:0/3165177547 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f46b4013b20 con 0x7f46c40b2e70 2023-11-28T11:09:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.310+0000 7f46b2ffd700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f46a40682f0 0x7f46a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.310+0000 7f46c1d9b700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f46a40682f0 0x7f46a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.310+0000 7f46b2ffd700 1 -- 172.21.15.156:0/3165177547 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f46b4085750 con 0x7f46c40b2e70 2023-11-28T11:09:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.310+0000 7f46c1d9b700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f46a40682f0 0x7f46a406a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46b80098f0 tx=0x7f46b8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.314+0000 7f46b2ffd700 1 -- 172.21.15.156:0/3165177547 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f46b40522c0 con 0x7f46c40b2e70 2023-11-28T11:09:11.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.573+0000 7f46c8864700 1 -- 172.21.15.156:0/3165177547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f46c4068690 con 0x7f46c40b2e70 2023-11-28T11:09:11.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.577+0000 7f46b2ffd700 1 -- 172.21.15.156:0/3165177547 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f46b4013db0 con 0x7f46c40b2e70 2023-11-28T11:09:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.579+0000 7f46c8864700 1 -- 172.21.15.156:0/3165177547 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f46a40682f0 msgr2=0x7f46a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.579+0000 7f46c8864700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f46a40682f0 0x7f46a406a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46b80098f0 tx=0x7f46b8008030 comp rx=0 tx=0).stop 2023-11-28T11:09:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.579+0000 7f46c8864700 1 -- 172.21.15.156:0/3165177547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c40b2e70 msgr2=0x7f46c41f3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.579+0000 7f46c8864700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c40b2e70 0x7f46c41f3430 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f46b400e730 tx=0x7f46b400ea40 comp rx=0 tx=0).stop 2023-11-28T11:09:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.579+0000 7f46c8864700 1 -- 172.21.15.156:0/3165177547 shutdown_connections 2023-11-28T11:09:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.579+0000 7f46c8864700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c40cdc90 0x7f46c4143700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.579+0000 7f46c8864700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c40b2e70 0x7f46c41f3430 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.579+0000 7f46c8864700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f46a40682f0 0x7f46a406a7a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.579+0000 7f46c8864700 1 --2- 172.21.15.156:0/3165177547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c40cd8c0 0x7f46c41431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.579+0000 7f46c8864700 1 -- 172.21.15.156:0/3165177547 >> 172.21.15.156:0/3165177547 conn(0x7f46c406aed0 msgr2=0x7f46c40d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.579+0000 7f46c8864700 1 -- 172.21.15.156:0/3165177547 shutdown_connections 2023-11-28T11:09:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.579+0000 7f46c8864700 1 -- 172.21.15.156:0/3165177547 wait complete. 2023-11-28T11:09:11.592 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-28T11:09:11.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.714+0000 7f143a89d700 1 Processor -- start 2023-11-28T11:09:11.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.714+0000 7f143a89d700 1 -- start start 2023-11-28T11:09:11.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f143a89d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f14340cb0e0 0x7f1434065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f143a89d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14340cec70 0x7f143406d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f143a89d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1434066580 con 0x7f143406ec90 2023-11-28T11:09:11.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f143a89d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14340cded0 con 0x7f14340cb0e0 2023-11-28T11:09:11.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f143a89d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14340ce1b0 con 0x7f14340cec70 2023-11-28T11:09:11.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f14337fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f14340cb0e0 0x7f1434065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f1433fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f143406ec90 0x7f14340ca9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48460/0 (socket says 172.21.15.156:48460) 2023-11-28T11:09:11.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f1433fff700 1 -- 172.21.15.156:0/10438064 learned_addr learned my addr 172.21.15.156:0/10438064 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:11.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f14337fe700 1 -- 172.21.15.156:0/10438064 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14340cec70 msgr2=0x7f143406d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:11.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f14337fe700 1 --2- 172.21.15.156:0/10438064 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14340cec70 0x7f143406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f14337fe700 1 -- 172.21.15.156:0/10438064 >> v1:172.21.15.156:6789/0 conn(0x7f143406ec90 legacy=0x7f14340ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f14337fe700 1 -- 172.21.15.156:0/10438064 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340ce490 con 0x7f14340cb0e0 2023-11-28T11:09:11.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f1432ffd700 1 -- 172.21.15.156:0/10438064 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1873981609 0 0) 0x7f1434066580 con 0x7f143406ec90 2023-11-28T11:09:11.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f14337fe700 1 --2- 172.21.15.156:0/10438064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14340cb0e0 0x7f1434065ef0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f142800bae0 tx=0x7f142800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc50c07961d931b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f1432ffd700 1 -- 172.21.15.156:0/10438064 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142800dd30 con 0x7f14340cb0e0 2023-11-28T11:09:11.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f1432ffd700 1 -- 172.21.15.156:0/10438064 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142800de90 con 0x7f14340cb0e0 2023-11-28T11:09:11.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.715+0000 7f1432ffd700 1 -- 172.21.15.156:0/10438064 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428010690 con 0x7f14340cb0e0 2023-11-28T11:09:11.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.716+0000 7f143a89d700 1 -- 172.21.15.156:0/10438064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14340cb0e0 msgr2=0x7f1434065ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.716+0000 7f143a89d700 1 --2- 172.21.15.156:0/10438064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14340cb0e0 0x7f1434065ef0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f142800bae0 tx=0x7f142800bdf0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.716+0000 7f143a89d700 1 -- 172.21.15.156:0/10438064 shutdown_connections 2023-11-28T11:09:11.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.716+0000 7f143a89d700 1 --2- 172.21.15.156:0/10438064 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14340cec70 0x7f143406d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.716+0000 7f143a89d700 1 --2- 172.21.15.156:0/10438064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14340cb0e0 0x7f1434065ef0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.716+0000 7f143a89d700 1 -- 172.21.15.156:0/10438064 >> 172.21.15.156:0/10438064 conn(0x7f14340b12b0 msgr2=0x7f14340b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.716+0000 7f143a89d700 1 -- 172.21.15.156:0/10438064 shutdown_connections 2023-11-28T11:09:11.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.716+0000 7f143a89d700 1 -- 172.21.15.156:0/10438064 wait complete. 2023-11-28T11:09:11.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.717+0000 7f143a89d700 1 Processor -- start 2023-11-28T11:09:11.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.717+0000 7f143a89d700 1 -- start start 2023-11-28T11:09:11.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f143a89d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143406ec90 0x7f1434151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f143a89d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14340cb0e0 0x7f14340ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f143a89d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14340cec70 0x7f143415b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f143a89d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340c8e80 con 0x7f143406ec90 2023-11-28T11:09:11.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f143a89d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340c7de0 con 0x7f14340cb0e0 2023-11-28T11:09:11.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f143a89d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14340c80c0 con 0x7f14340cec70 2023-11-28T11:09:11.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f1433fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143406ec90 0x7f1434151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f1433fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143406ec90 0x7f1434151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33588/0 (socket says 172.21.15.156:33588) 2023-11-28T11:09:11.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f1433fff700 1 -- 172.21.15.156:0/3988682935 learned_addr learned my addr 172.21.15.156:0/3988682935 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:11.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f1438e3a700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14340cec70 0x7f143415b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f14337fe700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14340cb0e0 0x7f14340ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f1433fff700 1 -- 172.21.15.156:0/3988682935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14340cec70 msgr2=0x7f143415b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f1433fff700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14340cec70 0x7f143415b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f1433fff700 1 -- 172.21.15.156:0/3988682935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14340cb0e0 msgr2=0x7f14340ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f1433fff700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14340cb0e0 0x7f14340ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f1433fff700 1 -- 172.21.15.156:0/3988682935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142800b740 con 0x7f143406ec90 2023-11-28T11:09:11.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.718+0000 7f1438e3a700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14340cec70 0x7f143415b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:11.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.719+0000 7f1433fff700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143406ec90 0x7f1434151d80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f142400c550 tx=0x7f142400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.719+0000 7f1430ff9700 1 -- 172.21.15.156:0/3988682935 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424004c80 con 0x7f143406ec90 2023-11-28T11:09:11.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.719+0000 7f1430ff9700 1 -- 172.21.15.156:0/3988682935 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1424004e00 con 0x7f143406ec90 2023-11-28T11:09:11.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.719+0000 7f143a89d700 1 -- 172.21.15.156:0/3988682935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14340cb6c0 con 0x7f143406ec90 2023-11-28T11:09:11.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.719+0000 7f1430ff9700 1 -- 172.21.15.156:0/3988682935 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424007310 con 0x7f143406ec90 2023-11-28T11:09:11.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.719+0000 7f143a89d700 1 -- 172.21.15.156:0/3988682935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14340ce770 con 0x7f143406ec90 2023-11-28T11:09:11.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.720+0000 7f143a89d700 1 -- 172.21.15.156:0/3988682935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143406f060 con 0x7f143406ec90 2023-11-28T11:09:11.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.720+0000 7f1430ff9700 1 -- 172.21.15.156:0/3988682935 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f142401a020 con 0x7f143406ec90 2023-11-28T11:09:11.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.720+0000 7f1430ff9700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14200682f0 0x7f142006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.720+0000 7f1430ff9700 1 -- 172.21.15.156:0/3988682935 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f14240854f0 con 0x7f143406ec90 2023-11-28T11:09:11.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.720+0000 7f14337fe700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14200682f0 0x7f142006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.721+0000 7f14337fe700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14200682f0 0x7f142006a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1434072180 tx=0x7f1428013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.725+0000 7f1430ff9700 1 -- 172.21.15.156:0/3988682935 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1424052150 con 0x7f143406ec90 2023-11-28T11:09:11.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.980+0000 7f143a89d700 1 -- 172.21.15.156:0/3988682935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f14340cf110 con 0x7f143406ec90 2023-11-28T11:09:11.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.987+0000 7f1430ff9700 1 -- 172.21.15.156:0/3988682935 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1424055770 con 0x7f143406ec90 2023-11-28T11:09:11.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.989+0000 7f143a89d700 1 -- 172.21.15.156:0/3988682935 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14200682f0 msgr2=0x7f142006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.989+0000 7f143a89d700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14200682f0 0x7f142006a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1434072180 tx=0x7f1428013000 comp rx=0 tx=0).stop 2023-11-28T11:09:11.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.989+0000 7f143a89d700 1 -- 172.21.15.156:0/3988682935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143406ec90 msgr2=0x7f1434151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.989+0000 7f143a89d700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143406ec90 0x7f1434151d80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f142400c550 tx=0x7f142400c910 comp rx=0 tx=0).stop 2023-11-28T11:09:11.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.989+0000 7f143a89d700 1 -- 172.21.15.156:0/3988682935 shutdown_connections 2023-11-28T11:09:11.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.989+0000 7f143a89d700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14200682f0 0x7f142006a7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.989+0000 7f143a89d700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14340cec70 0x7f143415b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.989+0000 7f143a89d700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143406ec90 0x7f1434151d80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.989+0000 7f143a89d700 1 --2- 172.21.15.156:0/3988682935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14340cb0e0 0x7f14340ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.989+0000 7f143a89d700 1 -- 172.21.15.156:0/3988682935 >> 172.21.15.156:0/3988682935 conn(0x7f14340b12b0 msgr2=0x7f143406b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.989+0000 7f143a89d700 1 -- 172.21.15.156:0/3988682935 shutdown_connections 2023-11-28T11:09:11.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:11.989+0000 7f143a89d700 1 -- 172.21.15.156:0/3988682935 wait complete. 2023-11-28T11:09:12.003 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:12.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.123+0000 7fb1f6cfe700 1 Processor -- start 2023-11-28T11:09:12.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.123+0000 7fb1f6cfe700 1 -- start start 2023-11-28T11:09:12.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.123+0000 7fb1f6cfe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb1f0140950 0x7fb1f00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.123+0000 7fb1f6cfe700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f013ff00 con 0x7fb1f00cd8c0 2023-11-28T11:09:12.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.123+0000 7fb1f6cfe700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f01404c0 con 0x7fb1f00b2e70 2023-11-28T11:09:12.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.123+0000 7fb1f6cfe700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f0140dc0 con 0x7fb1f0140950 2023-11-28T11:09:12.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1f4a9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb1f00cd8c0 0x7fb1f00cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48474/0 (socket says 172.21.15.156:48474) 2023-11-28T11:09:12.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1f4a9a700 1 -- 172.21.15.156:0/3397214559 learned_addr learned my addr 172.21.15.156:0/3397214559 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:12.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1effff700 1 --2- 172.21.15.156:0/3397214559 >> v2:172.21.15.186:3300/0 conn(0x7fb1f0140950 0x7fb1f00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499388602 0 0) 0x7fb1f013ff00 con 0x7fb1f00cd8c0 2023-11-28T11:09:12.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1e00035b0 con 0x7fb1f00cd8c0 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1778056776 0 0) 0x7fb1e00035b0 con 0x7fb1f00cd8c0 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 >> v2:172.21.15.186:3300/0 conn(0x7fb1f0140950 msgr2=0x7fb1f00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 --2- 172.21.15.156:0/3397214559 >> v2:172.21.15.186:3300/0 conn(0x7fb1f0140950 0x7fb1f00b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 >> v1:172.21.15.186:6789/0 conn(0x7fb1f00b2e70 legacy=0x7fb1f006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f015aa80 con 0x7fb1f00cd8c0 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fb1e40017f0 con 0x7fb1f00cd8c0 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 >> v1:172.21.15.156:6789/0 conn(0x7fb1f00cd8c0 legacy=0x7fb1f00cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 --2- 172.21.15.156:0/3397214559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e0003a00 0x7fb1e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 --2- 172.21.15.156:0/3397214559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1e0004850 0x7fb1e0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 --2- 172.21.15.156:0/3397214559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1e0005200 0x7fb1e000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f0140dc0 con 0x7fb1e0005200 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e000c110 con 0x7fb1e0003a00 2023-11-28T11:09:12.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e000c2d0 con 0x7fb1e0004850 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1effff700 1 --2- 172.21.15.156:0/3397214559 >> v2:172.21.15.186:3300/0 conn(0x7fb1f0140950 0x7fb1f00b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1f529b700 1 --2- 172.21.15.156:0/3397214559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1e0005200 0x7fb1e000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.124+0000 7fb1f4a9a700 1 --2- 172.21.15.156:0/3397214559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e0003a00 0x7fb1e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1effff700 1 --2- 172.21.15.156:0/3397214559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1e0004850 0x7fb1e0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1f529b700 1 -- 172.21.15.156:0/3397214559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1e0004850 msgr2=0x7fb1e0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1f529b700 1 --2- 172.21.15.156:0/3397214559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1e0004850 0x7fb1e0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1f529b700 1 -- 172.21.15.156:0/3397214559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e0003a00 msgr2=0x7fb1e0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1f529b700 1 --2- 172.21.15.156:0/3397214559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e0003a00 0x7fb1e0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1f529b700 1 -- 172.21.15.156:0/3397214559 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e00035b0 con 0x7fb1e0005200 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1effff700 1 --2- 172.21.15.156:0/3397214559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1e0004850 0x7fb1e0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1f529b700 1 -- 172.21.15.156:0/3397214559 reap_dead start 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1f529b700 1 --2- 172.21.15.156:0/3397214559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1e0005200 0x7fb1e000ba10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800c190 tx=0x7fb1e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6566827744664ba7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8001db0 con 0x7fb1e0005200 2023-11-28T11:09:12.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e800ccf0 con 0x7fb1e0005200 2023-11-28T11:09:12.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1ef7fe700 1 -- 172.21.15.156:0/3397214559 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8003c40 con 0x7fb1e0005200 2023-11-28T11:09:12.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/3397214559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1e0005200 msgr2=0x7fb1e000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.125+0000 7fb1f6cfe700 1 --2- 172.21.15.156:0/3397214559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1e0005200 0x7fb1e000ba10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800c190 tx=0x7fb1e0005880 comp rx=0 tx=0).stop 2023-11-28T11:09:12.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.126+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/3397214559 shutdown_connections 2023-11-28T11:09:12.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.126+0000 7fb1f6cfe700 1 --2- 172.21.15.156:0/3397214559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1e0005200 0x7fb1e000ba10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.126+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/3397214559 >> 172.21.15.156:0/3397214559 conn(0x7fb1f006aed0 msgr2=0x7fb1f006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.126+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/3397214559 shutdown_connections 2023-11-28T11:09:12.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.126+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/3397214559 wait complete. 2023-11-28T11:09:12.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.127+0000 7fb1f6cfe700 1 Processor -- start 2023-11-28T11:09:12.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.127+0000 7fb1f6cfe700 1 -- start start 2023-11-28T11:09:12.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f6cfe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e0005200 0x7fb1f00c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f6cfe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1f00b6900 0x7fb1f013e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f6cfe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1f013ebe0 0x7fb1f013f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f6cfe700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f013f050 con 0x7fb1f013ebe0 2023-11-28T11:09:12.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f6cfe700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f00b58e0 con 0x7fb1e0005200 2023-11-28T11:09:12.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f6cfe700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f00b5b60 con 0x7fb1f00b6900 2023-11-28T11:09:12.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f529b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1f013ebe0 0x7fb1f013f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f529b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1f013ebe0 0x7fb1f013f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33612/0 (socket says 172.21.15.156:33612) 2023-11-28T11:09:12.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f529b700 1 -- 172.21.15.156:0/2329532043 learned_addr learned my addr 172.21.15.156:0/2329532043 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:12.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1effff700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1f00b6900 0x7fb1f013e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f4a9a700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e0005200 0x7fb1f00c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f529b700 1 -- 172.21.15.156:0/2329532043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1f00b6900 msgr2=0x7fb1f013e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f529b700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1f00b6900 0x7fb1f013e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f529b700 1 -- 172.21.15.156:0/2329532043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e0005200 msgr2=0x7fb1f00c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f529b700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e0005200 0x7fb1f00c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f529b700 1 -- 172.21.15.156:0/2329532043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f00b35f0 con 0x7fb1f013ebe0 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1effff700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1f00b6900 0x7fb1f013e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f4a9a700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e0005200 0x7fb1f00c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f529b700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1f013ebe0 0x7fb1f013f360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00cb490 tx=0x7fb1e80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1ed7fa700 1 -- 172.21.15.156:0/2329532043 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8005070 con 0x7fb1f013ebe0 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1ed7fa700 1 -- 172.21.15.156:0/2329532043 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e80051d0 con 0x7fb1f013ebe0 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1ed7fa700 1 -- 172.21.15.156:0/2329532043 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8005560 con 0x7fb1f013ebe0 2023-11-28T11:09:12.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/2329532043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f00b3310 con 0x7fb1f013ebe0 2023-11-28T11:09:12.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.128+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/2329532043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f0140150 con 0x7fb1f013ebe0 2023-11-28T11:09:12.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.129+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/2329532043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f0068690 con 0x7fb1f013ebe0 2023-11-28T11:09:12.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.130+0000 7fb1ed7fa700 1 -- 172.21.15.156:0/2329532043 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb1e8010050 con 0x7fb1f013ebe0 2023-11-28T11:09:12.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.130+0000 7fb1ed7fa700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb1d40682f0 0x7fb1d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.130+0000 7fb1ed7fa700 1 -- 172.21.15.156:0/2329532043 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb1e8086b80 con 0x7fb1f013ebe0 2023-11-28T11:09:12.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.130+0000 7fb1f4a9a700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb1d40682f0 0x7fb1d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.131+0000 7fb1f4a9a700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb1d40682f0 0x7fb1d406a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4000ef0 tx=0x7fb1e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.134+0000 7fb1ed7fa700 1 -- 172.21.15.156:0/2329532043 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb1e80536d0 con 0x7fb1f013ebe0 2023-11-28T11:09:12.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.316+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/2329532043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1f0068690 con 0x7fb1f013ebe0 2023-11-28T11:09:12.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.317+0000 7fb1ed7fa700 1 -- 172.21.15.156:0/2329532043 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+14315 (secure 0 0 0) 0x7fb1e8016020 con 0x7fb1f013ebe0 2023-11-28T11:09:12.318 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:12.319 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":33,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:09.120445+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:12.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.319+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/2329532043 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb1d40682f0 msgr2=0x7fb1d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.319+0000 7fb1f6cfe700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb1d40682f0 0x7fb1d406a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4000ef0 tx=0x7fb1e4009040 comp rx=0 tx=0).stop 2023-11-28T11:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.319+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/2329532043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1f013ebe0 msgr2=0x7fb1f013f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.319+0000 7fb1f6cfe700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1f013ebe0 0x7fb1f013f360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00cb490 tx=0x7fb1e80014a0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.319+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/2329532043 shutdown_connections 2023-11-28T11:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.319+0000 7fb1f6cfe700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1f013ebe0 0x7fb1f013f360 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.319+0000 7fb1f6cfe700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e0005200 0x7fb1f00c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.319+0000 7fb1f6cfe700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb1d40682f0 0x7fb1d406a7a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.319+0000 7fb1f6cfe700 1 --2- 172.21.15.156:0/2329532043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1f00b6900 0x7fb1f013e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.319+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/2329532043 >> 172.21.15.156:0/2329532043 conn(0x7fb1f006aed0 msgr2=0x7fb1f00b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.319+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/2329532043 shutdown_connections 2023-11-28T11:09:12.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.319+0000 7fb1f6cfe700 1 -- 172.21.15.156:0/2329532043 wait complete. 2023-11-28T11:09:12.333 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:09:12.333 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:12.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.455+0000 7f380c489700 1 Processor -- start 2023-11-28T11:09:12.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.455+0000 7f380c489700 1 -- start start 2023-11-28T11:09:12.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.456+0000 7f380c489700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3804071040 0x7f3804071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.456+0000 7f380c489700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f380406dee0 0x7f380406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.456+0000 7f380c489700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3804071950 con 0x7f380406cdc0 2023-11-28T11:09:12.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.456+0000 7f380c489700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3804161550 con 0x7f380406dee0 2023-11-28T11:09:12.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.456+0000 7f380c489700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380415d060 con 0x7f3804071040 2023-11-28T11:09:12.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.456+0000 7f3809a24700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f380406dee0 0x7f380406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.456+0000 7f3809a24700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f380406dee0 0x7f380406e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33616/0 (socket says 172.21.15.156:33616) 2023-11-28T11:09:12.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.456+0000 7f3809a24700 1 -- 172.21.15.156:0/1366020770 learned_addr learned my addr 172.21.15.156:0/1366020770 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:12.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.456+0000 7f380a225700 1 --2- 172.21.15.156:0/1366020770 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3804071040 0x7f3804071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f3809a24700 1 -- 172.21.15.156:0/1366020770 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3804071040 msgr2=0x7f3804071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f3809a24700 1 --2- 172.21.15.156:0/1366020770 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3804071040 0x7f3804071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f3809a24700 1 -- 172.21.15.156:0/1366020770 >> v1:172.21.15.186:6789/0 conn(0x7f380406cdc0 legacy=0x7f380406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f3809a24700 1 -- 172.21.15.156:0/1366020770 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380415d340 con 0x7f380406dee0 2023-11-28T11:09:12.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f380a225700 1 --2- 172.21.15.156:0/1366020770 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3804071040 0x7f3804071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:12.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f3809a24700 1 --2- 172.21.15.156:0/1366020770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f380406dee0 0x7f380406e350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f38000098f0 tx=0x7f3800009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8489c960d1e4f82 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f3809223700 1 -- 172.21.15.156:0/1366020770 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800004230 con 0x7f380406dee0 2023-11-28T11:09:12.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f3809223700 1 -- 172.21.15.156:0/1366020770 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3800004390 con 0x7f380406dee0 2023-11-28T11:09:12.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f3809223700 1 -- 172.21.15.156:0/1366020770 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38000036a0 con 0x7f380406dee0 2023-11-28T11:09:12.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f380c489700 1 -- 172.21.15.156:0/1366020770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f380406dee0 msgr2=0x7f380406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f380c489700 1 --2- 172.21.15.156:0/1366020770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f380406dee0 0x7f380406e350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f38000098f0 tx=0x7f3800009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:12.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f380c489700 1 -- 172.21.15.156:0/1366020770 shutdown_connections 2023-11-28T11:09:12.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f380c489700 1 --2- 172.21.15.156:0/1366020770 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3804071040 0x7f3804071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f380c489700 1 --2- 172.21.15.156:0/1366020770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f380406dee0 0x7f380406e350 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f380c489700 1 -- 172.21.15.156:0/1366020770 >> 172.21.15.156:0/1366020770 conn(0x7f38040d0000 msgr2=0x7f38040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.457+0000 7f380c489700 1 -- 172.21.15.156:0/1366020770 shutdown_connections 2023-11-28T11:09:12.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.458+0000 7f380c489700 1 -- 172.21.15.156:0/1366020770 wait complete. 2023-11-28T11:09:12.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.459+0000 7f380c489700 1 Processor -- start 2023-11-28T11:09:12.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.459+0000 7f380c489700 1 -- start start 2023-11-28T11:09:12.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.459+0000 7f380c489700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f380406cdc0 0x7f3804152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.459+0000 7f380c489700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f380406dee0 0x7f38040c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.459+0000 7f380c489700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3804071040 0x7f38040c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.459+0000 7f380c489700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38040b2e30 con 0x7f380406dee0 2023-11-28T11:09:12.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.459+0000 7f380c489700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38040b4120 con 0x7f380406cdc0 2023-11-28T11:09:12.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.459+0000 7f380c489700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38040b3080 con 0x7f3804071040 2023-11-28T11:09:12.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.459+0000 7f3809a24700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f380406dee0 0x7f38040c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f3809a24700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f380406dee0 0x7f38040c57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33622/0 (socket says 172.21.15.156:33622) 2023-11-28T11:09:12.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f3809a24700 1 -- 172.21.15.156:0/2760272529 learned_addr learned my addr 172.21.15.156:0/2760272529 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:12.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f380a225700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f380406cdc0 0x7f3804152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f3809a24700 1 -- 172.21.15.156:0/2760272529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3804071040 msgr2=0x7f38040c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f380aa26700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3804071040 0x7f38040c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f3809a24700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3804071040 0x7f38040c5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f3809a24700 1 -- 172.21.15.156:0/2760272529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f380406cdc0 msgr2=0x7f3804152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f3809a24700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f380406cdc0 0x7f3804152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f3809a24700 1 -- 172.21.15.156:0/2760272529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3800009550 con 0x7f380406dee0 2023-11-28T11:09:12.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f3809a24700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f380406dee0 0x7f38040c57a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3800009130 tx=0x7f3800004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f37faffd700 1 -- 172.21.15.156:0/2760272529 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380000b860 con 0x7f380406dee0 2023-11-28T11:09:12.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f37faffd700 1 -- 172.21.15.156:0/2760272529 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380000b9c0 con 0x7f380406dee0 2023-11-28T11:09:12.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f37faffd700 1 -- 172.21.15.156:0/2760272529 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380000bd70 con 0x7f380406dee0 2023-11-28T11:09:12.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f380c489700 1 -- 172.21.15.156:0/2760272529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380415d900 con 0x7f380406dee0 2023-11-28T11:09:12.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.460+0000 7f380c489700 1 -- 172.21.15.156:0/2760272529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380415d620 con 0x7f380406dee0 2023-11-28T11:09:12.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.461+0000 7f380c489700 1 -- 172.21.15.156:0/2760272529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380406e890 con 0x7f380406dee0 2023-11-28T11:09:12.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.462+0000 7f37faffd700 1 -- 172.21.15.156:0/2760272529 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3800017020 con 0x7f380406dee0 2023-11-28T11:09:12.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.462+0000 7f37faffd700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f37e80682f0 0x7f37e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.462+0000 7f380a225700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f37e80682f0 0x7f37e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.462+0000 7f37faffd700 1 -- 172.21.15.156:0/2760272529 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3800087150 con 0x7f380406dee0 2023-11-28T11:09:12.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.462+0000 7f380a225700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f37e80682f0 0x7f37e806a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f37f4006fd0 tx=0x7f37f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.466+0000 7f37faffd700 1 -- 172.21.15.156:0/2760272529 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3800053d00 con 0x7f380406dee0 2023-11-28T11:09:12.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.648+0000 7f380c489700 1 -- 172.21.15.156:0/2760272529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38040720a0 con 0x7f380406dee0 2023-11-28T11:09:12.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.648+0000 7f37faffd700 1 -- 172.21.15.156:0/2760272529 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3800057320 con 0x7f380406dee0 2023-11-28T11:09:12.650 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:09:12.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.650+0000 7f380c489700 1 -- 172.21.15.156:0/2760272529 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f37e80682f0 msgr2=0x7f37e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.650+0000 7f380c489700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f37e80682f0 0x7f37e806a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f37f4006fd0 tx=0x7f37f4008040 comp rx=0 tx=0).stop 2023-11-28T11:09:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.650+0000 7f380c489700 1 -- 172.21.15.156:0/2760272529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f380406dee0 msgr2=0x7f38040c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.650+0000 7f380c489700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f380406dee0 0x7f38040c57a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3800009130 tx=0x7f3800004750 comp rx=0 tx=0).stop 2023-11-28T11:09:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.650+0000 7f380c489700 1 -- 172.21.15.156:0/2760272529 shutdown_connections 2023-11-28T11:09:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.650+0000 7f380c489700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f37e80682f0 0x7f37e806a7a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.650+0000 7f380c489700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3804071040 0x7f38040c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.650+0000 7f380c489700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f380406cdc0 0x7f3804152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.650+0000 7f380c489700 1 --2- 172.21.15.156:0/2760272529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f380406dee0 0x7f38040c57a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.650+0000 7f380c489700 1 -- 172.21.15.156:0/2760272529 >> 172.21.15.156:0/2760272529 conn(0x7f38040d0000 msgr2=0x7f380406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.650+0000 7f380c489700 1 -- 172.21.15.156:0/2760272529 shutdown_connections 2023-11-28T11:09:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.650+0000 7f380c489700 1 -- 172.21.15.156:0/2760272529 wait complete. 2023-11-28T11:09:12.663 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.786+0000 7f13fc289700 1 Processor -- start 2023-11-28T11:09:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.786+0000 7f13fc289700 1 -- start start 2023-11-28T11:09:12.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.786+0000 7f13fc289700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13f4071040 0x7f13f4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13fc289700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f13f406dee0 0x7f13f406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13fc289700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f4071950 con 0x7f13f406cdc0 2023-11-28T11:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13fc289700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f4161550 con 0x7f13f406dee0 2023-11-28T11:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13fc289700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f415d060 con 0x7f13f4071040 2023-11-28T11:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13f9824700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f13f406dee0 0x7f13f406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13f9824700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f13f406dee0 0x7f13f406e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33628/0 (socket says 172.21.15.156:33628) 2023-11-28T11:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13f9824700 1 -- 172.21.15.156:0/1573047618 learned_addr learned my addr 172.21.15.156:0/1573047618 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13fa025700 1 --2- 172.21.15.156:0/1573047618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13f4071040 0x7f13f4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13f9824700 1 -- 172.21.15.156:0/1573047618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13f4071040 msgr2=0x7f13f4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13f9824700 1 --2- 172.21.15.156:0/1573047618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13f4071040 0x7f13f4071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13f9824700 1 -- 172.21.15.156:0/1573047618 >> v1:172.21.15.186:6789/0 conn(0x7f13f406cdc0 legacy=0x7f13f406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13f9824700 1 -- 172.21.15.156:0/1573047618 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f415d340 con 0x7f13f406dee0 2023-11-28T11:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13fa025700 1 --2- 172.21.15.156:0/1573047618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13f4071040 0x7f13f4071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13f9824700 1 --2- 172.21.15.156:0/1573047618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f406dee0 0x7f13f406e350 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f13f00098f0 tx=0x7f13f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be837c025a79f0d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13f9023700 1 -- 172.21.15.156:0/1573047618 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f000b8a0 con 0x7f13f406dee0 2023-11-28T11:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13f9023700 1 -- 172.21.15.156:0/1573047618 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f000ba00 con 0x7f13f406dee0 2023-11-28T11:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.787+0000 7f13f9023700 1 -- 172.21.15.156:0/1573047618 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f000bd90 con 0x7f13f406dee0 2023-11-28T11:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.788+0000 7f13fc289700 1 -- 172.21.15.156:0/1573047618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f406dee0 msgr2=0x7f13f406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.788+0000 7f13fc289700 1 --2- 172.21.15.156:0/1573047618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f406dee0 0x7f13f406e350 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f13f00098f0 tx=0x7f13f0009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.788+0000 7f13fc289700 1 -- 172.21.15.156:0/1573047618 shutdown_connections 2023-11-28T11:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.788+0000 7f13fc289700 1 --2- 172.21.15.156:0/1573047618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13f4071040 0x7f13f4071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.788+0000 7f13fc289700 1 --2- 172.21.15.156:0/1573047618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f406dee0 0x7f13f406e350 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.788+0000 7f13fc289700 1 -- 172.21.15.156:0/1573047618 >> 172.21.15.156:0/1573047618 conn(0x7f13f40d0000 msgr2=0x7f13f40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.788+0000 7f13fc289700 1 -- 172.21.15.156:0/1573047618 shutdown_connections 2023-11-28T11:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.788+0000 7f13fc289700 1 -- 172.21.15.156:0/1573047618 wait complete. 2023-11-28T11:09:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.789+0000 7f13fc289700 1 Processor -- start 2023-11-28T11:09:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.789+0000 7f13fc289700 1 -- start start 2023-11-28T11:09:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13fc289700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13f406cdc0 0x7f13f414a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13fc289700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f406dee0 0x7f13f40b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13fc289700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13f4071040 0x7f13f40c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13fc289700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40afa00 con 0x7f13f406dee0 2023-11-28T11:09:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13fc289700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f414a980 con 0x7f13f4071040 2023-11-28T11:09:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13fc289700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40b0f50 con 0x7f13f406cdc0 2023-11-28T11:09:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13f9824700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f406dee0 0x7f13f40b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13f9824700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f406dee0 0x7f13f40b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33636/0 (socket says 172.21.15.156:33636) 2023-11-28T11:09:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13f9824700 1 -- 172.21.15.156:0/1917573449 learned_addr learned my addr 172.21.15.156:0/1917573449 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13fa826700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13f4071040 0x7f13f40c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13fa025700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13f406cdc0 0x7f13f414a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13f9824700 1 -- 172.21.15.156:0/1917573449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13f406cdc0 msgr2=0x7f13f414a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13f9824700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13f406cdc0 0x7f13f414a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13f9824700 1 -- 172.21.15.156:0/1917573449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13f4071040 msgr2=0x7f13f40c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13f9824700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13f4071040 0x7f13f40c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.790+0000 7f13f9824700 1 -- 172.21.15.156:0/1917573449 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f0009550 con 0x7f13f406dee0 2023-11-28T11:09:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.791+0000 7f13f9824700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f406dee0 0x7f13f40b0a10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f13f0014040 tx=0x7f13f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.791+0000 7f13eaffd700 1 -- 172.21.15.156:0/1917573449 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0004590 con 0x7f13f406dee0 2023-11-28T11:09:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.791+0000 7f13eaffd700 1 -- 172.21.15.156:0/1917573449 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f001e070 con 0x7f13f406dee0 2023-11-28T11:09:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.791+0000 7f13fc289700 1 -- 172.21.15.156:0/1917573449 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f415d900 con 0x7f13f406dee0 2023-11-28T11:09:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.791+0000 7f13eaffd700 1 -- 172.21.15.156:0/1917573449 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f00038e0 con 0x7f13f406dee0 2023-11-28T11:09:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.791+0000 7f13fc289700 1 -- 172.21.15.156:0/1917573449 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f415d620 con 0x7f13f406dee0 2023-11-28T11:09:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.792+0000 7f13fc289700 1 -- 172.21.15.156:0/1917573449 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f406e890 con 0x7f13f406dee0 2023-11-28T11:09:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.792+0000 7f13eaffd700 1 -- 172.21.15.156:0/1917573449 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f13f000f050 con 0x7f13f406dee0 2023-11-28T11:09:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.792+0000 7f13eaffd700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f13d80682f0 0x7f13d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.792+0000 7f13eaffd700 1 -- 172.21.15.156:0/1917573449 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f13f0020070 con 0x7f13f406dee0 2023-11-28T11:09:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.793+0000 7f13fa025700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f13d80682f0 0x7f13d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.793+0000 7f13fa025700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f13d80682f0 0x7f13d806a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f13e400c940 tx=0x7f13e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.797+0000 7f13eaffd700 1 -- 172.21.15.156:0/1917573449 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f13f00538b0 con 0x7f13f406dee0 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.979+0000 7f13fc289700 1 -- 172.21.15.156:0/1917573449 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f13f406c360 con 0x7f13f406dee0 2023-11-28T11:09:12.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.980+0000 7f13eaffd700 1 -- 172.21.15.156:0/1917573449 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7f13f0056ed0 con 0x7f13f406dee0 2023-11-28T11:09:12.981 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:09:12.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.982+0000 7f13fc289700 1 -- 172.21.15.156:0/1917573449 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f13d80682f0 msgr2=0x7f13d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.982+0000 7f13fc289700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f13d80682f0 0x7f13d806a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f13e400c940 tx=0x7f13e4008040 comp rx=0 tx=0).stop 2023-11-28T11:09:12.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.982+0000 7f13fc289700 1 -- 172.21.15.156:0/1917573449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f406dee0 msgr2=0x7f13f40b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.982+0000 7f13fc289700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f406dee0 0x7f13f40b0a10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f13f0014040 tx=0x7f13f0004750 comp rx=0 tx=0).stop 2023-11-28T11:09:12.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.982+0000 7f13fc289700 1 -- 172.21.15.156:0/1917573449 shutdown_connections 2023-11-28T11:09:12.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.982+0000 7f13fc289700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13f4071040 0x7f13f40c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.982+0000 7f13fc289700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f13d80682f0 0x7f13d806a7a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.982+0000 7f13fc289700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13f406cdc0 0x7f13f414a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.982+0000 7f13fc289700 1 --2- 172.21.15.156:0/1917573449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f406dee0 0x7f13f40b0a10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.982+0000 7f13fc289700 1 -- 172.21.15.156:0/1917573449 >> 172.21.15.156:0/1917573449 conn(0x7f13f40d0000 msgr2=0x7f13f4157950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.982+0000 7f13fc289700 1 -- 172.21.15.156:0/1917573449 shutdown_connections 2023-11-28T11:09:12.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:12.982+0000 7f13fc289700 1 -- 172.21.15.156:0/1917573449 wait complete. 2023-11-28T11:09:12.995 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:09:13.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.115+0000 7fa2c91de700 1 Processor -- start 2023-11-28T11:09:13.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.115+0000 7fa2c91de700 1 -- start start 2023-11-28T11:09:13.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.116+0000 7fa2c91de700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa2c40b2090 0x7fa2c40b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.116+0000 7fa2c91de700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c40afd90 con 0x7fa2c40d14a0 2023-11-28T11:09:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.116+0000 7fa2c91de700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c40affe0 con 0x7fa2c415cbb0 2023-11-28T11:09:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.116+0000 7fa2c91de700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c40d3060 con 0x7fa2c40b2090 2023-11-28T11:09:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.116+0000 7fa2c259c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa2c40d14a0 0x7fa2c40b28c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48522/0 (socket says 172.21.15.156:48522) 2023-11-28T11:09:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.116+0000 7fa2c259c700 1 -- 172.21.15.156:0/1109501050 learned_addr learned my addr 172.21.15.156:0/1109501050 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.116+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672723246 0 0) 0x7fa2c40afd90 con 0x7fa2c40d14a0 2023-11-28T11:09:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.116+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac0035b0 con 0x7fa2c40d14a0 2023-11-28T11:09:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.116+0000 7fa2c359e700 1 --2- 172.21.15.156:0/1109501050 >> v2:172.21.15.186:3300/0 conn(0x7fa2c40b2090 0x7fa2c40b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3719745381 0 0) 0x7fa2ac0035b0 con 0x7fa2c40d14a0 2023-11-28T11:09:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 >> v2:172.21.15.186:3300/0 conn(0x7fa2c40b2090 msgr2=0x7fa2c40b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 --2- 172.21.15.156:0/1109501050 >> v2:172.21.15.186:3300/0 conn(0x7fa2c40b2090 0x7fa2c40b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 >> v1:172.21.15.186:6789/0 conn(0x7fa2c415cbb0 legacy=0x7fa2c40d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40b02c0 con 0x7fa2c40d14a0 2023-11-28T11:09:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fa2b4001670 con 0x7fa2c40d14a0 2023-11-28T11:09:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c359e700 1 --2- 172.21.15.156:0/1109501050 >> v2:172.21.15.186:3300/0 conn(0x7fa2c40b2090 0x7fa2c40b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 >> v1:172.21.15.156:6789/0 conn(0x7fa2c40d14a0 legacy=0x7fa2c40b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 --2- 172.21.15.156:0/1109501050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac003a00 0x7fa2ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 --2- 172.21.15.156:0/1109501050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2ac004850 0x7fa2ac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 --2- 172.21.15.156:0/1109501050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 0x7fa2ac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40d3060 con 0x7fa2ac005200 2023-11-28T11:09:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac00c190 con 0x7fa2ac004850 2023-11-28T11:09:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac00c350 con 0x7fa2ac003a00 2023-11-28T11:09:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c359e700 1 --2- 172.21.15.156:0/1109501050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 0x7fa2ac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c259c700 1 --2- 172.21.15.156:0/1109501050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2ac004850 0x7fa2ac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c2d9d700 1 --2- 172.21.15.156:0/1109501050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac003a00 0x7fa2ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c359e700 1 -- 172.21.15.156:0/1109501050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac003a00 msgr2=0x7fa2ac003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c359e700 1 --2- 172.21.15.156:0/1109501050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac003a00 0x7fa2ac003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c359e700 1 -- 172.21.15.156:0/1109501050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2ac004850 msgr2=0x7fa2ac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c359e700 1 --2- 172.21.15.156:0/1109501050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2ac004850 0x7fa2ac004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c359e700 1 -- 172.21.15.156:0/1109501050 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac0035b0 con 0x7fa2ac005200 2023-11-28T11:09:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c359e700 1 -- 172.21.15.156:0/1109501050 reap_dead start 2023-11-28T11:09:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c359e700 1 --2- 172.21.15.156:0/1109501050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 0x7fa2ac00ba10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0073b0 tx=0x7fa2ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d9e3b9e6871a87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc009490 con 0x7fa2ac005200 2023-11-28T11:09:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc014400 con 0x7fa2ac005200 2023-11-28T11:09:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.117+0000 7fa2c1d9b700 1 -- 172.21.15.156:0/1109501050 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0147b0 con 0x7fa2ac005200 2023-11-28T11:09:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.118+0000 7fa2c91de700 1 -- 172.21.15.156:0/1109501050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 msgr2=0x7fa2ac00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.118+0000 7fa2c91de700 1 --2- 172.21.15.156:0/1109501050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 0x7fa2ac00ba10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0073b0 tx=0x7fa2ac005880 comp rx=0 tx=0).stop 2023-11-28T11:09:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.118+0000 7fa2c91de700 1 -- 172.21.15.156:0/1109501050 shutdown_connections 2023-11-28T11:09:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.118+0000 7fa2c91de700 1 --2- 172.21.15.156:0/1109501050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 0x7fa2ac00ba10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.118+0000 7fa2c91de700 1 -- 172.21.15.156:0/1109501050 >> 172.21.15.156:0/1109501050 conn(0x7fa2c409c430 msgr2=0x7fa2c40d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.118+0000 7fa2c91de700 1 -- 172.21.15.156:0/1109501050 shutdown_connections 2023-11-28T11:09:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.118+0000 7fa2c91de700 1 -- 172.21.15.156:0/1109501050 wait complete. 2023-11-28T11:09:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.119+0000 7fa2c91de700 1 Processor -- start 2023-11-28T11:09:13.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.119+0000 7fa2c91de700 1 -- start start 2023-11-28T11:09:13.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c91de700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 0x7fa2c40c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c91de700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2c41580b0 0x7fa2c4155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c91de700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2c40c7330 0x7fa2c40c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c91de700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41f2290 con 0x7fa2ac005200 2023-11-28T11:09:13.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c91de700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40c7060 con 0x7fa2c40c7330 2023-11-28T11:09:13.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c91de700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40c7ce0 con 0x7fa2c41580b0 2023-11-28T11:09:13.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 0x7fa2c40c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 0x7fa2c40c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33654/0 (socket says 172.21.15.156:33654) 2023-11-28T11:09:13.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c2d9d700 1 -- 172.21.15.156:0/3826029445 learned_addr learned my addr 172.21.15.156:0/3826029445 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:13.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c259c700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2c41580b0 0x7fa2c4155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c359e700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2c40c7330 0x7fa2c40c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c2d9d700 1 -- 172.21.15.156:0/3826029445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2c41580b0 msgr2=0x7fa2c4155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c2d9d700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2c41580b0 0x7fa2c4155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c2d9d700 1 -- 172.21.15.156:0/3826029445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2c40c7330 msgr2=0x7fa2c40c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.120+0000 7fa2c2d9d700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2c40c7330 0x7fa2c40c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.121+0000 7fa2c2d9d700 1 -- 172.21.15.156:0/3826029445 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40b0880 con 0x7fa2ac005200 2023-11-28T11:09:13.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.121+0000 7fa2c359e700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2c40c7330 0x7fa2c40c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:13.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.121+0000 7fa2c2d9d700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 0x7fa2c40c6b20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800f650 tx=0x7fa2b800fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.121+0000 7fa2ab7fe700 1 -- 172.21.15.156:0/3826029445 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8005320 con 0x7fa2ac005200 2023-11-28T11:09:13.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.121+0000 7fa2ab7fe700 1 -- 172.21.15.156:0/3826029445 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b8005480 con 0x7fa2ac005200 2023-11-28T11:09:13.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.121+0000 7fa2c91de700 1 -- 172.21.15.156:0/3826029445 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c40b05a0 con 0x7fa2ac005200 2023-11-28T11:09:13.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.121+0000 7fa2ab7fe700 1 -- 172.21.15.156:0/3826029445 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8006af0 con 0x7fa2ac005200 2023-11-28T11:09:13.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.121+0000 7fa2c91de700 1 -- 172.21.15.156:0/3826029445 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c40b0b60 con 0x7fa2ac005200 2023-11-28T11:09:13.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.122+0000 7fa2c91de700 1 -- 172.21.15.156:0/3826029445 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c40745c0 con 0x7fa2ac005200 2023-11-28T11:09:13.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.122+0000 7fa2ab7fe700 1 -- 172.21.15.156:0/3826029445 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa2b801c030 con 0x7fa2ac005200 2023-11-28T11:09:13.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.123+0000 7fa2ab7fe700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa2b0068320 0x7fa2b006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.123+0000 7fa2ab7fe700 1 -- 172.21.15.156:0/3826029445 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa2b8084f00 con 0x7fa2ac005200 2023-11-28T11:09:13.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.123+0000 7fa2c259c700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa2b0068320 0x7fa2b006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.123+0000 7fa2c259c700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa2b0068320 0x7fa2b006a7d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40d3ad0 tx=0x7fa2b4001c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.127+0000 7fa2ab7fe700 1 -- 172.21.15.156:0/3826029445 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa2b8051a50 con 0x7fa2ac005200 2023-11-28T11:09:13.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.312+0000 7fa2c91de700 1 -- 172.21.15.156:0/3826029445 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa2c4158650 con 0x7fa2ac005200 2023-11-28T11:09:13.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.312+0000 7fa2ab7fe700 1 -- 172.21.15.156:0/3826029445 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v33) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa2b8055070 con 0x7fa2ac005200 2023-11-28T11:09:13.314 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:13.314 INFO:teuthology.orchestra.run.smithi156.stdout:[ 2023-11-28T11:09:13.314 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:13.314 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 1, 2023-11-28T11:09:13.314 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": ".mgr" 2023-11-28T11:09:13.314 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:13.315 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:13.315 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 2, 2023-11-28T11:09:13.315 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": "rbd" 2023-11-28T11:09:13.315 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:13.315 INFO:teuthology.orchestra.run.smithi156.stdout:] 2023-11-28T11:09:13.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.314+0000 7fa2c91de700 1 -- 172.21.15.156:0/3826029445 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa2b0068320 msgr2=0x7fa2b006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.314+0000 7fa2c91de700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa2b0068320 0x7fa2b006a7d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40d3ad0 tx=0x7fa2b4001c00 comp rx=0 tx=0).stop 2023-11-28T11:09:13.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.314+0000 7fa2c91de700 1 -- 172.21.15.156:0/3826029445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 msgr2=0x7fa2c40c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.314+0000 7fa2c91de700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 0x7fa2c40c6b20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800f650 tx=0x7fa2b800fa10 comp rx=0 tx=0).stop 2023-11-28T11:09:13.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.314+0000 7fa2c91de700 1 -- 172.21.15.156:0/3826029445 shutdown_connections 2023-11-28T11:09:13.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.314+0000 7fa2c91de700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2c40c7330 0x7fa2c40c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.314+0000 7fa2c91de700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac005200 0x7fa2c40c6b20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.314+0000 7fa2c91de700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa2b0068320 0x7fa2b006a7d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.314+0000 7fa2c91de700 1 --2- 172.21.15.156:0/3826029445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2c41580b0 0x7fa2c4155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.314+0000 7fa2c91de700 1 -- 172.21.15.156:0/3826029445 >> 172.21.15.156:0/3826029445 conn(0x7fa2c409c430 msgr2=0x7fa2c41e36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:13.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.314+0000 7fa2c91de700 1 -- 172.21.15.156:0/3826029445 shutdown_connections 2023-11-28T11:09:13.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.314+0000 7fa2c91de700 1 -- 172.21.15.156:0/3826029445 wait complete. 2023-11-28T11:09:13.328 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:09:13.328 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.451+0000 7f2237354700 1 Processor -- start 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.452+0000 7f2237354700 1 -- start start 2023-11-28T11:09:13.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.452+0000 7f2237354700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f223015aba0 0x7f22300b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.452+0000 7f2237354700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223015af70 con 0x7f22300b6ae0 2023-11-28T11:09:13.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.452+0000 7f2237354700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22300cc870 con 0x7f22300d0140 2023-11-28T11:09:13.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.452+0000 7f2237354700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22300cfec0 con 0x7f223015aba0 2023-11-28T11:09:13.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.453+0000 7f22350f0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f223015aba0 0x7f22300b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.453+0000 7f22348ef700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f22300b6ae0 0x7f22300cf4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48540/0 (socket says 172.21.15.156:48540) 2023-11-28T11:09:13.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.453+0000 7f22348ef700 1 -- 172.21.15.156:0/2230571985 learned_addr learned my addr 172.21.15.156:0/2230571985 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:13.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.453+0000 7f22350f0700 1 -- 172.21.15.156:0/2230571985 >> v1:172.21.15.186:6789/0 conn(0x7f22300d0140 legacy=0x7f22300cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.453+0000 7f22350f0700 1 -- 172.21.15.156:0/2230571985 >> v1:172.21.15.156:6789/0 conn(0x7f22300b6ae0 legacy=0x7f22300cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.453+0000 7f2227fff700 1 -- 172.21.15.156:0/2230571985 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422012936 0 0) 0x7f223015af70 con 0x7f22300b6ae0 2023-11-28T11:09:13.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.453+0000 7f22350f0700 1 -- 172.21.15.156:0/2230571985 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300b4290 con 0x7f223015aba0 2023-11-28T11:09:13.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.453+0000 7f22350f0700 1 --2- 172.21.15.156:0/2230571985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223015aba0 0x7f22300b3d50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2220009840 tx=0x7f2220009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75414aa74e150b70 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.453+0000 7f2227fff700 1 -- 172.21.15.156:0/2230571985 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220004230 con 0x7f223015aba0 2023-11-28T11:09:13.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.453+0000 7f2227fff700 1 -- 172.21.15.156:0/2230571985 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2220004390 con 0x7f223015aba0 2023-11-28T11:09:13.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.453+0000 7f2227fff700 1 -- 172.21.15.156:0/2230571985 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22200036a0 con 0x7f223015aba0 2023-11-28T11:09:13.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.454+0000 7f2237354700 1 -- 172.21.15.156:0/2230571985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223015aba0 msgr2=0x7f22300b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.454+0000 7f2237354700 1 --2- 172.21.15.156:0/2230571985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223015aba0 0x7f22300b3d50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2220009840 tx=0x7f2220009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:13.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.454+0000 7f2237354700 1 -- 172.21.15.156:0/2230571985 shutdown_connections 2023-11-28T11:09:13.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.454+0000 7f2237354700 1 --2- 172.21.15.156:0/2230571985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223015aba0 0x7f22300b3d50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.454+0000 7f2237354700 1 -- 172.21.15.156:0/2230571985 >> 172.21.15.156:0/2230571985 conn(0x7f223006df70 msgr2=0x7f223006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:13.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.454+0000 7f2237354700 1 -- 172.21.15.156:0/2230571985 shutdown_connections 2023-11-28T11:09:13.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.454+0000 7f2237354700 1 -- 172.21.15.156:0/2230571985 wait complete. 2023-11-28T11:09:13.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.455+0000 7f2237354700 1 Processor -- start 2023-11-28T11:09:13.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.455+0000 7f2237354700 1 -- start start 2023-11-28T11:09:13.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f2237354700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22300d0140 0x7f22300cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f2237354700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223015aba0 0x7f223015d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f2237354700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22300c8630 0x7f22300c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f2237354700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300c8ff0 con 0x7f223015aba0 2023-11-28T11:09:13.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f2237354700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223006bd90 con 0x7f22300d0140 2023-11-28T11:09:13.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f2237354700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223006c070 con 0x7f22300c8630 2023-11-28T11:09:13.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22348ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223015aba0 0x7f223015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22348ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223015aba0 0x7f223015d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33668/0 (socket says 172.21.15.156:33668) 2023-11-28T11:09:13.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22348ef700 1 -- 172.21.15.156:0/152600167 learned_addr learned my addr 172.21.15.156:0/152600167 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:13.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22350f0700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22300d0140 0x7f22300cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22358f1700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22300c8630 0x7f22300c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22348ef700 1 -- 172.21.15.156:0/152600167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22300c8630 msgr2=0x7f22300c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22348ef700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22300c8630 0x7f22300c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22348ef700 1 -- 172.21.15.156:0/152600167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22300d0140 msgr2=0x7f22300cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22348ef700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22300d0140 0x7f22300cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22348ef700 1 -- 172.21.15.156:0/152600167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22200094a0 con 0x7f223015aba0 2023-11-28T11:09:13.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22350f0700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22300d0140 0x7f22300cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:13.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22358f1700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22300c8630 0x7f22300c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:13.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f22348ef700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223015aba0 0x7f223015d880 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f222c00c520 tx=0x7f222c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f2225ffb700 1 -- 172.21.15.156:0/152600167 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c004c80 con 0x7f223015aba0 2023-11-28T11:09:13.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f2225ffb700 1 -- 172.21.15.156:0/152600167 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c004e00 con 0x7f223015aba0 2023-11-28T11:09:13.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f2237354700 1 -- 172.21.15.156:0/152600167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22300ccaa0 con 0x7f223015aba0 2023-11-28T11:09:13.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.456+0000 7f2225ffb700 1 -- 172.21.15.156:0/152600167 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c007310 con 0x7f223015aba0 2023-11-28T11:09:13.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.457+0000 7f2237354700 1 -- 172.21.15.156:0/152600167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22300cfbf0 con 0x7f223015aba0 2023-11-28T11:09:13.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.457+0000 7f2237354700 1 -- 172.21.15.156:0/152600167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22300ccd80 con 0x7f223015aba0 2023-11-28T11:09:13.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.458+0000 7f2225ffb700 1 -- 172.21.15.156:0/152600167 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f222c01a020 con 0x7f223015aba0 2023-11-28T11:09:13.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.458+0000 7f2225ffb700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f22140682f0 0x7f221406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.458+0000 7f2225ffb700 1 -- 172.21.15.156:0/152600167 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f222c085540 con 0x7f223015aba0 2023-11-28T11:09:13.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.458+0000 7f22350f0700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f22140682f0 0x7f221406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.459+0000 7f22350f0700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f22140682f0 0x7f221406a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f22200090b0 tx=0x7f2220013040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.462+0000 7f2225ffb700 1 -- 172.21.15.156:0/152600167 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f222c0520f0 con 0x7f223015aba0 2023-11-28T11:09:13.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.644+0000 7f2237354700 1 -- 172.21.15.156:0/152600167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f22300b3d30 con 0x7f223015aba0 2023-11-28T11:09:13.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.989+0000 7f2225ffb700 1 -- 172.21.15.156:0/152600167 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v34) v1 ==== 138+0+0 (secure 0 0 0) 0x7f222c055710 con 0x7f223015aba0 2023-11-28T11:09:13.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.992+0000 7f2237354700 1 -- 172.21.15.156:0/152600167 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f22140682f0 msgr2=0x7f221406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.992+0000 7f2237354700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f22140682f0 0x7f221406a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f22200090b0 tx=0x7f2220013040 comp rx=0 tx=0).stop 2023-11-28T11:09:13.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.992+0000 7f2237354700 1 -- 172.21.15.156:0/152600167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223015aba0 msgr2=0x7f223015d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.992+0000 7f2237354700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223015aba0 0x7f223015d880 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f222c00c520 tx=0x7f222c00c8e0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.993+0000 7f2237354700 1 -- 172.21.15.156:0/152600167 shutdown_connections 2023-11-28T11:09:13.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.993+0000 7f2237354700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f22140682f0 0x7f221406a7a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.993+0000 7f2237354700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22300c8630 0x7f22300c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.993+0000 7f2237354700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22300d0140 0x7f22300cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.993+0000 7f2237354700 1 --2- 172.21.15.156:0/152600167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223015aba0 0x7f223015d880 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.993+0000 7f2237354700 1 -- 172.21.15.156:0/152600167 >> 172.21.15.156:0/152600167 conn(0x7f223006df70 msgr2=0x7f22300b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:13.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.993+0000 7f2237354700 1 -- 172.21.15.156:0/152600167 shutdown_connections 2023-11-28T11:09:13.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:13.993+0000 7f2237354700 1 -- 172.21.15.156:0/152600167 wait complete. 2023-11-28T11:09:13.995 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' created 2023-11-28T11:09:14.008 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:09:14.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.135+0000 7fad21968700 1 Processor -- start 2023-11-28T11:09:14.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.135+0000 7fad21968700 1 -- start start 2023-11-28T11:09:14.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.136+0000 7fad21968700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fad1c0ae9d0 0x7fad1c15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.136+0000 7fad21968700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fad1c06a5e0 0x7fad1c06a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.136+0000 7fad21968700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1c0aee40 con 0x7fad1c06e500 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.136+0000 7fad21968700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1c15bf10 con 0x7fad1c0ae9d0 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.136+0000 7fad21968700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1c06c930 con 0x7fad1c06a5e0 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.136+0000 7fad1a7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fad1c0ae9d0 0x7fad1c15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.136+0000 7fad1a7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fad1c0ae9d0 0x7fad1c15b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33676/0 (socket says 172.21.15.156:33676) 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.136+0000 7fad1a7fc700 1 -- 172.21.15.156:0/1568131793 learned_addr learned my addr 172.21.15.156:0/1568131793 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.136+0000 7fad1b7fe700 1 --2- 172.21.15.156:0/1568131793 >> v2:172.21.15.186:3300/0 conn(0x7fad1c06a5e0 0x7fad1c06a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.137+0000 7fad1a7fc700 1 -- 172.21.15.156:0/1568131793 >> v2:172.21.15.186:3300/0 conn(0x7fad1c06a5e0 msgr2=0x7fad1c06a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.137+0000 7fad1a7fc700 1 --2- 172.21.15.156:0/1568131793 >> v2:172.21.15.186:3300/0 conn(0x7fad1c06a5e0 0x7fad1c06a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.137+0000 7fad1a7fc700 1 -- 172.21.15.156:0/1568131793 >> v1:172.21.15.186:6789/0 conn(0x7fad1c06e500 legacy=0x7fad1c06e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.137+0000 7fad1a7fc700 1 -- 172.21.15.156:0/1568131793 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c06cc10 con 0x7fad1c0ae9d0 2023-11-28T11:09:14.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.137+0000 7fad1a7fc700 1 --2- 172.21.15.156:0/1568131793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad1c0ae9d0 0x7fad1c15b9d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fad1c072160 tx=0x7fad0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ee779a8824f81c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:14.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.137+0000 7fad19ffb700 1 -- 172.21.15.156:0/1568131793 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00b870 con 0x7fad1c0ae9d0 2023-11-28T11:09:14.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.137+0000 7fad19ffb700 1 -- 172.21.15.156:0/1568131793 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c00b9d0 con 0x7fad1c0ae9d0 2023-11-28T11:09:14.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.137+0000 7fad19ffb700 1 -- 172.21.15.156:0/1568131793 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00bd60 con 0x7fad1c0ae9d0 2023-11-28T11:09:14.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.137+0000 7fad21968700 1 -- 172.21.15.156:0/1568131793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad1c0ae9d0 msgr2=0x7fad1c15b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.137+0000 7fad21968700 1 --2- 172.21.15.156:0/1568131793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad1c0ae9d0 0x7fad1c15b9d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fad1c072160 tx=0x7fad0c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.138+0000 7fad21968700 1 -- 172.21.15.156:0/1568131793 shutdown_connections 2023-11-28T11:09:14.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.138+0000 7fad21968700 1 --2- 172.21.15.156:0/1568131793 >> v2:172.21.15.186:3300/0 conn(0x7fad1c06a5e0 0x7fad1c06a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.138+0000 7fad21968700 1 --2- 172.21.15.156:0/1568131793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad1c0ae9d0 0x7fad1c15b9d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.138+0000 7fad21968700 1 -- 172.21.15.156:0/1568131793 >> 172.21.15.156:0/1568131793 conn(0x7fad1c0b12b0 msgr2=0x7fad1c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:14.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.138+0000 7fad21968700 1 -- 172.21.15.156:0/1568131793 shutdown_connections 2023-11-28T11:09:14.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.138+0000 7fad21968700 1 -- 172.21.15.156:0/1568131793 wait complete. 2023-11-28T11:09:14.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.139+0000 7fad21968700 1 Processor -- start 2023-11-28T11:09:14.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.139+0000 7fad21968700 1 -- start start 2023-11-28T11:09:14.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad21968700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad1c06a5e0 0x7fad1c1555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad21968700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad1c06e500 0x7fad1c15c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad21968700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad1c0ae9d0 0x7fad1c0c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad21968700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0c8480 con 0x7fad1c06e500 2023-11-28T11:09:14.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad21968700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0c7520 con 0x7fad1c0ae9d0 2023-11-28T11:09:14.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad21968700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0c7800 con 0x7fad1c06a5e0 2023-11-28T11:09:14.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad1a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad1c06e500 0x7fad1c15c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad1a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad1c06e500 0x7fad1c15c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33682/0 (socket says 172.21.15.156:33682) 2023-11-28T11:09:14.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad1a7fc700 1 -- 172.21.15.156:0/604151092 learned_addr learned my addr 172.21.15.156:0/604151092 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:14.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad1b7fe700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad1c0ae9d0 0x7fad1c0c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad1affd700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad1c06a5e0 0x7fad1c1555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad1a7fc700 1 -- 172.21.15.156:0/604151092 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad1c06a5e0 msgr2=0x7fad1c1555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad1a7fc700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad1c06a5e0 0x7fad1c1555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad1a7fc700 1 -- 172.21.15.156:0/604151092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad1c0ae9d0 msgr2=0x7fad1c0c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:14.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad1a7fc700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad1c0ae9d0 0x7fad1c0c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:14.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad1a7fc700 1 -- 172.21.15.156:0/604151092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c009550 con 0x7fad1c06e500 2023-11-28T11:09:14.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad1a7fc700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad1c06e500 0x7fad1c15c050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fad0c000c00 tx=0x7fad0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:14.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad03fff700 1 -- 172.21.15.156:0/604151092 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c004540 con 0x7fad1c06e500 2023-11-28T11:09:14.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad03fff700 1 -- 172.21.15.156:0/604151092 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c0036a0 con 0x7fad1c06e500 2023-11-28T11:09:14.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad21968700 1 -- 172.21.15.156:0/604151092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c06cef0 con 0x7fad1c06e500 2023-11-28T11:09:14.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad03fff700 1 -- 172.21.15.156:0/604151092 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c003a30 con 0x7fad1c06e500 2023-11-28T11:09:14.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.140+0000 7fad21968700 1 -- 172.21.15.156:0/604151092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c0664c0 con 0x7fad1c06e500 2023-11-28T11:09:14.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.141+0000 7fad21968700 1 -- 172.21.15.156:0/604151092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c15ac80 con 0x7fad1c06e500 2023-11-28T11:09:14.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.142+0000 7fad03fff700 1 -- 172.21.15.156:0/604151092 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fad0c02b070 con 0x7fad1c06e500 2023-11-28T11:09:14.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.142+0000 7fad03fff700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fad08068320 0x7fad0806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:14.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.142+0000 7fad03fff700 1 -- 172.21.15.156:0/604151092 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fad0c087010 con 0x7fad1c06e500 2023-11-28T11:09:14.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.142+0000 7fad1affd700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fad08068320 0x7fad0806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:14.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.143+0000 7fad1affd700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fad08068320 0x7fad0806a7d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad10000f60 tx=0x7fad10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:14.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.147+0000 7fad03fff700 1 -- 172.21.15.156:0/604151092 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fad0c053a50 con 0x7fad1c06e500 2023-11-28T11:09:14.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.328+0000 7fad21968700 1 -- 172.21.15.156:0/604151092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fad1c0c85c0 con 0x7fad1c06e500 2023-11-28T11:09:15.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:14.997+0000 7fad03fff700 1 -- 172.21.15.156:0/604151092 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v35) v1 ==== 170+0+0 (secure 0 0 0) 0x7fad1c0c85c0 con 0x7fad1c06e500 2023-11-28T11:09:15.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.003+0000 7fad21968700 1 -- 172.21.15.156:0/604151092 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fad08068320 msgr2=0x7fad0806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.003+0000 7fad21968700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fad08068320 0x7fad0806a7d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad10000f60 tx=0x7fad10009040 comp rx=0 tx=0).stop 2023-11-28T11:09:15.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.003+0000 7fad21968700 1 -- 172.21.15.156:0/604151092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad1c06e500 msgr2=0x7fad1c15c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.003+0000 7fad21968700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad1c06e500 0x7fad1c15c050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fad0c000c00 tx=0x7fad0c004750 comp rx=0 tx=0).stop 2023-11-28T11:09:15.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.003+0000 7fad21968700 1 -- 172.21.15.156:0/604151092 shutdown_connections 2023-11-28T11:09:15.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.003+0000 7fad21968700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad1c0ae9d0 0x7fad1c0c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.003+0000 7fad21968700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fad08068320 0x7fad0806a7d0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad10000f60 tx=0x7fad10009040 comp rx=0 tx=0).stop 2023-11-28T11:09:15.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.003+0000 7fad21968700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad1c06a5e0 0x7fad1c1555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.003+0000 7fad21968700 1 --2- 172.21.15.156:0/604151092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad1c06e500 0x7fad1c15c050 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.003+0000 7fad21968700 1 -- 172.21.15.156:0/604151092 >> 172.21.15.156:0/604151092 conn(0x7fad1c0b12b0 msgr2=0x7fad1c06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.004+0000 7fad21968700 1 -- 172.21.15.156:0/604151092 shutdown_connections 2023-11-28T11:09:15.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.004+0000 7fad21968700 1 -- 172.21.15.156:0/604151092 wait complete. 2023-11-28T11:09:15.007 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' created 2023-11-28T11:09:15.026 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:09:15.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.154+0000 7f02a196b700 1 Processor -- start 2023-11-28T11:09:15.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.154+0000 7f02a196b700 1 -- start start 2023-11-28T11:09:15.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.155+0000 7f02a196b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f029c158ce0 0x7f029c15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.155+0000 7f02a196b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029c06bf10 con 0x7f029c0ce4b0 2023-11-28T11:09:15.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.155+0000 7f02a196b700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029c15d730 con 0x7f029c06fb30 2023-11-28T11:09:15.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.155+0000 7f02a196b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029c1590b0 con 0x7f029c158ce0 2023-11-28T11:09:15.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.155+0000 7f029affd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f029c158ce0 0x7f029c15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.155+0000 7f029affd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f029c158ce0 0x7f029c15d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33688/0 (socket says 172.21.15.156:33688) 2023-11-28T11:09:15.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.155+0000 7f029b7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f029c0ce4b0 0x7f029c06c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48570/0 (socket says 172.21.15.156:48570) 2023-11-28T11:09:15.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.155+0000 7f029affd700 1 -- 172.21.15.156:0/3628846435 learned_addr learned my addr 172.21.15.156:0/3628846435 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:15.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.156+0000 7f029affd700 1 -- 172.21.15.156:0/3628846435 >> v1:172.21.15.186:6789/0 conn(0x7f029c06fb30 legacy=0x7f029c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:15.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.156+0000 7f029affd700 1 -- 172.21.15.156:0/3628846435 >> v1:172.21.15.156:6789/0 conn(0x7f029c0ce4b0 legacy=0x7f029c06c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.156+0000 7f029affd700 1 -- 172.21.15.156:0/3628846435 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c0cb0d0 con 0x7f029c158ce0 2023-11-28T11:09:15.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.156+0000 7f029affd700 1 --2- 172.21.15.156:0/3628846435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c158ce0 0x7f029c15d1f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f02900098f0 tx=0x7f0290009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4526620ea1471138 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.156+0000 7f0299ffb700 1 -- 172.21.15.156:0/3628846435 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290004230 con 0x7f029c158ce0 2023-11-28T11:09:15.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.156+0000 7f0299ffb700 1 -- 172.21.15.156:0/3628846435 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0290004390 con 0x7f029c158ce0 2023-11-28T11:09:15.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.156+0000 7f0299ffb700 1 -- 172.21.15.156:0/3628846435 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02900036a0 con 0x7f029c158ce0 2023-11-28T11:09:15.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.156+0000 7f02a196b700 1 -- 172.21.15.156:0/3628846435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c158ce0 msgr2=0x7f029c15d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.156+0000 7f02a196b700 1 --2- 172.21.15.156:0/3628846435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c158ce0 0x7f029c15d1f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f02900098f0 tx=0x7f0290009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:15.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.157+0000 7f02a196b700 1 -- 172.21.15.156:0/3628846435 shutdown_connections 2023-11-28T11:09:15.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.157+0000 7f02a196b700 1 --2- 172.21.15.156:0/3628846435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c158ce0 0x7f029c15d1f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.157+0000 7f02a196b700 1 -- 172.21.15.156:0/3628846435 >> 172.21.15.156:0/3628846435 conn(0x7f029c0cb670 msgr2=0x7f029c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.157+0000 7f02a196b700 1 -- 172.21.15.156:0/3628846435 shutdown_connections 2023-11-28T11:09:15.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.157+0000 7f02a196b700 1 -- 172.21.15.156:0/3628846435 wait complete. 2023-11-28T11:09:15.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.158+0000 7f02a196b700 1 Processor -- start 2023-11-28T11:09:15.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.158+0000 7f02a196b700 1 -- start start 2023-11-28T11:09:15.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f02a196b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f029c06fb30 0x7f029c0ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f02a196b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c0ce4b0 0x7f029c150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f02a196b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f029c158ce0 0x7f029c150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f02a196b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c0b36d0 con 0x7f029c0ce4b0 2023-11-28T11:09:15.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f02a196b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c149650 con 0x7f029c158ce0 2023-11-28T11:09:15.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f02a196b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029c0b3330 con 0x7f029c06fb30 2023-11-28T11:09:15.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f029a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c0ce4b0 0x7f029c150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f029a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c0ce4b0 0x7f029c150500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33696/0 (socket says 172.21.15.156:33696) 2023-11-28T11:09:15.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f029a7fc700 1 -- 172.21.15.156:0/3060669820 learned_addr learned my addr 172.21.15.156:0/3060669820 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:15.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f029affd700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f029c06fb30 0x7f029c0ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f029b7fe700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f029c158ce0 0x7f029c150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f029a7fc700 1 -- 172.21.15.156:0/3060669820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f029c06fb30 msgr2=0x7f029c0ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f029a7fc700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f029c06fb30 0x7f029c0ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f029a7fc700 1 -- 172.21.15.156:0/3060669820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f029c158ce0 msgr2=0x7f029c150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f029a7fc700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f029c158ce0 0x7f029c150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f029a7fc700 1 -- 172.21.15.156:0/3060669820 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0290009550 con 0x7f029c0ce4b0 2023-11-28T11:09:15.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f029a7fc700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c0ce4b0 0x7f029c150500 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f028c00c4d0 tx=0x7f028c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f0283fff700 1 -- 172.21.15.156:0/3060669820 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c011790 con 0x7f029c0ce4b0 2023-11-28T11:09:15.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f0283fff700 1 -- 172.21.15.156:0/3060669820 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028c0118f0 con 0x7f029c0ce4b0 2023-11-28T11:09:15.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f02a196b700 1 -- 172.21.15.156:0/3060669820 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029c06b8c0 con 0x7f029c0ce4b0 2023-11-28T11:09:15.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.159+0000 7f0283fff700 1 -- 172.21.15.156:0/3060669820 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c011ca0 con 0x7f029c0ce4b0 2023-11-28T11:09:15.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.160+0000 7f02a196b700 1 -- 172.21.15.156:0/3060669820 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029c0cb3b0 con 0x7f029c0ce4b0 2023-11-28T11:09:15.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.161+0000 7f02a196b700 1 -- 172.21.15.156:0/3060669820 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029c06bba0 con 0x7f029c0ce4b0 2023-11-28T11:09:15.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.161+0000 7f0283fff700 1 -- 172.21.15.156:0/3060669820 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f028c010050 con 0x7f029c0ce4b0 2023-11-28T11:09:15.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.161+0000 7f0283fff700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f02880682f0 0x7f028806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.161+0000 7f0283fff700 1 -- 172.21.15.156:0/3060669820 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f028c0859b0 con 0x7f029c0ce4b0 2023-11-28T11:09:15.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.161+0000 7f029affd700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f02880682f0 0x7f028806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.162+0000 7f029affd700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f02880682f0 0x7f028806a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0290009130 tx=0x7f0290008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.166+0000 7f0283fff700 1 -- 172.21.15.156:0/3060669820 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f028c052270 con 0x7f029c0ce4b0 2023-11-28T11:09:15.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.389+0000 7f02a196b700 1 -- 172.21.15.156:0/3060669820 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f029c0b3190 con 0x7f029c0ce4b0 2023-11-28T11:09:15.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.395+0000 7f0283fff700 1 -- 172.21.15.156:0/3060669820 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f029c0b3190 con 0x7f029c0ce4b0 2023-11-28T11:09:15.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.398+0000 7f02a196b700 1 -- 172.21.15.156:0/3060669820 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f02880682f0 msgr2=0x7f028806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.398+0000 7f02a196b700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f02880682f0 0x7f028806a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0290009130 tx=0x7f0290008030 comp rx=0 tx=0).stop 2023-11-28T11:09:15.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.399+0000 7f02a196b700 1 -- 172.21.15.156:0/3060669820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c0ce4b0 msgr2=0x7f029c150500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.399+0000 7f02a196b700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c0ce4b0 0x7f029c150500 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f028c00c4d0 tx=0x7f028c00c890 comp rx=0 tx=0).stop 2023-11-28T11:09:15.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.399+0000 7f02a196b700 1 -- 172.21.15.156:0/3060669820 shutdown_connections 2023-11-28T11:09:15.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.399+0000 7f02a196b700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f029c158ce0 0x7f029c150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.399+0000 7f02a196b700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f02880682f0 0x7f028806a7a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.399+0000 7f02a196b700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f029c06fb30 0x7f029c0ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.399+0000 7f02a196b700 1 --2- 172.21.15.156:0/3060669820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c0ce4b0 0x7f029c150500 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.399+0000 7f02a196b700 1 -- 172.21.15.156:0/3060669820 >> 172.21.15.156:0/3060669820 conn(0x7f029c0cb670 msgr2=0x7f029c070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.400+0000 7f02a196b700 1 -- 172.21.15.156:0/3060669820 shutdown_connections 2023-11-28T11:09:15.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.401+0000 7f02a196b700 1 -- 172.21.15.156:0/3060669820 wait complete. 2023-11-28T11:09:15.411 INFO:teuthology.orchestra.run.smithi156.stderr:new fs with metadata pool 5 and data pool 6 2023-11-28T11:09:15.423 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:15.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.543+0000 7fd16ecb5700 1 Processor -- start 2023-11-28T11:09:15.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.543+0000 7fd16ecb5700 1 -- start start 2023-11-28T11:09:15.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd16ecb5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd16806e9f0 0x7fd16806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd16ecb5700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16806ae60 con 0x7fd16806b830 2023-11-28T11:09:15.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd16ecb5700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16806a270 con 0x7fd16806d0a0 2023-11-28T11:09:15.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd16ecb5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16806a530 con 0x7fd16806e9f0 2023-11-28T11:09:15.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd167fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd16806b830 0x7fd16806c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48582/0 (socket says 172.21.15.156:48582) 2023-11-28T11:09:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd167fff700 1 -- 172.21.15.156:0/1929377861 learned_addr learned my addr 172.21.15.156:0/1929377861 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd16ca51700 1 --2- 172.21.15.156:0/1929377861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd16806e9f0 0x7fd16806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171115516 0 0) 0x7fd16806ae60 con 0x7fd16806b830 2023-11-28T11:09:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1580035b0 con 0x7fd16806b830 2023-11-28T11:09:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573174999 0 0) 0x7fd16806a270 con 0x7fd16806d0a0 2023-11-28T11:09:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd16806ae60 con 0x7fd16806d0a0 2023-11-28T11:09:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1901956206 0 0) 0x7fd1580035b0 con 0x7fd16806b830 2023-11-28T11:09:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd16806e9f0 msgr2=0x7fd16806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd1677fe700 1 --2- 172.21.15.156:0/1929377861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd16806e9f0 0x7fd16806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 >> v1:172.21.15.186:6789/0 conn(0x7fd16806d0a0 legacy=0x7fd1680ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16806a810 con 0x7fd16806b830 2023-11-28T11:09:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.544+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fd154001670 con 0x7fd16806b830 2023-11-28T11:09:15.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 >> v1:172.21.15.156:6789/0 conn(0x7fd16806b830 legacy=0x7fd16806c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd1677fe700 1 --2- 172.21.15.156:0/1929377861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd158003bc0 0x7fd158003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd1677fe700 1 --2- 172.21.15.156:0/1929377861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd16806e9f0 0x7fd158004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd1677fe700 1 --2- 172.21.15.156:0/1929377861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd158004d90 0x7fd158005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16806a530 con 0x7fd158004d90 2023-11-28T11:09:15.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15800cbd0 con 0x7fd158003bc0 2023-11-28T11:09:15.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15800cd90 con 0x7fd16806e9f0 2023-11-28T11:09:15.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd16d252700 1 --2- 172.21.15.156:0/1929377861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd158004d90 0x7fd158005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd16ca51700 1 --2- 172.21.15.156:0/1929377861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd158003bc0 0x7fd158003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd167fff700 1 --2- 172.21.15.156:0/1929377861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd16806e9f0 0x7fd158004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd16d252700 1 -- 172.21.15.156:0/1929377861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd16806e9f0 msgr2=0x7fd158004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:15.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd16d252700 1 --2- 172.21.15.156:0/1929377861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd16806e9f0 0x7fd158004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd16d252700 1 -- 172.21.15.156:0/1929377861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd158003bc0 msgr2=0x7fd158003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:15.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd16d252700 1 --2- 172.21.15.156:0/1929377861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd158003bc0 0x7fd158003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd16d252700 1 -- 172.21.15.156:0/1929377861 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1580035b0 con 0x7fd158004d90 2023-11-28T11:09:15.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd16d252700 1 --2- 172.21.15.156:0/1929377861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd158004d90 0x7fd158005200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd1680af1f0 tx=0x7fd158005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f92bdd3b7b19d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160001f30 con 0x7fd158004d90 2023-11-28T11:09:15.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16000ce50 con 0x7fd158004d90 2023-11-28T11:09:15.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.545+0000 7fd1677fe700 1 -- 172.21.15.156:0/1929377861 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160003c40 con 0x7fd158004d90 2023-11-28T11:09:15.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.546+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1929377861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd158004d90 msgr2=0x7fd158005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.546+0000 7fd16ecb5700 1 --2- 172.21.15.156:0/1929377861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd158004d90 0x7fd158005200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd1680af1f0 tx=0x7fd158005880 comp rx=0 tx=0).stop 2023-11-28T11:09:15.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.546+0000 7fd16d252700 1 -- 172.21.15.156:0/1929377861 reap_dead start 2023-11-28T11:09:15.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.546+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1929377861 shutdown_connections 2023-11-28T11:09:15.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.546+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1929377861 >> 172.21.15.156:0/1929377861 conn(0x7fd1680b12b0 msgr2=0x7fd1680b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.546+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1929377861 shutdown_connections 2023-11-28T11:09:15.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.546+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1929377861 wait complete. 2023-11-28T11:09:15.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.547+0000 7fd16ecb5700 1 Processor -- start 2023-11-28T11:09:15.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.547+0000 7fd16ecb5700 1 -- start start 2023-11-28T11:09:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ecb5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1681447c0 0x7fd1680b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ecb5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168144b90 0x7fd1680b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ecb5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1681e0500 0x7fd1681e09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ecb5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680b4e30 con 0x7fd1681447c0 2023-11-28T11:09:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ecb5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1681e0ee0 con 0x7fd168144b90 2023-11-28T11:09:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ecb5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680b41b0 con 0x7fd1681e0500 2023-11-28T11:09:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ca51700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1681447c0 0x7fd1680b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ca51700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1681447c0 0x7fd1680b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33714/0 (socket says 172.21.15.156:33714) 2023-11-28T11:09:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ca51700 1 -- 172.21.15.156:0/1291706807 learned_addr learned my addr 172.21.15.156:0/1291706807 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd167fff700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168144b90 0x7fd1680b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16d252700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1681e0500 0x7fd1681e09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ca51700 1 -- 172.21.15.156:0/1291706807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1681e0500 msgr2=0x7fd1681e09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ca51700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1681e0500 0x7fd1681e09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ca51700 1 -- 172.21.15.156:0/1291706807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168144b90 msgr2=0x7fd1680b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ca51700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168144b90 0x7fd1680b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.548+0000 7fd16ca51700 1 -- 172.21.15.156:0/1291706807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16806edc0 con 0x7fd1681447c0 2023-11-28T11:09:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.549+0000 7fd16ca51700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1681447c0 0x7fd1680b3640 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd15c011bb0 tx=0x7fd15c011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.549+0000 7fd1657fa700 1 -- 172.21.15.156:0/1291706807 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c016070 con 0x7fd1681447c0 2023-11-28T11:09:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.549+0000 7fd1657fa700 1 -- 172.21.15.156:0/1291706807 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c010b40 con 0x7fd1681447c0 2023-11-28T11:09:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.549+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1291706807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16806aaf0 con 0x7fd1681447c0 2023-11-28T11:09:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.549+0000 7fd1657fa700 1 -- 172.21.15.156:0/1291706807 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c01a410 con 0x7fd1681447c0 2023-11-28T11:09:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.549+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1291706807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16806ef50 con 0x7fd1681447c0 2023-11-28T11:09:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.550+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1291706807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1680cd5b0 con 0x7fd1681447c0 2023-11-28T11:09:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.550+0000 7fd1657fa700 1 -- 172.21.15.156:0/1291706807 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd15c021030 con 0x7fd1681447c0 2023-11-28T11:09:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.550+0000 7fd1657fa700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd14c068320 0x7fd14c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.550+0000 7fd1657fa700 1 -- 172.21.15.156:0/1291706807 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd15c087d50 con 0x7fd1681447c0 2023-11-28T11:09:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.551+0000 7fd167fff700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd14c068320 0x7fd14c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.551+0000 7fd167fff700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd14c068320 0x7fd14c06a7d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd1680c9890 tx=0x7fd154000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.555+0000 7fd1657fa700 1 -- 172.21.15.156:0/1291706807 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd15c0535e0 con 0x7fd1681447c0 2023-11-28T11:09:15.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.736+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1291706807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1680b0dd0 con 0x7fd1681447c0 2023-11-28T11:09:15.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.737+0000 7fd1657fa700 1 -- 172.21.15.156:0/1291706807 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18040 (secure 0 0 0) 0x7fd15c01f030 con 0x7fd1681447c0 2023-11-28T11:09:15.738 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:15.739 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":36,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:15.390922+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:13.663041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:14.347248+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:15.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.739+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1291706807 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd14c068320 msgr2=0x7fd14c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.739+0000 7fd16ecb5700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd14c068320 0x7fd14c06a7d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd1680c9890 tx=0x7fd154000ed0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.739+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1291706807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1681447c0 msgr2=0x7fd1680b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.739+0000 7fd16ecb5700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1681447c0 0x7fd1680b3640 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd15c011bb0 tx=0x7fd15c011ec0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.739+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1291706807 shutdown_connections 2023-11-28T11:09:15.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.739+0000 7fd16ecb5700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd14c068320 0x7fd14c06a7d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.739+0000 7fd16ecb5700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1681e0500 0x7fd1681e09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.739+0000 7fd16ecb5700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1681447c0 0x7fd1680b3640 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.739+0000 7fd16ecb5700 1 --2- 172.21.15.156:0/1291706807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168144b90 0x7fd1680b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.739+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1291706807 >> 172.21.15.156:0/1291706807 conn(0x7fd1680b12b0 msgr2=0x7fd168158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.739+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1291706807 shutdown_connections 2023-11-28T11:09:15.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.739+0000 7fd16ecb5700 1 -- 172.21.15.156:0/1291706807 wait complete. 2023-11-28T11:09:15.753 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:15.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.872+0000 7f62919ad700 1 Processor -- start 2023-11-28T11:09:15.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.872+0000 7f62919ad700 1 -- start start 2023-11-28T11:09:15.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f62919ad700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c15c9b0 0x7f628c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f62919ad700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f628c161340 0x7f628c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f62919ad700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f628c0d2320 0x7f628c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f62919ad700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c0b5800 con 0x7f628c0d2320 2023-11-28T11:09:15.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f62919ad700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c0d1950 con 0x7f628c161340 2023-11-28T11:09:15.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f62919ad700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b5520 con 0x7f628c15c9b0 2023-11-28T11:09:15.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628b7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f628c0d2320 0x7f628c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628b7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f628c0d2320 0x7f628c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33726/0 (socket says 172.21.15.156:33726) 2023-11-28T11:09:15.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628b7fe700 1 -- 172.21.15.156:0/2484023244 learned_addr learned my addr 172.21.15.156:0/2484023244 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:15.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628affd700 1 --2- 172.21.15.156:0/2484023244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c15c9b0 0x7f628c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628a7fc700 1 --2- 172.21.15.156:0/2484023244 >> v2:172.21.15.186:3300/0 conn(0x7f628c161340 0x7f628c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628b7fe700 1 -- 172.21.15.156:0/2484023244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c15c9b0 msgr2=0x7f628c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:15.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628b7fe700 1 --2- 172.21.15.156:0/2484023244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c15c9b0 0x7f628c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628b7fe700 1 -- 172.21.15.156:0/2484023244 >> v2:172.21.15.186:3300/0 conn(0x7f628c161340 msgr2=0x7f628c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:15.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628b7fe700 1 --2- 172.21.15.156:0/2484023244 >> v2:172.21.15.186:3300/0 conn(0x7f628c161340 0x7f628c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628b7fe700 1 -- 172.21.15.156:0/2484023244 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0d1300 con 0x7f628c0d2320 2023-11-28T11:09:15.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628a7fc700 1 --2- 172.21.15.156:0/2484023244 >> v2:172.21.15.186:3300/0 conn(0x7f628c161340 0x7f628c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628affd700 1 --2- 172.21.15.156:0/2484023244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c15c9b0 0x7f628c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f628b7fe700 1 --2- 172.21.15.156:0/2484023244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c0d2320 0x7f628c0cd7c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f628c06bc00 tx=0x7f628400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5932872223c31991 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.873+0000 7f6289ffb700 1 -- 172.21.15.156:0/2484023244 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628400e070 con 0x7f628c0d2320 2023-11-28T11:09:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.874+0000 7f6289ffb700 1 -- 172.21.15.156:0/2484023244 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628400b9c0 con 0x7f628c0d2320 2023-11-28T11:09:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.874+0000 7f6289ffb700 1 -- 172.21.15.156:0/2484023244 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628400bd70 con 0x7f628c0d2320 2023-11-28T11:09:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.874+0000 7f62919ad700 1 -- 172.21.15.156:0/2484023244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c0d2320 msgr2=0x7f628c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.874+0000 7f62919ad700 1 --2- 172.21.15.156:0/2484023244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c0d2320 0x7f628c0cd7c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f628c06bc00 tx=0x7f628400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.874+0000 7f62919ad700 1 -- 172.21.15.156:0/2484023244 shutdown_connections 2023-11-28T11:09:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.874+0000 7f62919ad700 1 --2- 172.21.15.156:0/2484023244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c0d2320 0x7f628c0cd7c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.874+0000 7f62919ad700 1 --2- 172.21.15.156:0/2484023244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c15c9b0 0x7f628c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.874+0000 7f62919ad700 1 --2- 172.21.15.156:0/2484023244 >> v2:172.21.15.186:3300/0 conn(0x7f628c161340 0x7f628c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.874+0000 7f62919ad700 1 -- 172.21.15.156:0/2484023244 >> 172.21.15.156:0/2484023244 conn(0x7f628c072600 msgr2=0x7f628c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.874+0000 7f62919ad700 1 -- 172.21.15.156:0/2484023244 shutdown_connections 2023-11-28T11:09:15.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.874+0000 7f62919ad700 1 -- 172.21.15.156:0/2484023244 wait complete. 2023-11-28T11:09:15.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.875+0000 7f62919ad700 1 Processor -- start 2023-11-28T11:09:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.875+0000 7f62919ad700 1 -- start start 2023-11-28T11:09:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f62919ad700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c0d2320 0x7f628c1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f62919ad700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c15c9b0 0x7f628c150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f62919ad700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c161340 0x7f628c151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f62919ad700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b6690 con 0x7f628c0d2320 2023-11-28T11:09:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f62919ad700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c1e8ae0 con 0x7f628c161340 2023-11-28T11:09:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f62919ad700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c1e8800 con 0x7f628c15c9b0 2023-11-28T11:09:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f628affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c0d2320 0x7f628c1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f628affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c0d2320 0x7f628c1542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33728/0 (socket says 172.21.15.156:33728) 2023-11-28T11:09:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f628affd700 1 -- 172.21.15.156:0/2242181509 learned_addr learned my addr 172.21.15.156:0/2242181509 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f628b7fe700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c161340 0x7f628c151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f628a7fc700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c15c9b0 0x7f628c150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f628affd700 1 -- 172.21.15.156:0/2242181509 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c15c9b0 msgr2=0x7f628c150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f628affd700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c15c9b0 0x7f628c150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f628affd700 1 -- 172.21.15.156:0/2242181509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c161340 msgr2=0x7f628c151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f628affd700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c161340 0x7f628c151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f628affd700 1 -- 172.21.15.156:0/2242181509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284009b30 con 0x7f628c0d2320 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.876+0000 7f628b7fe700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c161340 0x7f628c151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.877+0000 7f628affd700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c0d2320 0x7f628c1542b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f628000e810 tx=0x7f628000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.877+0000 7f6273fff700 1 -- 172.21.15.156:0/2242181509 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628000c020 con 0x7f628c0d2320 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.877+0000 7f6273fff700 1 -- 172.21.15.156:0/2242181509 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6280004250 con 0x7f628c0d2320 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.877+0000 7f6273fff700 1 -- 172.21.15.156:0/2242181509 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62800036a0 con 0x7f628c0d2320 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.877+0000 7f62919ad700 1 -- 172.21.15.156:0/2242181509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0b5130 con 0x7f628c0d2320 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.877+0000 7f62919ad700 1 -- 172.21.15.156:0/2242181509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c0d15e0 con 0x7f628c0d2320 2023-11-28T11:09:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.878+0000 7f62919ad700 1 -- 172.21.15.156:0/2242181509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c0b52c0 con 0x7f628c0d2320 2023-11-28T11:09:15.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.878+0000 7f6273fff700 1 -- 172.21.15.156:0/2242181509 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6280022070 con 0x7f628c0d2320 2023-11-28T11:09:15.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.878+0000 7f6273fff700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f62780682f0 0x7f627806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.879+0000 7f6273fff700 1 -- 172.21.15.156:0/2242181509 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6280085a20 con 0x7f628c0d2320 2023-11-28T11:09:15.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.879+0000 7f628a7fc700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f62780682f0 0x7f627806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.879+0000 7f628a7fc700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f62780682f0 0x7f627806a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f628c1f5f40 tx=0x7f627c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:15.883+0000 7f6273fff700 1 -- 172.21.15.156:0/2242181509 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6280052280 con 0x7f628c0d2320 2023-11-28T11:09:16.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.067+0000 7f62919ad700 1 -- 172.21.15.156:0/2242181509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f628c0745c0 con 0x7f628c0d2320 2023-11-28T11:09:16.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.069+0000 7f6273fff700 1 -- 172.21.15.156:0/2242181509 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18040 (secure 0 0 0) 0x7f628000f040 con 0x7f628c0d2320 2023-11-28T11:09:16.070 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:16.071 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":36,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:15.390922+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:13.663041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:14.347248+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:16.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.071+0000 7f62919ad700 1 -- 172.21.15.156:0/2242181509 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f62780682f0 msgr2=0x7f627806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.071+0000 7f62919ad700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f62780682f0 0x7f627806a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f628c1f5f40 tx=0x7f627c008040 comp rx=0 tx=0).stop 2023-11-28T11:09:16.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.071+0000 7f62919ad700 1 -- 172.21.15.156:0/2242181509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c0d2320 msgr2=0x7f628c1542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.071+0000 7f62919ad700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c0d2320 0x7f628c1542b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f628000e810 tx=0x7f628000ebd0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.071+0000 7f62919ad700 1 -- 172.21.15.156:0/2242181509 shutdown_connections 2023-11-28T11:09:16.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.071+0000 7f62919ad700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c161340 0x7f628c151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.071+0000 7f62919ad700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c0d2320 0x7f628c1542b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.071+0000 7f62919ad700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f62780682f0 0x7f627806a7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.071+0000 7f62919ad700 1 --2- 172.21.15.156:0/2242181509 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c15c9b0 0x7f628c150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.071+0000 7f62919ad700 1 -- 172.21.15.156:0/2242181509 >> 172.21.15.156:0/2242181509 conn(0x7f628c072600 msgr2=0x7f628c15fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:16.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.071+0000 7f62919ad700 1 -- 172.21.15.156:0/2242181509 shutdown_connections 2023-11-28T11:09:16.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.071+0000 7f62919ad700 1 -- 172.21.15.156:0/2242181509 wait complete. 2023-11-28T11:09:16.085 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:09:16.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.208+0000 7f4e70f1c700 1 Processor -- start 2023-11-28T11:09:16.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.208+0000 7f4e70f1c700 1 -- start start 2023-11-28T11:09:16.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.209+0000 7f4e70f1c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4e6c071040 0x7f4e6c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.209+0000 7f4e70f1c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06dee0 0x7f4e6c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.209+0000 7f4e70f1c700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c071950 con 0x7f4e6c06cdc0 2023-11-28T11:09:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.209+0000 7f4e70f1c700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c161550 con 0x7f4e6c071040 2023-11-28T11:09:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.209+0000 7f4e70f1c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c15d060 con 0x7f4e6c06dee0 2023-11-28T11:09:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e69d9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06dee0 0x7f4e6c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e6a59c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4e6c071040 0x7f4e6c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e6ad9d700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f4e6c06cdc0 0x7f4e6c06d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:58978/0 (socket says 172.21.15.156:58978) 2023-11-28T11:09:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e69d9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06dee0 0x7f4e6c06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37600/0 (socket says 172.21.15.156:37600) 2023-11-28T11:09:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e6ad9d700 1 -- 172.21.15.156:0/3767893327 learned_addr learned my addr 172.21.15.156:0/3767893327 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e6a59c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4e6c071040 0x7f4e6c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:44750/0 (socket says 172.21.15.156:44750) 2023-11-28T11:09:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e6959a700 1 -- 172.21.15.156:0/3767893327 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418467292 0 0) 0x7f4e6c071950 con 0x7f4e6c06cdc0 2023-11-28T11:09:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e6959a700 1 -- 172.21.15.156:0/3767893327 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e540035b0 con 0x7f4e6c06cdc0 2023-11-28T11:09:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e6a59c700 1 -- 172.21.15.156:0/3767893327 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06dee0 msgr2=0x7f4e6c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e6a59c700 1 --2- 172.21.15.156:0/3767893327 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06dee0 0x7f4e6c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e6a59c700 1 -- 172.21.15.156:0/3767893327 >> v1:172.21.15.186:6789/0 conn(0x7f4e6c06cdc0 legacy=0x7f4e6c06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e6a59c700 1 -- 172.21.15.156:0/3767893327 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c15d340 con 0x7f4e6c071040 2023-11-28T11:09:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e69d9b700 1 --2- 172.21.15.156:0/3767893327 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06dee0 0x7f4e6c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e6a59c700 1 --2- 172.21.15.156:0/3767893327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c071040 0x7f4e6c071410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009840 tx=0x7f4e60009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8ed0482bdd521bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.210+0000 7f4e6959a700 1 -- 172.21.15.156:0/3767893327 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60011830 con 0x7f4e6c071040 2023-11-28T11:09:16.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.211+0000 7f4e6959a700 1 -- 172.21.15.156:0/3767893327 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e60011990 con 0x7f4e6c071040 2023-11-28T11:09:16.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.211+0000 7f4e6959a700 1 -- 172.21.15.156:0/3767893327 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60011d20 con 0x7f4e6c071040 2023-11-28T11:09:16.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.211+0000 7f4e70f1c700 1 -- 172.21.15.156:0/3767893327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c071040 msgr2=0x7f4e6c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.211+0000 7f4e70f1c700 1 --2- 172.21.15.156:0/3767893327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c071040 0x7f4e6c071410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009840 tx=0x7f4e60009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:16.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.211+0000 7f4e70f1c700 1 -- 172.21.15.156:0/3767893327 shutdown_connections 2023-11-28T11:09:16.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.211+0000 7f4e70f1c700 1 --2- 172.21.15.156:0/3767893327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c071040 0x7f4e6c071410 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.211+0000 7f4e70f1c700 1 --2- 172.21.15.156:0/3767893327 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06dee0 0x7f4e6c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.211+0000 7f4e70f1c700 1 -- 172.21.15.156:0/3767893327 >> 172.21.15.156:0/3767893327 conn(0x7f4e6c0d0000 msgr2=0x7f4e6c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:16.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.211+0000 7f4e70f1c700 1 -- 172.21.15.156:0/3767893327 shutdown_connections 2023-11-28T11:09:16.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.211+0000 7f4e70f1c700 1 -- 172.21.15.156:0/3767893327 wait complete. 2023-11-28T11:09:16.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.213+0000 7f4e70f1c700 1 Processor -- start 2023-11-28T11:09:16.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.213+0000 7f4e70f1c700 1 -- start start 2023-11-28T11:09:16.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.213+0000 7f4e70f1c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06cdc0 0x7f4e6c1f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.213+0000 7f4e70f1c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e6c06dee0 0x7f4e6c1556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.213+0000 7f4e70f1c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c071040 0x7f4e6c145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.213+0000 7f4e70f1c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c146ee0 con 0x7f4e6c06dee0 2023-11-28T11:09:16.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.213+0000 7f4e70f1c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c146920 con 0x7f4e6c071040 2023-11-28T11:09:16.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.213+0000 7f4e70f1c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c146c00 con 0x7f4e6c06cdc0 2023-11-28T11:09:16.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.213+0000 7f4e69d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e6c06dee0 0x7f4e6c1556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.213+0000 7f4e69d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e6c06dee0 0x7f4e6c1556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33742/0 (socket says 172.21.15.156:33742) 2023-11-28T11:09:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.213+0000 7f4e69d9b700 1 -- 172.21.15.156:0/546890084 learned_addr learned my addr 172.21.15.156:0/546890084 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e6ad9d700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c071040 0x7f4e6c145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e6a59c700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06cdc0 0x7f4e6c1f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e69d9b700 1 -- 172.21.15.156:0/546890084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06cdc0 msgr2=0x7f4e6c1f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e69d9b700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06cdc0 0x7f4e6c1f5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e69d9b700 1 -- 172.21.15.156:0/546890084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c071040 msgr2=0x7f4e6c145f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e69d9b700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c071040 0x7f4e6c145f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e69d9b700 1 -- 172.21.15.156:0/546890084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c0094a0 con 0x7f4e6c06dee0 2023-11-28T11:09:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e6ad9d700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c071040 0x7f4e6c145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e6a59c700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06cdc0 0x7f4e6c1f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e69d9b700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e6c06dee0 0x7f4e6c1556c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00ebe0 tx=0x7f4e5c009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e5affd700 1 -- 172.21.15.156:0/546890084 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00c8b0 con 0x7f4e6c06dee0 2023-11-28T11:09:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e5affd700 1 -- 172.21.15.156:0/546890084 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c00ca10 con 0x7f4e6c06dee0 2023-11-28T11:09:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e70f1c700 1 -- 172.21.15.156:0/546890084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e600094a0 con 0x7f4e6c06dee0 2023-11-28T11:09:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e5affd700 1 -- 172.21.15.156:0/546890084 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00cda0 con 0x7f4e6c06dee0 2023-11-28T11:09:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.214+0000 7f4e70f1c700 1 -- 172.21.15.156:0/546890084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c15d900 con 0x7f4e6c06dee0 2023-11-28T11:09:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.215+0000 7f4e70f1c700 1 -- 172.21.15.156:0/546890084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6c15d620 con 0x7f4e6c06dee0 2023-11-28T11:09:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.216+0000 7f4e5affd700 1 -- 172.21.15.156:0/546890084 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e5c019030 con 0x7f4e6c06dee0 2023-11-28T11:09:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.216+0000 7f4e5affd700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4e4c0682f0 0x7f4e4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:16.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.216+0000 7f4e5affd700 1 -- 172.21.15.156:0/546890084 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e5c085d80 con 0x7f4e6c06dee0 2023-11-28T11:09:16.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.216+0000 7f4e6a59c700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4e4c0682f0 0x7f4e4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:16.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.216+0000 7f4e6a59c700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4e4c0682f0 0x7f4e4c06a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4e600090b0 tx=0x7f4e60008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:16.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.220+0000 7f4e5affd700 1 -- 172.21.15.156:0/546890084 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e5c052560 con 0x7f4e6c06dee0 2023-11-28T11:09:16.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:16.452+0000 7f4e70f1c700 1 -- 172.21.15.156:0/546890084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4e6c155510 con 0x7f4e6c06dee0 2023-11-28T11:09:17.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.412+0000 7f4e5affd700 1 -- 172.21.15.156:0/546890084 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4e5c055b80 con 0x7f4e6c06dee0 2023-11-28T11:09:17.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.415+0000 7f4e70f1c700 1 -- 172.21.15.156:0/546890084 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4e4c0682f0 msgr2=0x7f4e4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.415+0000 7f4e70f1c700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4e4c0682f0 0x7f4e4c06a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4e600090b0 tx=0x7f4e60008030 comp rx=0 tx=0).stop 2023-11-28T11:09:17.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.415+0000 7f4e70f1c700 1 -- 172.21.15.156:0/546890084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e6c06dee0 msgr2=0x7f4e6c1556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.415+0000 7f4e70f1c700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e6c06dee0 0x7f4e6c1556c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00ebe0 tx=0x7f4e5c009e20 comp rx=0 tx=0).stop 2023-11-28T11:09:17.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.415+0000 7f4e70f1c700 1 -- 172.21.15.156:0/546890084 shutdown_connections 2023-11-28T11:09:17.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.415+0000 7f4e70f1c700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c071040 0x7f4e6c145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.415+0000 7f4e70f1c700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4e4c0682f0 0x7f4e4c06a7a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.415+0000 7f4e70f1c700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c06cdc0 0x7f4e6c1f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.415+0000 7f4e70f1c700 1 --2- 172.21.15.156:0/546890084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e6c06dee0 0x7f4e6c1556c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.415+0000 7f4e70f1c700 1 -- 172.21.15.156:0/546890084 >> 172.21.15.156:0/546890084 conn(0x7f4e6c0d0000 msgr2=0x7f4e6c06d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:17.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.416+0000 7f4e70f1c700 1 -- 172.21.15.156:0/546890084 shutdown_connections 2023-11-28T11:09:17.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.416+0000 7f4e70f1c700 1 -- 172.21.15.156:0/546890084 wait complete. 2023-11-28T11:09:17.432 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:09:17.432 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:09:17.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.560+0000 7f44697c7700 1 Processor -- start 2023-11-28T11:09:17.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.560+0000 7f44697c7700 1 -- start start 2023-11-28T11:09:17.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.561+0000 7f44697c7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f446406ba90 0x7f44640af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.561+0000 7f44697c7700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f44640cbdf0 0x7f44640ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.561+0000 7f44697c7700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44640cc5f0 con 0x7f446415cef0 2023-11-28T11:09:17.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.561+0000 7f44697c7700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446415d470 con 0x7f44640cbdf0 2023-11-28T11:09:17.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.561+0000 7f44697c7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446415d750 con 0x7f446406ba90 2023-11-28T11:09:17.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f44627fc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f446406ba90 0x7f44640af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f44637fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f44640cbdf0 0x7f44640ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f44627fc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f446406ba90 0x7f44640af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37616/0 (socket says 172.21.15.156:37616) 2023-11-28T11:09:17.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f4462ffd700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f446415cef0 0x7f446406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:58988/0 (socket says 172.21.15.156:58988) 2023-11-28T11:09:17.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f44627fc700 1 -- 172.21.15.156:0/1987978331 learned_addr learned my addr 172.21.15.156:0/1987978331 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:17.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f44637fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f44640cbdf0 0x7f44640ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:44768/0 (socket says 172.21.15.156:44768) 2023-11-28T11:09:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f4461ffb700 1 -- 172.21.15.156:0/1987978331 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863552342 0 0) 0x7f44640cc5f0 con 0x7f446415cef0 2023-11-28T11:09:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f4461ffb700 1 -- 172.21.15.156:0/1987978331 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f444c0035b0 con 0x7f446415cef0 2023-11-28T11:09:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f44637fe700 1 -- 172.21.15.156:0/1987978331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f446406ba90 msgr2=0x7f44640af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f44637fe700 1 --2- 172.21.15.156:0/1987978331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f446406ba90 0x7f44640af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f44637fe700 1 -- 172.21.15.156:0/1987978331 >> v1:172.21.15.186:6789/0 conn(0x7f446415cef0 legacy=0x7f446406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f44637fe700 1 -- 172.21.15.156:0/1987978331 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446415da30 con 0x7f44640cbdf0 2023-11-28T11:09:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f44627fc700 1 --2- 172.21.15.156:0/1987978331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f446406ba90 0x7f44640af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f44637fe700 1 --2- 172.21.15.156:0/1987978331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44640cbdf0 0x7f44640ca650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f445c00bfa0 tx=0x7f445c00d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8248f66f7f349bea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f4461ffb700 1 -- 172.21.15.156:0/1987978331 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c010070 con 0x7f44640cbdf0 2023-11-28T11:09:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f4461ffb700 1 -- 172.21.15.156:0/1987978331 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c00b2e0 con 0x7f44640cbdf0 2023-11-28T11:09:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.562+0000 7f4461ffb700 1 -- 172.21.15.156:0/1987978331 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c00dc00 con 0x7f44640cbdf0 2023-11-28T11:09:17.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.563+0000 7f44697c7700 1 -- 172.21.15.156:0/1987978331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44640cbdf0 msgr2=0x7f44640ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.563+0000 7f44697c7700 1 --2- 172.21.15.156:0/1987978331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44640cbdf0 0x7f44640ca650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f445c00bfa0 tx=0x7f445c00d330 comp rx=0 tx=0).stop 2023-11-28T11:09:17.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.563+0000 7f44697c7700 1 -- 172.21.15.156:0/1987978331 shutdown_connections 2023-11-28T11:09:17.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.563+0000 7f44697c7700 1 --2- 172.21.15.156:0/1987978331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44640cbdf0 0x7f44640ca650 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.563+0000 7f44697c7700 1 --2- 172.21.15.156:0/1987978331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f446406ba90 0x7f44640af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.563+0000 7f44697c7700 1 -- 172.21.15.156:0/1987978331 >> 172.21.15.156:0/1987978331 conn(0x7f44640d0000 msgr2=0x7f44640d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:17.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.563+0000 7f44697c7700 1 -- 172.21.15.156:0/1987978331 shutdown_connections 2023-11-28T11:09:17.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.563+0000 7f44697c7700 1 -- 172.21.15.156:0/1987978331 wait complete. 2023-11-28T11:09:17.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.564+0000 7f44697c7700 1 Processor -- start 2023-11-28T11:09:17.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.564+0000 7f44697c7700 1 -- start start 2023-11-28T11:09:17.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.565+0000 7f44697c7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f446406ba90 0x7f446415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.565+0000 7f44697c7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44640cbdf0 0x7f44640b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.565+0000 7f44697c7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f446415cef0 0x7f44640b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.565+0000 7f44697c7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44640b3800 con 0x7f446406ba90 2023-11-28T11:09:17.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.565+0000 7f44697c7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44640b2ed0 con 0x7f44640cbdf0 2023-11-28T11:09:17.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.565+0000 7f44697c7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44640b3140 con 0x7f446415cef0 2023-11-28T11:09:17.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.565+0000 7f4462ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f446406ba90 0x7f446415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.565+0000 7f4462ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f446406ba90 0x7f446415ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33756/0 (socket says 172.21.15.156:33756) 2023-11-28T11:09:17.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.565+0000 7f4462ffd700 1 -- 172.21.15.156:0/3722716388 learned_addr learned my addr 172.21.15.156:0/3722716388 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:17.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.565+0000 7f44627fc700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44640cbdf0 0x7f44640b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.565+0000 7f44637fe700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f446415cef0 0x7f44640b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.566+0000 7f4462ffd700 1 -- 172.21.15.156:0/3722716388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f446415cef0 msgr2=0x7f44640b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.566+0000 7f4462ffd700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f446415cef0 0x7f44640b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.566+0000 7f4462ffd700 1 -- 172.21.15.156:0/3722716388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44640cbdf0 msgr2=0x7f44640b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.566+0000 7f4462ffd700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44640cbdf0 0x7f44640b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.566+0000 7f4462ffd700 1 -- 172.21.15.156:0/3722716388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4454009550 con 0x7f446406ba90 2023-11-28T11:09:17.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.566+0000 7f44627fc700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44640cbdf0 0x7f44640b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:17.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.566+0000 7f4462ffd700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f446406ba90 0x7f446415ac50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f445800c440 tx=0x7f445800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.566+0000 7f444b7fe700 1 -- 172.21.15.156:0/3722716388 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458005cb0 con 0x7f446406ba90 2023-11-28T11:09:17.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.566+0000 7f444b7fe700 1 -- 172.21.15.156:0/3722716388 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4458005e10 con 0x7f446406ba90 2023-11-28T11:09:17.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.566+0000 7f44697c7700 1 -- 172.21.15.156:0/3722716388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c00bc80 con 0x7f446406ba90 2023-11-28T11:09:17.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.566+0000 7f444b7fe700 1 -- 172.21.15.156:0/3722716388 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458007310 con 0x7f446406ba90 2023-11-28T11:09:17.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.566+0000 7f44697c7700 1 -- 172.21.15.156:0/3722716388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446415dff0 con 0x7f446406ba90 2023-11-28T11:09:17.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.567+0000 7f44697c7700 1 -- 172.21.15.156:0/3722716388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446415dd10 con 0x7f446406ba90 2023-11-28T11:09:17.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.567+0000 7f444b7fe700 1 -- 172.21.15.156:0/3722716388 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4458019030 con 0x7f446406ba90 2023-11-28T11:09:17.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.568+0000 7f444b7fe700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f44500682f0 0x7f445006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.568+0000 7f444b7fe700 1 -- 172.21.15.156:0/3722716388 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4458084ee0 con 0x7f446406ba90 2023-11-28T11:09:17.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.568+0000 7f44627fc700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f44500682f0 0x7f445006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.568+0000 7f44627fc700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f44500682f0 0x7f445006a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44640cf230 tx=0x7f44540092c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.572+0000 7f444b7fe700 1 -- 172.21.15.156:0/3722716388 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4458051770 con 0x7f446406ba90 2023-11-28T11:09:17.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:17.804+0000 7f44697c7700 1 -- 172.21.15.156:0/3722716388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f44640cab90 con 0x7f446406ba90 2023-11-28T11:09:18.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.417+0000 7f444b7fe700 1 -- 172.21.15.156:0/3722716388 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4458054d90 con 0x7f446406ba90 2023-11-28T11:09:18.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.419+0000 7f44697c7700 1 -- 172.21.15.156:0/3722716388 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f44500682f0 msgr2=0x7f445006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.419+0000 7f44697c7700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f44500682f0 0x7f445006a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44640cf230 tx=0x7f44540092c0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.419+0000 7f44697c7700 1 -- 172.21.15.156:0/3722716388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f446406ba90 msgr2=0x7f446415ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.419+0000 7f44697c7700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f446406ba90 0x7f446415ac50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f445800c440 tx=0x7f445800c750 comp rx=0 tx=0).stop 2023-11-28T11:09:18.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.419+0000 7f44697c7700 1 -- 172.21.15.156:0/3722716388 shutdown_connections 2023-11-28T11:09:18.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.419+0000 7f44697c7700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f44500682f0 0x7f445006a7a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.420+0000 7f44697c7700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f446415cef0 0x7f44640b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.420+0000 7f44697c7700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f446406ba90 0x7f446415ac50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.420+0000 7f44697c7700 1 --2- 172.21.15.156:0/3722716388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44640cbdf0 0x7f44640b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.420+0000 7f44697c7700 1 -- 172.21.15.156:0/3722716388 >> 172.21.15.156:0/3722716388 conn(0x7f44640d0000 msgr2=0x7f446406d480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:18.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.420+0000 7f44697c7700 1 -- 172.21.15.156:0/3722716388 shutdown_connections 2023-11-28T11:09:18.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.420+0000 7f44697c7700 1 -- 172.21.15.156:0/3722716388 wait complete. 2023-11-28T11:09:18.436 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:09:18.436 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:18.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.561+0000 7fdc30e6e700 1 Processor -- start 2023-11-28T11:09:18.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.561+0000 7fdc30e6e700 1 -- start start 2023-11-28T11:09:18.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc30e6e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdc2c0ae9d0 0x7fdc2c15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:18.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc30e6e700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdc2c06a5e0 0x7fdc2c06a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:18.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc30e6e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2c0aee40 con 0x7fdc2c06e500 2023-11-28T11:09:18.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc30e6e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c15bf10 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc30e6e700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c06c930 con 0x7fdc2c06a5e0 2023-11-28T11:09:18.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc29d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdc2c0ae9d0 0x7fdc2c15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:18.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc2a59c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fdc2c06e500 0x7fdc2c06e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48642/0 (socket says 172.21.15.156:48642) 2023-11-28T11:09:18.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc2a59c700 1 -- 172.21.15.156:0/4064510137 learned_addr learned my addr 172.21.15.156:0/4064510137 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:18.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc29d9b700 1 -- 172.21.15.156:0/4064510137 >> v2:172.21.15.186:3300/0 conn(0x7fdc2c06a5e0 msgr2=0x7fdc2c06a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:18.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc29d9b700 1 --2- 172.21.15.156:0/4064510137 >> v2:172.21.15.186:3300/0 conn(0x7fdc2c06a5e0 0x7fdc2c06a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc29d9b700 1 -- 172.21.15.156:0/4064510137 >> v1:172.21.15.156:6789/0 conn(0x7fdc2c06e500 legacy=0x7fdc2c06e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc29d9b700 1 -- 172.21.15.156:0/4064510137 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c06cc10 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.562+0000 7fdc2959a700 1 -- 172.21.15.156:0/4064510137 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968973115 0 0) 0x7fdc2c0aee40 con 0x7fdc2c06e500 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.563+0000 7fdc29d9b700 1 --2- 172.21.15.156:0/4064510137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2c0ae9d0 0x7fdc2c15b9d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c072160 tx=0x7fdc1c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6320efd68241c56e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.563+0000 7fdc2959a700 1 -- 172.21.15.156:0/4064510137 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c011830 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.563+0000 7fdc2959a700 1 -- 172.21.15.156:0/4064510137 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c011990 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.563+0000 7fdc2959a700 1 -- 172.21.15.156:0/4064510137 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c011d20 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.563+0000 7fdc30e6e700 1 -- 172.21.15.156:0/4064510137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2c0ae9d0 msgr2=0x7fdc2c15b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.563+0000 7fdc30e6e700 1 --2- 172.21.15.156:0/4064510137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2c0ae9d0 0x7fdc2c15b9d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c072160 tx=0x7fdc1c009b20 comp rx=0 tx=0).stop 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.563+0000 7fdc30e6e700 1 -- 172.21.15.156:0/4064510137 shutdown_connections 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.563+0000 7fdc30e6e700 1 --2- 172.21.15.156:0/4064510137 >> v2:172.21.15.186:3300/0 conn(0x7fdc2c06a5e0 0x7fdc2c06a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.563+0000 7fdc30e6e700 1 --2- 172.21.15.156:0/4064510137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2c0ae9d0 0x7fdc2c15b9d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.563+0000 7fdc30e6e700 1 -- 172.21.15.156:0/4064510137 >> 172.21.15.156:0/4064510137 conn(0x7fdc2c0b12b0 msgr2=0x7fdc2c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:18.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.563+0000 7fdc30e6e700 1 -- 172.21.15.156:0/4064510137 shutdown_connections 2023-11-28T11:09:18.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.563+0000 7fdc30e6e700 1 -- 172.21.15.156:0/4064510137 wait complete. 2023-11-28T11:09:18.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.565+0000 7fdc30e6e700 1 Processor -- start 2023-11-28T11:09:18.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.565+0000 7fdc30e6e700 1 -- start start 2023-11-28T11:09:18.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.565+0000 7fdc30e6e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c06a5e0 0x7fdc2c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:18.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.565+0000 7fdc30e6e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc2c06e500 0x7fdc2c1e4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:18.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc30e6e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2c0ae9d0 0x7fdc2c1e52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:18.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc30e6e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1e57f0 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc30e6e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c146d60 con 0x7fdc2c06a5e0 2023-11-28T11:09:18.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc30e6e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c146a80 con 0x7fdc2c06e500 2023-11-28T11:09:18.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc2ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2c0ae9d0 0x7fdc2c1e52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:18.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc2ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2c0ae9d0 0x7fdc2c1e52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33774/0 (socket says 172.21.15.156:33774) 2023-11-28T11:09:18.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc2ad9d700 1 -- 172.21.15.156:0/1286281821 learned_addr learned my addr 172.21.15.156:0/1286281821 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:18.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc2ad9d700 1 -- 172.21.15.156:0/1286281821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc2c06e500 msgr2=0x7fdc2c1e4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc2a59c700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c06a5e0 0x7fdc2c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:18.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc2ad9d700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc2c06e500 0x7fdc2c1e4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc2ad9d700 1 -- 172.21.15.156:0/1286281821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c06a5e0 msgr2=0x7fdc2c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc2ad9d700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c06a5e0 0x7fdc2c0caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc2ad9d700 1 -- 172.21.15.156:0/1286281821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0094a0 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc2a59c700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c06a5e0 0x7fdc2c0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:18.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc2ad9d700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2c0ae9d0 0x7fdc2c1e52b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b9a0 tx=0x7fdc2400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:18.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc1affd700 1 -- 172.21.15.156:0/1286281821 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2400c9b0 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc1affd700 1 -- 172.21.15.156:0/1286281821 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2400cb30 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc30e6e700 1 -- 172.21.15.156:0/1286281821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c06cef0 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc1affd700 1 -- 172.21.15.156:0/1286281821 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24014440 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.566+0000 7fdc30e6e700 1 -- 172.21.15.156:0/1286281821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c0664c0 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.567+0000 7fdc30e6e700 1 -- 172.21.15.156:0/1286281821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c15ac80 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.568+0000 7fdc1affd700 1 -- 172.21.15.156:0/1286281821 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdc2401b030 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.568+0000 7fdc1affd700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdc0c0682f0 0x7fdc0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:18.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.568+0000 7fdc1affd700 1 -- 172.21.15.156:0/1286281821 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc24085980 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.568+0000 7fdc2a59c700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdc0c0682f0 0x7fdc0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:18.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.568+0000 7fdc2a59c700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdc0c0682f0 0x7fdc0c06a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc20000f60 tx=0x7fdc20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:18.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.572+0000 7fdc1affd700 1 -- 172.21.15.156:0/1286281821 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdc24052160 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.796+0000 7fdc30e6e700 1 -- 172.21.15.156:0/1286281821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc2c146f10 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.796+0000 7fdc1affd700 1 -- 172.21.15.156:0/1286281821 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fdc2c146f10 con 0x7fdc2c0ae9d0 2023-11-28T11:09:18.798 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:18.798 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1085440106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1085440106},{"type":"v1","addr":"172.21.15.186:6834","nonce":1085440106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1548245207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1548245207},{"type":"v1","addr":"172.21.15.156:6836","nonce":1548245207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4882,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3102077957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":3102077957},{"type":"v1","addr":"172.21.15.156:6837","nonce":3102077957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:15.391255+0000","modified":"2023-11-28T11:09:18.414744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3897746301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3897746301},{"type":"v1","addr":"172.21.15.186:6835","nonce":3897746301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:09:18.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.798+0000 7fdc30e6e700 1 -- 172.21.15.156:0/1286281821 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdc0c0682f0 msgr2=0x7fdc0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.798+0000 7fdc30e6e700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdc0c0682f0 0x7fdc0c06a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc20000f60 tx=0x7fdc20009040 comp rx=0 tx=0).stop 2023-11-28T11:09:18.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.798+0000 7fdc30e6e700 1 -- 172.21.15.156:0/1286281821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2c0ae9d0 msgr2=0x7fdc2c1e52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.798+0000 7fdc30e6e700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2c0ae9d0 0x7fdc2c1e52b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b9a0 tx=0x7fdc2400bd60 comp rx=0 tx=0).stop 2023-11-28T11:09:18.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.798+0000 7fdc30e6e700 1 -- 172.21.15.156:0/1286281821 shutdown_connections 2023-11-28T11:09:18.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.798+0000 7fdc30e6e700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2c0ae9d0 0x7fdc2c1e52b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.798+0000 7fdc30e6e700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c06a5e0 0x7fdc2c0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.798+0000 7fdc30e6e700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdc0c0682f0 0x7fdc0c06a7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.798+0000 7fdc30e6e700 1 --2- 172.21.15.156:0/1286281821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc2c06e500 0x7fdc2c1e4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.798+0000 7fdc30e6e700 1 -- 172.21.15.156:0/1286281821 >> 172.21.15.156:0/1286281821 conn(0x7fdc2c0b12b0 msgr2=0x7fdc2c06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:18.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.799+0000 7fdc30e6e700 1 -- 172.21.15.156:0/1286281821 shutdown_connections 2023-11-28T11:09:18.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.799+0000 7fdc30e6e700 1 -- 172.21.15.156:0/1286281821 wait complete. 2023-11-28T11:09:18.802 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T11:09:18.812 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:18.919 INFO:tasks.ceph.mds.d.smithi186.stderr:2023-11-28T11:09:18.918+0000 7f51bf4af700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:09:18.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.932+0000 7f14557d4700 1 Processor -- start 2023-11-28T11:09:18.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.932+0000 7f14557d4700 1 -- start start 2023-11-28T11:09:18.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f14557d4700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f145006ba90 0x7f14500cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:18.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f14557d4700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f14500ccc40 0x7f14500cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:18.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f14557d4700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14500cf5e0 con 0x7f145015ca50 2023-11-28T11:09:18.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f14557d4700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14500cd470 con 0x7f14500ccc40 2023-11-28T11:09:18.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f14557d4700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14500cecb0 con 0x7f145006ba90 2023-11-28T11:09:18.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f144f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f14500ccc40 0x7f14500cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:18.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f144f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f14500ccc40 0x7f14500cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33782/0 (socket says 172.21.15.156:33782) 2023-11-28T11:09:18.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f144f7fe700 1 -- 172.21.15.156:0/3517643998 learned_addr learned my addr 172.21.15.156:0/3517643998 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:18.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f144e7fc700 1 --2- 172.21.15.156:0/3517643998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f145006ba90 0x7f14500cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:18.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f144f7fe700 1 -- 172.21.15.156:0/3517643998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f145006ba90 msgr2=0x7f14500cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:18.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f144f7fe700 1 --2- 172.21.15.156:0/3517643998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f145006ba90 0x7f14500cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f144f7fe700 1 -- 172.21.15.156:0/3517643998 >> v1:172.21.15.186:6789/0 conn(0x7f145015ca50 legacy=0x7f145006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:18.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f144f7fe700 1 -- 172.21.15.156:0/3517643998 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14500cef90 con 0x7f14500ccc40 2023-11-28T11:09:18.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f144f7fe700 1 --2- 172.21.15.156:0/3517643998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14500ccc40 0x7f14500cfa70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1448009e50 tx=0x7f144800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2f7f4da9e327de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:18.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f144dffb700 1 -- 172.21.15.156:0/3517643998 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144800e070 con 0x7f14500ccc40 2023-11-28T11:09:18.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f144dffb700 1 -- 172.21.15.156:0/3517643998 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1448003a20 con 0x7f14500ccc40 2023-11-28T11:09:18.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.933+0000 7f144dffb700 1 -- 172.21.15.156:0/3517643998 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448003dd0 con 0x7f14500ccc40 2023-11-28T11:09:18.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.934+0000 7f14557d4700 1 -- 172.21.15.156:0/3517643998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14500ccc40 msgr2=0x7f14500cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.934+0000 7f14557d4700 1 --2- 172.21.15.156:0/3517643998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14500ccc40 0x7f14500cfa70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1448009e50 tx=0x7f144800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:18.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.934+0000 7f14557d4700 1 -- 172.21.15.156:0/3517643998 shutdown_connections 2023-11-28T11:09:18.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.934+0000 7f14557d4700 1 --2- 172.21.15.156:0/3517643998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14500ccc40 0x7f14500cfa70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.934+0000 7f14557d4700 1 --2- 172.21.15.156:0/3517643998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f145006ba90 0x7f14500cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.934+0000 7f14557d4700 1 -- 172.21.15.156:0/3517643998 >> 172.21.15.156:0/3517643998 conn(0x7f145009a230 msgr2=0x7f145008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:18.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.934+0000 7f14557d4700 1 -- 172.21.15.156:0/3517643998 shutdown_connections 2023-11-28T11:09:18.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.934+0000 7f14557d4700 1 -- 172.21.15.156:0/3517643998 wait complete. 2023-11-28T11:09:18.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.935+0000 7f14557d4700 1 Processor -- start 2023-11-28T11:09:18.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.935+0000 7f14557d4700 1 -- start start 2023-11-28T11:09:18.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.936+0000 7f14557d4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f145006ba90 0x7f14500ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:18.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.936+0000 7f14557d4700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14500ccc40 0x7f1450152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:18.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.936+0000 7f14557d4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f145015ca50 0x7f14500af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:18.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.936+0000 7f14557d4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145015f500 con 0x7f145006ba90 2023-11-28T11:09:18.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.936+0000 7f14557d4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145015e8f0 con 0x7f145015ca50 2023-11-28T11:09:18.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.936+0000 7f14557d4700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145015ebd0 con 0x7f14500ccc40 2023-11-28T11:09:18.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.936+0000 7f144effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f145006ba90 0x7f14500ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:18.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.936+0000 7f144effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f145006ba90 0x7f14500ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33784/0 (socket says 172.21.15.156:33784) 2023-11-28T11:09:18.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.936+0000 7f144effd700 1 -- 172.21.15.156:0/2758630548 learned_addr learned my addr 172.21.15.156:0/2758630548 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:18.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.937+0000 7f144effd700 1 -- 172.21.15.156:0/2758630548 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14500ccc40 msgr2=0x7f1450152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.937+0000 7f144effd700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14500ccc40 0x7f1450152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.937+0000 7f144effd700 1 -- 172.21.15.156:0/2758630548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f145015ca50 msgr2=0x7f14500af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.937+0000 7f144f7fe700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f145015ca50 0x7f14500af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:18.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.937+0000 7f144effd700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f145015ca50 0x7f14500af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.937+0000 7f144effd700 1 -- 172.21.15.156:0/2758630548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1448009b30 con 0x7f145006ba90 2023-11-28T11:09:18.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.937+0000 7f144effd700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f145006ba90 0x7f14500ae5a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f144400b4a0 tx=0x7f144400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:18.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.937+0000 7f14377fe700 1 -- 172.21.15.156:0/2758630548 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444011830 con 0x7f145006ba90 2023-11-28T11:09:18.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.937+0000 7f14377fe700 1 -- 172.21.15.156:0/2758630548 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1444011990 con 0x7f145006ba90 2023-11-28T11:09:18.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.937+0000 7f14557d4700 1 -- 172.21.15.156:0/2758630548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14500cd0e0 con 0x7f145006ba90 2023-11-28T11:09:18.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.937+0000 7f14377fe700 1 -- 172.21.15.156:0/2758630548 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444011d40 con 0x7f145006ba90 2023-11-28T11:09:18.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.937+0000 7f14557d4700 1 -- 172.21.15.156:0/2758630548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14500cf270 con 0x7f145006ba90 2023-11-28T11:09:18.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.938+0000 7f14557d4700 1 -- 172.21.15.156:0/2758630548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145015ce20 con 0x7f145006ba90 2023-11-28T11:09:18.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.938+0000 7f14377fe700 1 -- 172.21.15.156:0/2758630548 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f144400f050 con 0x7f145006ba90 2023-11-28T11:09:18.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.939+0000 7f14377fe700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f143c0682f0 0x7f143c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:18.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.939+0000 7f144e7fc700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f143c0682f0 0x7f143c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:18.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.939+0000 7f14377fe700 1 -- 172.21.15.156:0/2758630548 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1444085940 con 0x7f145006ba90 2023-11-28T11:09:18.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.939+0000 7f144e7fc700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f143c0682f0 0x7f143c06a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f145006b760 tx=0x7f1440008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:18.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:18.943+0000 7f14377fe700 1 -- 172.21.15.156:0/2758630548 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1444052110 con 0x7f145006ba90 2023-11-28T11:09:19.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.131+0000 7f14557d4700 1 -- 172.21.15.156:0/2758630548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14500745c0 con 0x7f145006ba90 2023-11-28T11:09:19.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.132+0000 7f14377fe700 1 -- 172.21.15.156:0/2758630548 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f1444015020 con 0x7f145006ba90 2023-11-28T11:09:19.133 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:19.134 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":37,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:16.393008+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:13.663041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:14.347248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.134+0000 7f14557d4700 1 -- 172.21.15.156:0/2758630548 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f143c0682f0 msgr2=0x7f143c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.134+0000 7f14557d4700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f143c0682f0 0x7f143c06a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f145006b760 tx=0x7f1440008040 comp rx=0 tx=0).stop 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.134+0000 7f14557d4700 1 -- 172.21.15.156:0/2758630548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f145006ba90 msgr2=0x7f14500ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.134+0000 7f14557d4700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f145006ba90 0x7f14500ae5a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f144400b4a0 tx=0x7f144400b860 comp rx=0 tx=0).stop 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.134+0000 7f14557d4700 1 -- 172.21.15.156:0/2758630548 shutdown_connections 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.134+0000 7f14557d4700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f145015ca50 0x7f14500af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.134+0000 7f14557d4700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f145006ba90 0x7f14500ae5a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.134+0000 7f14557d4700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f143c0682f0 0x7f143c06a7a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.134+0000 7f14557d4700 1 --2- 172.21.15.156:0/2758630548 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14500ccc40 0x7f1450152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.134+0000 7f14557d4700 1 -- 172.21.15.156:0/2758630548 >> 172.21.15.156:0/2758630548 conn(0x7f145009a230 msgr2=0x7f14500d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:19.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.134+0000 7f14557d4700 1 -- 172.21.15.156:0/2758630548 shutdown_connections 2023-11-28T11:09:19.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.134+0000 7f14557d4700 1 -- 172.21.15.156:0/2758630548 wait complete. 2023-11-28T11:09:19.148 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:09:19.149 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:09:19.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.267+0000 7fa0397af700 1 Processor -- start 2023-11-28T11:09:19.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.267+0000 7fa0397af700 1 -- start start 2023-11-28T11:09:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.268+0000 7fa0397af700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa03406ba90 0x7fa0340af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.268+0000 7fa0397af700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa0340cbdf0 0x7fa0340ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.268+0000 7fa0397af700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0340cc5f0 con 0x7fa03415cef0 2023-11-28T11:09:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.268+0000 7fa0397af700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa03415d470 con 0x7fa0340cbdf0 2023-11-28T11:09:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.268+0000 7fa0397af700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03415d750 con 0x7fa03406ba90 2023-11-28T11:09:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.268+0000 7fa032ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa03415cef0 0x7fa03406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48668/0 (socket says 172.21.15.156:48668) 2023-11-28T11:09:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.268+0000 7fa032ffd700 1 -- 172.21.15.156:0/257706580 learned_addr learned my addr 172.21.15.156:0/257706580 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa0337fe700 1 --2- 172.21.15.156:0/257706580 >> v2:172.21.15.186:3300/0 conn(0x7fa0340cbdf0 0x7fa0340ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa0327fc700 1 --2- 172.21.15.156:0/257706580 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa03406ba90 0x7fa0340af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa031ffb700 1 -- 172.21.15.156:0/257706580 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499722564 0 0) 0x7fa0340cc5f0 con 0x7fa03415cef0 2023-11-28T11:09:19.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa031ffb700 1 -- 172.21.15.156:0/257706580 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa01c0057c0 con 0x7fa03415cef0 2023-11-28T11:09:19.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa0327fc700 1 -- 172.21.15.156:0/257706580 >> v2:172.21.15.186:3300/0 conn(0x7fa0340cbdf0 msgr2=0x7fa0340ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:19.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa0327fc700 1 --2- 172.21.15.156:0/257706580 >> v2:172.21.15.186:3300/0 conn(0x7fa0340cbdf0 0x7fa0340ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa0327fc700 1 -- 172.21.15.156:0/257706580 >> v1:172.21.15.156:6789/0 conn(0x7fa03415cef0 legacy=0x7fa03406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa0327fc700 1 -- 172.21.15.156:0/257706580 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03415da30 con 0x7fa03406ba90 2023-11-28T11:09:19.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa0337fe700 1 --2- 172.21.15.156:0/257706580 >> v2:172.21.15.186:3300/0 conn(0x7fa0340cbdf0 0x7fa0340ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:19.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa0327fc700 1 --2- 172.21.15.156:0/257706580 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa03406ba90 0x7fa0340af400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa024009840 tx=0x7fa024009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a209c3a1d612ffe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:19.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa031ffb700 1 -- 172.21.15.156:0/257706580 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024011830 con 0x7fa03406ba90 2023-11-28T11:09:19.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa031ffb700 1 -- 172.21.15.156:0/257706580 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa024011990 con 0x7fa03406ba90 2023-11-28T11:09:19.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.269+0000 7fa031ffb700 1 -- 172.21.15.156:0/257706580 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024011d20 con 0x7fa03406ba90 2023-11-28T11:09:19.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.270+0000 7fa0397af700 1 -- 172.21.15.156:0/257706580 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa03406ba90 msgr2=0x7fa0340af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.270+0000 7fa0397af700 1 --2- 172.21.15.156:0/257706580 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa03406ba90 0x7fa0340af400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa024009840 tx=0x7fa024009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:19.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.270+0000 7fa0397af700 1 -- 172.21.15.156:0/257706580 shutdown_connections 2023-11-28T11:09:19.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.270+0000 7fa0397af700 1 --2- 172.21.15.156:0/257706580 >> v2:172.21.15.186:3300/0 conn(0x7fa0340cbdf0 0x7fa0340ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.270+0000 7fa0397af700 1 --2- 172.21.15.156:0/257706580 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa03406ba90 0x7fa0340af400 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.270+0000 7fa0397af700 1 -- 172.21.15.156:0/257706580 >> 172.21.15.156:0/257706580 conn(0x7fa0340d0000 msgr2=0x7fa0340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.270+0000 7fa0397af700 1 -- 172.21.15.156:0/257706580 shutdown_connections 2023-11-28T11:09:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.270+0000 7fa0397af700 1 -- 172.21.15.156:0/257706580 wait complete. 2023-11-28T11:09:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.271+0000 7fa0397af700 1 Processor -- start 2023-11-28T11:09:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.271+0000 7fa0397af700 1 -- start start 2023-11-28T11:09:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0397af700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03406ba90 0x7fa034156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0397af700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0340cbdf0 0x7fa034157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0397af700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa03415cef0 0x7fa0340b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0397af700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0340b5c10 con 0x7fa03415cef0 2023-11-28T11:09:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0397af700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa034156fe0 con 0x7fa03406ba90 2023-11-28T11:09:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0397af700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0341572c0 con 0x7fa0340cbdf0 2023-11-28T11:09:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0337fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa03415cef0 0x7fa0340b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0337fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa03415cef0 0x7fa0340b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33800/0 (socket says 172.21.15.156:33800) 2023-11-28T11:09:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0337fe700 1 -- 172.21.15.156:0/3558541011 learned_addr learned my addr 172.21.15.156:0/3558541011 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa032ffd700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03406ba90 0x7fa034156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0327fc700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0340cbdf0 0x7fa034157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0337fe700 1 -- 172.21.15.156:0/3558541011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0340cbdf0 msgr2=0x7fa034157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0337fe700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0340cbdf0 0x7fa034157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0337fe700 1 -- 172.21.15.156:0/3558541011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03406ba90 msgr2=0x7fa034156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0337fe700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03406ba90 0x7fa034156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.272+0000 7fa0337fe700 1 -- 172.21.15.156:0/3558541011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02c009a70 con 0x7fa03415cef0 2023-11-28T11:09:19.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.273+0000 7fa0337fe700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa03415cef0 0x7fa0340b51a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00efc0 tx=0x7fa02c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:19.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.273+0000 7fa01b7fe700 1 -- 172.21.15.156:0/3558541011 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c00ca90 con 0x7fa03415cef0 2023-11-28T11:09:19.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.273+0000 7fa01b7fe700 1 -- 172.21.15.156:0/3558541011 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02c00f040 con 0x7fa03415cef0 2023-11-28T11:09:19.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.273+0000 7fa0397af700 1 -- 172.21.15.156:0/3558541011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0240094a0 con 0x7fa03415cef0 2023-11-28T11:09:19.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.273+0000 7fa01b7fe700 1 -- 172.21.15.156:0/3558541011 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c00ce10 con 0x7fa03415cef0 2023-11-28T11:09:19.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.273+0000 7fa0397af700 1 -- 172.21.15.156:0/3558541011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03415dff0 con 0x7fa03415cef0 2023-11-28T11:09:19.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.274+0000 7fa0397af700 1 -- 172.21.15.156:0/3558541011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03415dd10 con 0x7fa03415cef0 2023-11-28T11:09:19.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.274+0000 7fa01b7fe700 1 -- 172.21.15.156:0/3558541011 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa02c014480 con 0x7fa03415cef0 2023-11-28T11:09:19.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.274+0000 7fa01b7fe700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa0200682f0 0x7fa02006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.275+0000 7fa01b7fe700 1 -- 172.21.15.156:0/3558541011 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa02c0864f0 con 0x7fa03415cef0 2023-11-28T11:09:19.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.275+0000 7fa032ffd700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa0200682f0 0x7fa02006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.275+0000 7fa032ffd700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa0200682f0 0x7fa02006a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa028007fd0 tx=0x7fa028007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:19.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.279+0000 7fa01b7fe700 1 -- 172.21.15.156:0/3558541011 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa02c052c70 con 0x7fa03415cef0 2023-11-28T11:09:19.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.459+0000 7fa0397af700 1 -- 172.21.15.156:0/3558541011 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa034157150 con 0x7fa0200682f0 2023-11-28T11:09:19.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.468+0000 7fa01b7fe700 1 -- 172.21.15.156:0/3558541011 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311600 (secure 0 0 0) 0x7fa034157150 con 0x7fa0200682f0 2023-11-28T11:09:19.473 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:19.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.471+0000 7fa0397af700 1 -- 172.21.15.156:0/3558541011 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa0200682f0 msgr2=0x7fa02006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.471+0000 7fa0397af700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa0200682f0 0x7fa02006a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa028007fd0 tx=0x7fa028007dc0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.471+0000 7fa0397af700 1 -- 172.21.15.156:0/3558541011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa03415cef0 msgr2=0x7fa0340b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.471+0000 7fa0397af700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa03415cef0 0x7fa0340b51a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00efc0 tx=0x7fa02c00c3a0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.471+0000 7fa0397af700 1 -- 172.21.15.156:0/3558541011 shutdown_connections 2023-11-28T11:09:19.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.471+0000 7fa0397af700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa03415cef0 0x7fa0340b51a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.471+0000 7fa0397af700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03406ba90 0x7fa034156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.471+0000 7fa0397af700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa0200682f0 0x7fa02006a7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.471+0000 7fa0397af700 1 --2- 172.21.15.156:0/3558541011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa0340cbdf0 0x7fa034157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.471+0000 7fa0397af700 1 -- 172.21.15.156:0/3558541011 >> 172.21.15.156:0/3558541011 conn(0x7fa0340d0000 msgr2=0x7fa03406e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:19.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.471+0000 7fa0397af700 1 -- 172.21.15.156:0/3558541011 shutdown_connections 2023-11-28T11:09:19.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.471+0000 7fa0397af700 1 -- 172.21.15.156:0/3558541011 wait complete. 2023-11-28T11:09:19.477 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:09:19.485 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2023-11-28T11:09:18.538194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":69896,"kb_used_data":4424,"kb_used_omap":0,"kb_used_meta":65408,"kb_avail":754904824,"statfs":{"total":773094113280,"available":773022539776,"internally_reserved":0,"allocated":4530176,"data_stored":2535276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66977792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.548800"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396608+0000","last_change":"2023-11-28T11:09:15.401196+0000","last_active":"2023-11-28T11:09:16.396608+0000","last_peered":"2023-11-28T11:09:16.396608+0000","last_clean":"2023-11-28T11:09:16.396608+0000","last_became_active":"2023-11-28T11:09:15.400945+0000","last_became_peered":"2023-11-28T11:09:15.400945+0000","last_unstale":"2023-11-28T11:09:16.396608+0000","last_undegraded":"2023-11-28T11:09:16.396608+0000","last_fullsized":"2023-11-28T11:09:16.396608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:22.625942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396148+0000","last_change":"2023-11-28T11:09:15.409873+0000","last_active":"2023-11-28T11:09:16.396148+0000","last_peered":"2023-11-28T11:09:16.396148+0000","last_clean":"2023-11-28T11:09:16.396148+0000","last_became_active":"2023-11-28T11:09:15.409593+0000","last_became_peered":"2023-11-28T11:09:15.409593+0000","last_unstale":"2023-11-28T11:09:16.396148+0000","last_undegraded":"2023-11-28T11:09:16.396148+0000","last_fullsized":"2023-11-28T11:09:16.396148+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:52.365078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.411151+0000","last_change":"2023-11-28T11:09:15.409034+0000","last_active":"2023-11-28T11:09:15.411151+0000","last_peered":"2023-11-28T11:09:15.411151+0000","last_clean":"2023-11-28T11:09:15.411151+0000","last_became_active":"2023-11-28T11:09:15.408740+0000","last_became_peered":"2023-11-28T11:09:15.408740+0000","last_unstale":"2023-11-28T11:09:15.411151+0000","last_undegraded":"2023-11-28T11:09:15.411151+0000","last_fullsized":"2023-11-28T11:09:15.411151+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:52.365078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401234+0000","last_change":"2023-11-28T11:09:15.415786+0000","last_active":"2023-11-28T11:09:16.401234+0000","last_peered":"2023-11-28T11:09:16.401234+0000","last_clean":"2023-11-28T11:09:16.401234+0000","last_became_active":"2023-11-28T11:09:15.415549+0000","last_became_peered":"2023-11-28T11:09:15.415549+0000","last_unstale":"2023-11-28T11:09:16.401234+0000","last_undegraded":"2023-11-28T11:09:16.401234+0000","last_fullsized":"2023-11-28T11:09:16.401234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:44:40.603431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401075+0000","last_change":"2023-11-28T11:09:15.423012+0000","last_active":"2023-11-28T11:09:16.401075+0000","last_peered":"2023-11-28T11:09:16.401075+0000","last_clean":"2023-11-28T11:09:16.401075+0000","last_became_active":"2023-11-28T11:09:15.422745+0000","last_became_peered":"2023-11-28T11:09:15.422745+0000","last_unstale":"2023-11-28T11:09:16.401075+0000","last_undegraded":"2023-11-28T11:09:16.401075+0000","last_fullsized":"2023-11-28T11:09:16.401075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:57.597840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.408155+0000","last_change":"2023-11-28T11:09:15.404365+0000","last_active":"2023-11-28T11:09:15.408155+0000","last_peered":"2023-11-28T11:09:15.408155+0000","last_clean":"2023-11-28T11:09:15.408155+0000","last_became_active":"2023-11-28T11:09:15.404126+0000","last_became_peered":"2023-11-28T11:09:15.404126+0000","last_unstale":"2023-11-28T11:09:15.408155+0000","last_undegraded":"2023-11-28T11:09:15.408155+0000","last_fullsized":"2023-11-28T11:09:15.408155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:40.096463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401325+0000","last_change":"2023-11-28T11:09:15.404108+0000","last_active":"2023-11-28T11:09:16.401325+0000","last_peered":"2023-11-28T11:09:16.401325+0000","last_clean":"2023-11-28T11:09:16.401325+0000","last_became_active":"2023-11-28T11:09:15.403803+0000","last_became_peered":"2023-11-28T11:09:15.403803+0000","last_unstale":"2023-11-28T11:09:16.401325+0000","last_undegraded":"2023-11-28T11:09:16.401325+0000","last_fullsized":"2023-11-28T11:09:16.401325+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:22.107839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.395874+0000","last_change":"2023-11-28T11:09:15.417326+0000","last_active":"2023-11-28T11:09:16.395874+0000","last_peered":"2023-11-28T11:09:16.395874+0000","last_clean":"2023-11-28T11:09:16.395874+0000","last_became_active":"2023-11-28T11:09:15.417081+0000","last_became_peered":"2023-11-28T11:09:15.417081+0000","last_unstale":"2023-11-28T11:09:16.395874+0000","last_undegraded":"2023-11-28T11:09:16.395874+0000","last_fullsized":"2023-11-28T11:09:16.395874+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:22.107839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419254+0000","last_change":"2023-11-28T11:09:15.416962+0000","last_active":"2023-11-28T11:09:15.419254+0000","last_peered":"2023-11-28T11:09:15.419254+0000","last_clean":"2023-11-28T11:09:15.419254+0000","last_became_active":"2023-11-28T11:09:15.416699+0000","last_became_peered":"2023-11-28T11:09:15.416699+0000","last_unstale":"2023-11-28T11:09:15.419254+0000","last_undegraded":"2023-11-28T11:09:15.419254+0000","last_fullsized":"2023-11-28T11:09:15.419254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:22.107839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401887+0000","last_change":"2023-11-28T11:09:15.417258+0000","last_active":"2023-11-28T11:09:16.401887+0000","last_peered":"2023-11-28T11:09:16.401887+0000","last_clean":"2023-11-28T11:09:16.401887+0000","last_became_active":"2023-11-28T11:09:15.416945+0000","last_became_peered":"2023-11-28T11:09:15.416945+0000","last_unstale":"2023-11-28T11:09:16.401887+0000","last_undegraded":"2023-11-28T11:09:16.401887+0000","last_fullsized":"2023-11-28T11:09:16.401887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:26:42.661150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401755+0000","last_change":"2023-11-28T11:09:15.417408+0000","last_active":"2023-11-28T11:09:16.401755+0000","last_peered":"2023-11-28T11:09:16.401755+0000","last_clean":"2023-11-28T11:09:16.401755+0000","last_became_active":"2023-11-28T11:09:15.416914+0000","last_became_peered":"2023-11-28T11:09:15.416914+0000","last_unstale":"2023-11-28T11:09:16.401755+0000","last_undegraded":"2023-11-28T11:09:16.401755+0000","last_fullsized":"2023-11-28T11:09:16.401755+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:01.481847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.413858+0000","last_change":"2023-11-28T11:09:15.410540+0000","last_active":"2023-11-28T11:09:15.413858+0000","last_peered":"2023-11-28T11:09:15.413858+0000","last_clean":"2023-11-28T11:09:15.413858+0000","last_became_active":"2023-11-28T11:09:15.410281+0000","last_became_peered":"2023-11-28T11:09:15.410281+0000","last_unstale":"2023-11-28T11:09:15.413858+0000","last_undegraded":"2023-11-28T11:09:15.413858+0000","last_fullsized":"2023-11-28T11:09:15.413858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:00.185455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401411+0000","last_change":"2023-11-28T11:09:15.417734+0000","last_active":"2023-11-28T11:09:16.401411+0000","last_peered":"2023-11-28T11:09:16.401411+0000","last_clean":"2023-11-28T11:09:16.401411+0000","last_became_active":"2023-11-28T11:09:15.417461+0000","last_became_peered":"2023-11-28T11:09:15.417461+0000","last_unstale":"2023-11-28T11:09:16.401411+0000","last_undegraded":"2023-11-28T11:09:16.401411+0000","last_fullsized":"2023-11-28T11:09:16.401411+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:52.365078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.423899+0000","last_change":"2023-11-28T11:09:15.419261+0000","last_active":"2023-11-28T11:09:15.423899+0000","last_peered":"2023-11-28T11:09:15.423899+0000","last_clean":"2023-11-28T11:09:15.423899+0000","last_became_active":"2023-11-28T11:09:15.418975+0000","last_became_peered":"2023-11-28T11:09:15.418975+0000","last_unstale":"2023-11-28T11:09:15.423899+0000","last_undegraded":"2023-11-28T11:09:15.423899+0000","last_fullsized":"2023-11-28T11:09:15.423899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:17:08.883199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401257+0000","last_change":"2023-11-28T11:09:15.411176+0000","last_active":"2023-11-28T11:09:16.401257+0000","last_peered":"2023-11-28T11:09:16.401257+0000","last_clean":"2023-11-28T11:09:16.401257+0000","last_became_active":"2023-11-28T11:09:15.410928+0000","last_became_peered":"2023-11-28T11:09:15.410928+0000","last_unstale":"2023-11-28T11:09:16.401257+0000","last_undegraded":"2023-11-28T11:09:16.401257+0000","last_fullsized":"2023-11-28T11:09:16.401257+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:00.185455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.426398+0000","last_change":"2023-11-28T11:09:15.424366+0000","last_active":"2023-11-28T11:09:15.426398+0000","last_peered":"2023-11-28T11:09:15.426398+0000","last_clean":"2023-11-28T11:09:15.426398+0000","last_became_active":"2023-11-28T11:09:15.424083+0000","last_became_peered":"2023-11-28T11:09:15.424083+0000","last_unstale":"2023-11-28T11:09:15.426398+0000","last_undegraded":"2023-11-28T11:09:15.426398+0000","last_fullsized":"2023-11-28T11:09:15.426398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:22.107839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.426486+0000","last_change":"2023-11-28T11:09:15.424388+0000","last_active":"2023-11-28T11:09:15.426486+0000","last_peered":"2023-11-28T11:09:15.426486+0000","last_clean":"2023-11-28T11:09:15.426486+0000","last_became_active":"2023-11-28T11:09:15.424031+0000","last_became_peered":"2023-11-28T11:09:15.424031+0000","last_unstale":"2023-11-28T11:09:15.426486+0000","last_undegraded":"2023-11-28T11:09:15.426486+0000","last_fullsized":"2023-11-28T11:09:15.426486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:17.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419519+0000","last_change":"2023-11-28T11:09:15.418609+0000","last_active":"2023-11-28T11:09:15.419519+0000","last_peered":"2023-11-28T11:09:15.419519+0000","last_clean":"2023-11-28T11:09:15.419519+0000","last_became_active":"2023-11-28T11:09:15.418325+0000","last_became_peered":"2023-11-28T11:09:15.418325+0000","last_unstale":"2023-11-28T11:09:15.419519+0000","last_undegraded":"2023-11-28T11:09:15.419519+0000","last_fullsized":"2023-11-28T11:09:15.419519+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:16.573814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401252+0000","last_change":"2023-11-28T11:09:15.408726+0000","last_active":"2023-11-28T11:09:16.401252+0000","last_peered":"2023-11-28T11:09:16.401252+0000","last_clean":"2023-11-28T11:09:16.401252+0000","last_became_active":"2023-11-28T11:09:15.408476+0000","last_became_peered":"2023-11-28T11:09:15.408476+0000","last_unstale":"2023-11-28T11:09:16.401252+0000","last_undegraded":"2023-11-28T11:09:16.401252+0000","last_fullsized":"2023-11-28T11:09:16.401252+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:00.535131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.418609+0000","last_change":"2023-11-28T11:09:15.413840+0000","last_active":"2023-11-28T11:09:15.418609+0000","last_peered":"2023-11-28T11:09:15.418609+0000","last_clean":"2023-11-28T11:09:15.418609+0000","last_became_active":"2023-11-28T11:09:15.413528+0000","last_became_peered":"2023-11-28T11:09:15.413528+0000","last_unstale":"2023-11-28T11:09:15.418609+0000","last_undegraded":"2023-11-28T11:09:15.418609+0000","last_fullsized":"2023-11-28T11:09:15.418609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:41.903553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401538+0000","last_change":"2023-11-28T11:09:15.407084+0000","last_active":"2023-11-28T11:09:16.401538+0000","last_peered":"2023-11-28T11:09:16.401538+0000","last_clean":"2023-11-28T11:09:16.401538+0000","last_became_active":"2023-11-28T11:09:15.406790+0000","last_became_peered":"2023-11-28T11:09:15.406790+0000","last_unstale":"2023-11-28T11:09:16.401538+0000","last_undegraded":"2023-11-28T11:09:16.401538+0000","last_fullsized":"2023-11-28T11:09:16.401538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:22.625942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.420417+0000","last_change":"2023-11-28T11:09:15.412311+0000","last_active":"2023-11-28T11:09:15.420417+0000","last_peered":"2023-11-28T11:09:15.420417+0000","last_clean":"2023-11-28T11:09:15.420417+0000","last_became_active":"2023-11-28T11:09:15.411968+0000","last_became_peered":"2023-11-28T11:09:15.411968+0000","last_unstale":"2023-11-28T11:09:15.420417+0000","last_undegraded":"2023-11-28T11:09:15.420417+0000","last_fullsized":"2023-11-28T11:09:15.420417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:58.303509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419139+0000","last_change":"2023-11-28T11:09:15.410052+0000","last_active":"2023-11-28T11:09:15.419139+0000","last_peered":"2023-11-28T11:09:15.419139+0000","last_clean":"2023-11-28T11:09:15.419139+0000","last_became_active":"2023-11-28T11:09:15.409756+0000","last_became_peered":"2023-11-28T11:09:15.409756+0000","last_unstale":"2023-11-28T11:09:15.419139+0000","last_undegraded":"2023-11-28T11:09:15.419139+0000","last_fullsized":"2023-11-28T11:09:15.419139+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:15.621788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.400987+0000","last_change":"2023-11-28T11:09:15.412130+0000","last_active":"2023-11-28T11:09:16.400987+0000","last_peered":"2023-11-28T11:09:16.400987+0000","last_clean":"2023-11-28T11:09:16.400987+0000","last_became_active":"2023-11-28T11:09:15.411857+0000","last_became_peered":"2023-11-28T11:09:15.411857+0000","last_unstale":"2023-11-28T11:09:16.400987+0000","last_undegraded":"2023-11-28T11:09:16.400987+0000","last_fullsized":"2023-11-28T11:09:16.400987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:40:23.343099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419066+0000","last_change":"2023-11-28T11:09:15.415566+0000","last_active":"2023-11-28T11:09:15.419066+0000","last_peered":"2023-11-28T11:09:15.419066+0000","last_clean":"2023-11-28T11:09:15.419066+0000","last_became_active":"2023-11-28T11:09:15.415238+0000","last_became_peered":"2023-11-28T11:09:15.415238+0000","last_unstale":"2023-11-28T11:09:15.419066+0000","last_undegraded":"2023-11-28T11:09:15.419066+0000","last_fullsized":"2023-11-28T11:09:15.419066+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:57.597840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419139+0000","last_change":"2023-11-28T11:09:15.417534+0000","last_active":"2023-11-28T11:09:15.419139+0000","last_peered":"2023-11-28T11:09:15.419139+0000","last_clean":"2023-11-28T11:09:15.419139+0000","last_became_active":"2023-11-28T11:09:15.417283+0000","last_became_peered":"2023-11-28T11:09:15.417283+0000","last_unstale":"2023-11-28T11:09:15.419139+0000","last_undegraded":"2023-11-28T11:09:15.419139+0000","last_fullsized":"2023-11-28T11:09:15.419139+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:45.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.418593+0000","last_change":"2023-11-28T11:09:15.413667+0000","last_active":"2023-11-28T11:09:15.418593+0000","last_peered":"2023-11-28T11:09:15.418593+0000","last_clean":"2023-11-28T11:09:15.418593+0000","last_became_active":"2023-11-28T11:09:15.413381+0000","last_became_peered":"2023-11-28T11:09:15.413381+0000","last_unstale":"2023-11-28T11:09:15.418593+0000","last_undegraded":"2023-11-28T11:09:15.418593+0000","last_fullsized":"2023-11-28T11:09:15.418593+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:21.992851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.402053+0000","last_change":"2023-11-28T11:09:15.410784+0000","last_active":"2023-11-28T11:09:16.402053+0000","last_peered":"2023-11-28T11:09:16.402053+0000","last_clean":"2023-11-28T11:09:16.402053+0000","last_became_active":"2023-11-28T11:09:15.409919+0000","last_became_peered":"2023-11-28T11:09:15.409919+0000","last_unstale":"2023-11-28T11:09:16.402053+0000","last_undegraded":"2023-11-28T11:09:16.402053+0000","last_fullsized":"2023-11-28T11:09:16.402053+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:17.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.418645+0000","last_change":"2023-11-28T11:09:15.413244+0000","last_active":"2023-11-28T11:09:15.418645+0000","last_peered":"2023-11-28T11:09:15.418645+0000","last_clean":"2023-11-28T11:09:15.418645+0000","last_became_active":"2023-11-28T11:09:15.412870+0000","last_became_peered":"2023-11-28T11:09:15.412870+0000","last_unstale":"2023-11-28T11:09:15.418645+0000","last_undegraded":"2023-11-28T11:09:15.418645+0000","last_fullsized":"2023-11-28T11:09:15.418645+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:25.569113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.418618+0000","last_change":"2023-11-28T11:09:15.413865+0000","last_active":"2023-11-28T11:09:15.418618+0000","last_peered":"2023-11-28T11:09:15.418618+0000","last_clean":"2023-11-28T11:09:15.418618+0000","last_became_active":"2023-11-28T11:09:15.413588+0000","last_became_peered":"2023-11-28T11:09:15.413588+0000","last_unstale":"2023-11-28T11:09:15.418618+0000","last_undegraded":"2023-11-28T11:09:15.418618+0000","last_fullsized":"2023-11-28T11:09:15.418618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:44.329643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419160+0000","last_change":"2023-11-28T11:09:15.410548+0000","last_active":"2023-11-28T11:09:15.419160+0000","last_peered":"2023-11-28T11:09:15.419160+0000","last_clean":"2023-11-28T11:09:15.419160+0000","last_became_active":"2023-11-28T11:09:15.410271+0000","last_became_peered":"2023-11-28T11:09:15.410271+0000","last_unstale":"2023-11-28T11:09:15.419160+0000","last_undegraded":"2023-11-28T11:09:15.419160+0000","last_fullsized":"2023-11-28T11:09:15.419160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:00.185455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.413944+0000","last_change":"2023-11-28T11:09:15.410581+0000","last_active":"2023-11-28T11:09:15.413944+0000","last_peered":"2023-11-28T11:09:15.413944+0000","last_clean":"2023-11-28T11:09:15.413944+0000","last_became_active":"2023-11-28T11:09:15.410318+0000","last_became_peered":"2023-11-28T11:09:15.410318+0000","last_unstale":"2023-11-28T11:09:15.413944+0000","last_undegraded":"2023-11-28T11:09:15.413944+0000","last_fullsized":"2023-11-28T11:09:15.413944+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:29:57.451363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.426506+0000","last_change":"2023-11-28T11:09:15.424426+0000","last_active":"2023-11-28T11:09:15.426506+0000","last_peered":"2023-11-28T11:09:15.426506+0000","last_clean":"2023-11-28T11:09:15.426506+0000","last_became_active":"2023-11-28T11:09:15.424132+0000","last_became_peered":"2023-11-28T11:09:15.424132+0000","last_unstale":"2023-11-28T11:09:15.426506+0000","last_undegraded":"2023-11-28T11:09:15.426506+0000","last_fullsized":"2023-11-28T11:09:15.426506+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:52.365078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.427696+0000","last_change":"2023-11-28T11:09:15.009274+0000","last_active":"2023-11-28T11:09:15.427696+0000","last_peered":"2023-11-28T11:09:15.427696+0000","last_clean":"2023-11-28T11:09:15.427696+0000","last_became_active":"2023-11-28T11:09:15.008992+0000","last_became_peered":"2023-11-28T11:09:15.008992+0000","last_unstale":"2023-11-28T11:09:15.427696+0000","last_undegraded":"2023-11-28T11:09:15.427696+0000","last_fullsized":"2023-11-28T11:09:15.427696+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:59:24.742836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419738+0000","last_change":"2023-11-28T11:09:15.419090+0000","last_active":"2023-11-28T11:09:15.419738+0000","last_peered":"2023-11-28T11:09:15.419738+0000","last_clean":"2023-11-28T11:09:15.419738+0000","last_became_active":"2023-11-28T11:09:15.418036+0000","last_became_peered":"2023-11-28T11:09:15.418036+0000","last_unstale":"2023-11-28T11:09:15.419738+0000","last_undegraded":"2023-11-28T11:09:15.419738+0000","last_fullsized":"2023-11-28T11:09:15.419738+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:15.621788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.410212+0000","last_change":"2023-11-28T11:09:15.015521+0000","last_active":"2023-11-28T11:09:15.410212+0000","last_peered":"2023-11-28T11:09:15.410212+0000","last_clean":"2023-11-28T11:09:15.410212+0000","last_became_active":"2023-11-28T11:09:15.015247+0000","last_became_peered":"2023-11-28T11:09:15.015247+0000","last_unstale":"2023-11-28T11:09:15.410212+0000","last_undegraded":"2023-11-28T11:09:15.410212+0000","last_fullsized":"2023-11-28T11:09:15.410212+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:56.596199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.427179+0000","last_change":"2023-11-28T11:09:15.424585+0000","last_active":"2023-11-28T11:09:15.427179+0000","last_peered":"2023-11-28T11:09:15.427179+0000","last_clean":"2023-11-28T11:09:15.427179+0000","last_became_active":"2023-11-28T11:09:15.424315+0000","last_became_peered":"2023-11-28T11:09:15.424315+0000","last_unstale":"2023-11-28T11:09:15.427179+0000","last_undegraded":"2023-11-28T11:09:15.427179+0000","last_fullsized":"2023-11-28T11:09:15.427179+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:46:03.407983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.423406+0000","last_change":"2023-11-28T11:09:15.013936+0000","last_active":"2023-11-28T11:09:15.423406+0000","last_peered":"2023-11-28T11:09:15.423406+0000","last_clean":"2023-11-28T11:09:15.423406+0000","last_became_active":"2023-11-28T11:09:15.013533+0000","last_became_peered":"2023-11-28T11:09:15.013533+0000","last_unstale":"2023-11-28T11:09:15.423406+0000","last_undegraded":"2023-11-28T11:09:15.423406+0000","last_fullsized":"2023-11-28T11:09:15.423406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:24.567472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396626+0000","last_change":"2023-11-28T11:09:15.416355+0000","last_active":"2023-11-28T11:09:16.396626+0000","last_peered":"2023-11-28T11:09:16.396626+0000","last_clean":"2023-11-28T11:09:16.396626+0000","last_became_active":"2023-11-28T11:09:15.416076+0000","last_became_peered":"2023-11-28T11:09:15.416076+0000","last_unstale":"2023-11-28T11:09:16.396626+0000","last_undegraded":"2023-11-28T11:09:16.396626+0000","last_fullsized":"2023-11-28T11:09:16.396626+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:01.481847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.424141+0000","last_change":"2023-11-28T11:09:15.010421+0000","last_active":"2023-11-28T11:09:15.424141+0000","last_peered":"2023-11-28T11:09:15.424141+0000","last_clean":"2023-11-28T11:09:15.424141+0000","last_became_active":"2023-11-28T11:09:15.010125+0000","last_became_peered":"2023-11-28T11:09:15.010125+0000","last_unstale":"2023-11-28T11:09:15.424141+0000","last_undegraded":"2023-11-28T11:09:15.424141+0000","last_fullsized":"2023-11-28T11:09:15.424141+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:39.094822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.405913+0000","last_change":"2023-11-28T11:09:15.403018+0000","last_active":"2023-11-28T11:09:15.405913+0000","last_peered":"2023-11-28T11:09:15.405913+0000","last_clean":"2023-11-28T11:09:15.405913+0000","last_became_active":"2023-11-28T11:09:15.402753+0000","last_became_peered":"2023-11-28T11:09:15.402753+0000","last_unstale":"2023-11-28T11:09:15.405913+0000","last_undegraded":"2023-11-28T11:09:15.405913+0000","last_fullsized":"2023-11-28T11:09:15.405913+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:17.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401755+0000","last_change":"2023-11-28T11:09:15.016338+0000","last_active":"2023-11-28T11:09:16.401755+0000","last_peered":"2023-11-28T11:09:16.401755+0000","last_clean":"2023-11-28T11:09:16.401755+0000","last_became_active":"2023-11-28T11:09:15.016036+0000","last_became_peered":"2023-11-28T11:09:15.016036+0000","last_unstale":"2023-11-28T11:09:16.401755+0000","last_undegraded":"2023-11-28T11:09:16.401755+0000","last_fullsized":"2023-11-28T11:09:16.401755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:56.596199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401843+0000","last_change":"2023-11-28T11:09:15.404047+0000","last_active":"2023-11-28T11:09:16.401843+0000","last_peered":"2023-11-28T11:09:16.401843+0000","last_clean":"2023-11-28T11:09:16.401843+0000","last_became_active":"2023-11-28T11:09:15.403721+0000","last_became_peered":"2023-11-28T11:09:15.403721+0000","last_unstale":"2023-11-28T11:09:16.401843+0000","last_undegraded":"2023-11-28T11:09:16.401843+0000","last_fullsized":"2023-11-28T11:09:16.401843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:03.492691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.422752+0000","last_change":"2023-11-28T11:09:15.014947+0000","last_active":"2023-11-28T11:09:15.422752+0000","last_peered":"2023-11-28T11:09:15.422752+0000","last_clean":"2023-11-28T11:09:15.422752+0000","last_became_active":"2023-11-28T11:09:15.014635+0000","last_became_peered":"2023-11-28T11:09:15.014635+0000","last_unstale":"2023-11-28T11:09:15.422752+0000","last_undegraded":"2023-11-28T11:09:15.422752+0000","last_fullsized":"2023-11-28T11:09:15.422752+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:40.901913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.425689+0000","last_change":"2023-11-28T11:09:15.422433+0000","last_active":"2023-11-28T11:09:15.425689+0000","last_peered":"2023-11-28T11:09:15.425689+0000","last_clean":"2023-11-28T11:09:15.425689+0000","last_became_active":"2023-11-28T11:09:15.422028+0000","last_became_peered":"2023-11-28T11:09:15.422028+0000","last_unstale":"2023-11-28T11:09:15.425689+0000","last_undegraded":"2023-11-28T11:09:15.425689+0000","last_fullsized":"2023-11-28T11:09:15.425689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:16:12.091426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.400427+0000","last_change":"2023-11-28T11:09:15.014440+0000","last_active":"2023-11-28T11:09:15.400427+0000","last_peered":"2023-11-28T11:09:15.400427+0000","last_clean":"2023-11-28T11:09:15.400427+0000","last_became_active":"2023-11-28T11:09:15.014141+0000","last_became_peered":"2023-11-28T11:09:15.014141+0000","last_unstale":"2023-11-28T11:09:15.400427+0000","last_undegraded":"2023-11-28T11:09:15.400427+0000","last_fullsized":"2023-11-28T11:09:15.400427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:15.572173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396196+0000","last_change":"2023-11-28T11:09:15.412690+0000","last_active":"2023-11-28T11:09:16.396196+0000","last_peered":"2023-11-28T11:09:16.396196+0000","last_clean":"2023-11-28T11:09:16.396196+0000","last_became_active":"2023-11-28T11:09:15.412439+0000","last_became_peered":"2023-11-28T11:09:15.412439+0000","last_unstale":"2023-11-28T11:09:16.396196+0000","last_undegraded":"2023-11-28T11:09:16.396196+0000","last_fullsized":"2023-11-28T11:09:16.396196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:36.493873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.405540+0000","last_change":"2023-11-28T11:09:15.009687+0000","last_active":"2023-11-28T11:09:15.405540+0000","last_peered":"2023-11-28T11:09:15.405540+0000","last_clean":"2023-11-28T11:09:15.405540+0000","last_became_active":"2023-11-28T11:09:15.009406+0000","last_became_peered":"2023-11-28T11:09:15.009406+0000","last_unstale":"2023-11-28T11:09:15.405540+0000","last_undegraded":"2023-11-28T11:09:15.405540+0000","last_fullsized":"2023-11-28T11:09:15.405540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:30:48.725285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401257+0000","last_change":"2023-11-28T11:09:15.415648+0000","last_active":"2023-11-28T11:09:16.401257+0000","last_peered":"2023-11-28T11:09:16.401257+0000","last_clean":"2023-11-28T11:09:16.401257+0000","last_became_active":"2023-11-28T11:09:15.415385+0000","last_became_peered":"2023-11-28T11:09:15.415385+0000","last_unstale":"2023-11-28T11:09:16.401257+0000","last_undegraded":"2023-11-28T11:09:16.401257+0000","last_fullsized":"2023-11-28T11:09:16.401257+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:44.329643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396654+0000","last_change":"2023-11-28T11:09:15.018782+0000","last_active":"2023-11-28T11:09:16.396654+0000","last_peered":"2023-11-28T11:09:16.396654+0000","last_clean":"2023-11-28T11:09:16.396654+0000","last_became_active":"2023-11-28T11:09:15.018519+0000","last_became_peered":"2023-11-28T11:09:15.018519+0000","last_unstale":"2023-11-28T11:09:16.396654+0000","last_undegraded":"2023-11-28T11:09:16.396654+0000","last_fullsized":"2023-11-28T11:09:16.396654+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:44:39.601790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.420177+0000","last_change":"2023-11-28T11:09:15.416595+0000","last_active":"2023-11-28T11:09:15.420177+0000","last_peered":"2023-11-28T11:09:15.420177+0000","last_clean":"2023-11-28T11:09:15.420177+0000","last_became_active":"2023-11-28T11:09:15.416286+0000","last_became_peered":"2023-11-28T11:09:15.416286+0000","last_unstale":"2023-11-28T11:09:15.420177+0000","last_undegraded":"2023-11-28T11:09:15.420177+0000","last_fullsized":"2023-11-28T11:09:15.420177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:40.096463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401942+0000","last_change":"2023-11-28T11:09:15.011308+0000","last_active":"2023-11-28T11:09:16.401942+0000","last_peered":"2023-11-28T11:09:16.401942+0000","last_clean":"2023-11-28T11:09:16.401942+0000","last_became_active":"2023-11-28T11:09:15.010987+0000","last_became_peered":"2023-11-28T11:09:15.010987+0000","last_unstale":"2023-11-28T11:09:16.401942+0000","last_undegraded":"2023-11-28T11:09:16.401942+0000","last_fullsized":"2023-11-28T11:09:16.401942+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:43.328002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.414451+0000","last_change":"2023-11-28T11:09:15.411073+0000","last_active":"2023-11-28T11:09:15.414451+0000","last_peered":"2023-11-28T11:09:15.414451+0000","last_clean":"2023-11-28T11:09:15.414451+0000","last_became_active":"2023-11-28T11:09:15.410117+0000","last_became_peered":"2023-11-28T11:09:15.410117+0000","last_unstale":"2023-11-28T11:09:15.414451+0000","last_undegraded":"2023-11-28T11:09:15.414451+0000","last_fullsized":"2023-11-28T11:09:15.414451+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:15.621788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401890+0000","last_change":"2023-11-28T11:09:15.015564+0000","last_active":"2023-11-28T11:09:16.401890+0000","last_peered":"2023-11-28T11:09:16.401890+0000","last_clean":"2023-11-28T11:09:16.401890+0000","last_became_active":"2023-11-28T11:09:15.015293+0000","last_became_peered":"2023-11-28T11:09:15.015293+0000","last_unstale":"2023-11-28T11:09:16.401890+0000","last_undegraded":"2023-11-28T11:09:16.401890+0000","last_fullsized":"2023-11-28T11:09:16.401890+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:08.798600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.424649+0000","last_change":"2023-11-28T11:09:15.420136+0000","last_active":"2023-11-28T11:09:15.424649+0000","last_peered":"2023-11-28T11:09:15.424649+0000","last_clean":"2023-11-28T11:09:15.424649+0000","last_became_active":"2023-11-28T11:09:15.419884+0000","last_became_peered":"2023-11-28T11:09:15.419884+0000","last_unstale":"2023-11-28T11:09:15.424649+0000","last_undegraded":"2023-11-28T11:09:15.424649+0000","last_fullsized":"2023-11-28T11:09:15.424649+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:03.503649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396230+0000","last_change":"2023-11-28T11:09:15.014485+0000","last_active":"2023-11-28T11:09:16.396230+0000","last_peered":"2023-11-28T11:09:16.396230+0000","last_clean":"2023-11-28T11:09:16.396230+0000","last_became_active":"2023-11-28T11:09:15.014206+0000","last_became_peered":"2023-11-28T11:09:15.014206+0000","last_unstale":"2023-11-28T11:09:16.396230+0000","last_undegraded":"2023-11-28T11:09:16.396230+0000","last_fullsized":"2023-11-28T11:09:16.396230+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:40:22.341458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.411211+0000","last_change":"2023-11-28T11:09:15.409280+0000","last_active":"2023-11-28T11:09:15.411211+0000","last_peered":"2023-11-28T11:09:15.411211+0000","last_clean":"2023-11-28T11:09:15.411211+0000","last_became_active":"2023-11-28T11:09:15.409032+0000","last_became_peered":"2023-11-28T11:09:15.409032+0000","last_unstale":"2023-11-28T11:09:15.411211+0000","last_undegraded":"2023-11-28T11:09:15.411211+0000","last_fullsized":"2023-11-28T11:09:15.411211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:45.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396178+0000","last_change":"2023-11-28T11:09:15.009929+0000","last_active":"2023-11-28T11:09:16.396178+0000","last_peered":"2023-11-28T11:09:16.396178+0000","last_clean":"2023-11-28T11:09:16.396178+0000","last_became_active":"2023-11-28T11:09:15.009432+0000","last_became_peered":"2023-11-28T11:09:15.009432+0000","last_unstale":"2023-11-28T11:09:16.396178+0000","last_undegraded":"2023-11-28T11:09:16.396178+0000","last_fullsized":"2023-11-28T11:09:16.396178+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:27:59.533490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396007+0000","last_change":"2023-11-28T11:09:15.400798+0000","last_active":"2023-11-28T11:09:16.396007+0000","last_peered":"2023-11-28T11:09:16.396007+0000","last_clean":"2023-11-28T11:09:16.396007+0000","last_became_active":"2023-11-28T11:09:15.400537+0000","last_became_peered":"2023-11-28T11:09:15.400537+0000","last_unstale":"2023-11-28T11:09:16.396007+0000","last_undegraded":"2023-11-28T11:09:16.396007+0000","last_fullsized":"2023-11-28T11:09:16.396007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:26:42.661150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.400247+0000","last_change":"2023-11-28T11:09:15.014548+0000","last_active":"2023-11-28T11:09:15.400247+0000","last_peered":"2023-11-28T11:09:15.400247+0000","last_clean":"2023-11-28T11:09:15.400247+0000","last_became_active":"2023-11-28T11:09:15.014227+0000","last_became_peered":"2023-11-28T11:09:15.014227+0000","last_unstale":"2023-11-28T11:09:15.400247+0000","last_undegraded":"2023-11-28T11:09:15.400247+0000","last_fullsized":"2023-11-28T11:09:15.400247+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:40.901913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.418788+0000","last_change":"2023-11-28T11:09:15.409322+0000","last_active":"2023-11-28T11:09:15.418788+0000","last_peered":"2023-11-28T11:09:15.418788+0000","last_clean":"2023-11-28T11:09:15.418788+0000","last_became_active":"2023-11-28T11:09:15.408709+0000","last_became_peered":"2023-11-28T11:09:15.408709+0000","last_unstale":"2023-11-28T11:09:15.418788+0000","last_undegraded":"2023-11-28T11:09:15.418788+0000","last_fullsized":"2023-11-28T11:09:15.418788+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:30:49.726926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.403063+0000","last_change":"2023-11-28T11:09:15.009058+0000","last_active":"2023-11-28T11:09:15.403063+0000","last_peered":"2023-11-28T11:09:15.403063+0000","last_clean":"2023-11-28T11:09:15.403063+0000","last_became_active":"2023-11-28T11:09:15.008742+0000","last_became_peered":"2023-11-28T11:09:15.008742+0000","last_unstale":"2023-11-28T11:09:15.403063+0000","last_undegraded":"2023-11-28T11:09:15.403063+0000","last_fullsized":"2023-11-28T11:09:15.403063+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:41.576941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396042+0000","last_change":"2023-11-28T11:09:15.400595+0000","last_active":"2023-11-28T11:09:16.396042+0000","last_peered":"2023-11-28T11:09:16.396042+0000","last_clean":"2023-11-28T11:09:16.396042+0000","last_became_active":"2023-11-28T11:09:15.400320+0000","last_became_peered":"2023-11-28T11:09:15.400320+0000","last_unstale":"2023-11-28T11:09:16.396042+0000","last_undegraded":"2023-11-28T11:09:16.396042+0000","last_fullsized":"2023-11-28T11:09:16.396042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:03.492691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.426614+0000","last_change":"2023-11-28T11:09:15.014748+0000","last_active":"2023-11-28T11:09:15.426614+0000","last_peered":"2023-11-28T11:09:15.426614+0000","last_clean":"2023-11-28T11:09:15.426614+0000","last_became_active":"2023-11-28T11:09:15.014357+0000","last_became_peered":"2023-11-28T11:09:15.014357+0000","last_unstale":"2023-11-28T11:09:15.426614+0000","last_undegraded":"2023-11-28T11:09:15.426614+0000","last_fullsized":"2023-11-28T11:09:15.426614+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:20.991210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396145+0000","last_change":"2023-11-28T11:09:15.413732+0000","last_active":"2023-11-28T11:09:16.396145+0000","last_peered":"2023-11-28T11:09:16.396145+0000","last_clean":"2023-11-28T11:09:16.396145+0000","last_became_active":"2023-11-28T11:09:15.413494+0000","last_became_peered":"2023-11-28T11:09:15.413494+0000","last_unstale":"2023-11-28T11:09:16.396145+0000","last_undegraded":"2023-11-28T11:09:16.396145+0000","last_fullsized":"2023-11-28T11:09:16.396145+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:21:03.120132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401836+0000","last_change":"2023-11-28T11:09:15.010927+0000","last_active":"2023-11-28T11:09:16.401836+0000","last_peered":"2023-11-28T11:09:16.401836+0000","last_clean":"2023-11-28T11:09:16.401836+0000","last_became_active":"2023-11-28T11:09:15.010575+0000","last_became_peered":"2023-11-28T11:09:15.010575+0000","last_unstale":"2023-11-28T11:09:16.401836+0000","last_undegraded":"2023-11-28T11:09:16.401836+0000","last_fullsized":"2023-11-28T11:09:16.401836+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:24.567472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401643+0000","last_change":"2023-11-28T11:09:15.410705+0000","last_active":"2023-11-28T11:09:16.401643+0000","last_peered":"2023-11-28T11:09:16.401643+0000","last_clean":"2023-11-28T11:09:16.401643+0000","last_became_active":"2023-11-28T11:09:15.410443+0000","last_became_peered":"2023-11-28T11:09:15.410443+0000","last_unstale":"2023-11-28T11:09:16.401643+0000","last_undegraded":"2023-11-28T11:09:16.401643+0000","last_fullsized":"2023-11-28T11:09:16.401643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:03.503649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.427128+0000","last_change":"2023-11-28T11:09:15.014109+0000","last_active":"2023-11-28T11:09:15.427128+0000","last_peered":"2023-11-28T11:09:15.427128+0000","last_clean":"2023-11-28T11:09:15.427128+0000","last_became_active":"2023-11-28T11:09:15.013834+0000","last_became_peered":"2023-11-28T11:09:15.013834+0000","last_unstale":"2023-11-28T11:09:15.427128+0000","last_undegraded":"2023-11-28T11:09:15.427128+0000","last_fullsized":"2023-11-28T11:09:15.427128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:51.363437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.424160+0000","last_change":"2023-11-28T11:09:15.415149+0000","last_active":"2023-11-28T11:09:15.424160+0000","last_peered":"2023-11-28T11:09:15.424160+0000","last_clean":"2023-11-28T11:09:15.424160+0000","last_became_active":"2023-11-28T11:09:15.414602+0000","last_became_peered":"2023-11-28T11:09:15.414602+0000","last_unstale":"2023-11-28T11:09:15.424160+0000","last_undegraded":"2023-11-28T11:09:15.424160+0000","last_fullsized":"2023-11-28T11:09:15.424160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:47:10.507056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401177+0000","last_change":"2023-11-28T11:09:15.015937+0000","last_active":"2023-11-28T11:09:16.401177+0000","last_peered":"2023-11-28T11:09:16.401177+0000","last_clean":"2023-11-28T11:09:16.401177+0000","last_became_active":"2023-11-28T11:09:15.015651+0000","last_became_peered":"2023-11-28T11:09:15.015651+0000","last_unstale":"2023-11-28T11:09:16.401177+0000","last_undegraded":"2023-11-28T11:09:16.401177+0000","last_fullsized":"2023-11-28T11:09:16.401177+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:39.332888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401349+0000","last_change":"2023-11-28T11:09:15.414480+0000","last_active":"2023-11-28T11:09:16.401349+0000","last_peered":"2023-11-28T11:09:16.401349+0000","last_clean":"2023-11-28T11:09:16.401349+0000","last_became_active":"2023-11-28T11:09:15.414206+0000","last_became_peered":"2023-11-28T11:09:15.414206+0000","last_unstale":"2023-11-28T11:09:16.401349+0000","last_undegraded":"2023-11-28T11:09:16.401349+0000","last_fullsized":"2023-11-28T11:09:16.401349+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:52.222986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.400023+0000","last_change":"2023-11-28T11:09:15.014777+0000","last_active":"2023-11-28T11:09:15.400023+0000","last_peered":"2023-11-28T11:09:15.400023+0000","last_clean":"2023-11-28T11:09:15.400023+0000","last_became_active":"2023-11-28T11:09:15.014532+0000","last_became_peered":"2023-11-28T11:09:15.014532+0000","last_unstale":"2023-11-28T11:09:15.400023+0000","last_undegraded":"2023-11-28T11:09:15.400023+0000","last_fullsized":"2023-11-28T11:09:15.400023+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:56.596199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.420148+0000","last_change":"2023-11-28T11:09:15.417774+0000","last_active":"2023-11-28T11:09:15.420148+0000","last_peered":"2023-11-28T11:09:15.420148+0000","last_clean":"2023-11-28T11:09:15.420148+0000","last_became_active":"2023-11-28T11:09:15.413759+0000","last_became_peered":"2023-11-28T11:09:15.413759+0000","last_unstale":"2023-11-28T11:09:15.420148+0000","last_undegraded":"2023-11-28T11:09:15.420148+0000","last_fullsized":"2023-11-28T11:09:15.420148+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:33.228932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401146+0000","last_change":"2023-11-28T11:09:15.015872+0000","last_active":"2023-11-28T11:09:16.401146+0000","last_peered":"2023-11-28T11:09:16.401146+0000","last_clean":"2023-11-28T11:09:16.401146+0000","last_became_active":"2023-11-28T11:09:15.015605+0000","last_became_peered":"2023-11-28T11:09:15.015605+0000","last_unstale":"2023-11-28T11:09:16.401146+0000","last_undegraded":"2023-11-28T11:09:16.401146+0000","last_fullsized":"2023-11-28T11:09:16.401146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:53.361811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.414410+0000","last_change":"2023-11-28T11:09:15.411044+0000","last_active":"2023-11-28T11:09:15.414410+0000","last_peered":"2023-11-28T11:09:15.414410+0000","last_clean":"2023-11-28T11:09:15.414410+0000","last_became_active":"2023-11-28T11:09:15.410160+0000","last_became_peered":"2023-11-28T11:09:15.410160+0000","last_unstale":"2023-11-28T11:09:15.414410+0000","last_undegraded":"2023-11-28T11:09:15.414410+0000","last_fullsized":"2023-11-28T11:09:15.414410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:40.096463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401098+0000","last_change":"2023-11-28T11:09:15.011836+0000","last_active":"2023-11-28T11:09:16.401098+0000","last_peered":"2023-11-28T11:09:16.401098+0000","last_clean":"2023-11-28T11:09:16.401098+0000","last_became_active":"2023-11-28T11:09:15.011524+0000","last_became_peered":"2023-11-28T11:09:15.011524+0000","last_unstale":"2023-11-28T11:09:16.401098+0000","last_undegraded":"2023-11-28T11:09:16.401098+0000","last_fullsized":"2023-11-28T11:09:16.401098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:33.493593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401418+0000","last_change":"2023-11-28T11:09:15.416487+0000","last_active":"2023-11-28T11:09:16.401418+0000","last_peered":"2023-11-28T11:09:16.401418+0000","last_clean":"2023-11-28T11:09:16.401418+0000","last_became_active":"2023-11-28T11:09:15.416207+0000","last_became_peered":"2023-11-28T11:09:15.416207+0000","last_unstale":"2023-11-28T11:09:16.401418+0000","last_undegraded":"2023-11-28T11:09:16.401418+0000","last_fullsized":"2023-11-28T11:09:16.401418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:30:49.726926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.407603+0000","last_change":"2023-11-28T11:09:15.014756+0000","last_active":"2023-11-28T11:09:15.407603+0000","last_peered":"2023-11-28T11:09:15.407603+0000","last_clean":"2023-11-28T11:09:15.407603+0000","last_became_active":"2023-11-28T11:09:15.014480+0000","last_became_peered":"2023-11-28T11:09:15.014480+0000","last_unstale":"2023-11-28T11:09:15.407603+0000","last_undegraded":"2023-11-28T11:09:15.407603+0000","last_fullsized":"2023-11-28T11:09:15.407603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:02:27.855713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396077+0000","last_change":"2023-11-28T11:09:15.416362+0000","last_active":"2023-11-28T11:09:16.396077+0000","last_peered":"2023-11-28T11:09:16.396077+0000","last_clean":"2023-11-28T11:09:16.396077+0000","last_became_active":"2023-11-28T11:09:15.416111+0000","last_became_peered":"2023-11-28T11:09:15.416111+0000","last_unstale":"2023-11-28T11:09:16.396077+0000","last_undegraded":"2023-11-28T11:09:16.396077+0000","last_fullsized":"2023-11-28T11:09:16.396077+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:03.503649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.400976+0000","last_change":"2023-11-28T11:09:15.011238+0000","last_active":"2023-11-28T11:09:16.400976+0000","last_peered":"2023-11-28T11:09:16.400976+0000","last_clean":"2023-11-28T11:09:16.400976+0000","last_became_active":"2023-11-28T11:09:15.010828+0000","last_became_peered":"2023-11-28T11:09:15.010828+0000","last_unstale":"2023-11-28T11:09:16.400976+0000","last_undegraded":"2023-11-28T11:09:16.400976+0000","last_fullsized":"2023-11-28T11:09:16.400976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:59:24.742836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.401633+0000","last_change":"2023-11-28T11:09:15.399704+0000","last_active":"2023-11-28T11:09:15.401633+0000","last_peered":"2023-11-28T11:09:15.401633+0000","last_clean":"2023-11-28T11:09:15.401633+0000","last_became_active":"2023-11-28T11:09:15.399457+0000","last_became_peered":"2023-11-28T11:09:15.399457+0000","last_unstale":"2023-11-28T11:09:15.401633+0000","last_undegraded":"2023-11-28T11:09:15.401633+0000","last_fullsized":"2023-11-28T11:09:15.401633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:30:49.726926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.398569+0000","last_change":"2023-11-28T11:09:15.010635+0000","last_active":"2023-11-28T11:09:15.398569+0000","last_peered":"2023-11-28T11:09:15.398569+0000","last_clean":"2023-11-28T11:09:15.398569+0000","last_became_active":"2023-11-28T11:09:15.010334+0000","last_became_peered":"2023-11-28T11:09:15.010334+0000","last_unstale":"2023-11-28T11:09:15.398569+0000","last_undegraded":"2023-11-28T11:09:15.398569+0000","last_fullsized":"2023-11-28T11:09:15.398569+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:43.328002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.018564+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:09:15.018564+0000","last_peered":"2023-11-28T11:09:15.018564+0000","last_clean":"2023-11-28T11:09:15.018564+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:09:15.018564+0000","last_undegraded":"2023-11-28T11:09:15.018564+0000","last_fullsized":"2023-11-28T11:09:15.018564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396236+0000","last_change":"2023-11-28T11:09:15.414033+0000","last_active":"2023-11-28T11:09:16.396236+0000","last_peered":"2023-11-28T11:09:16.396236+0000","last_clean":"2023-11-28T11:09:16.396236+0000","last_became_active":"2023-11-28T11:09:15.413794+0000","last_became_peered":"2023-11-28T11:09:15.413794+0000","last_unstale":"2023-11-28T11:09:16.396236+0000","last_undegraded":"2023-11-28T11:09:16.396236+0000","last_fullsized":"2023-11-28T11:09:16.396236+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:17.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.398595+0000","last_change":"2023-11-28T11:09:15.018622+0000","last_active":"2023-11-28T11:09:15.398595+0000","last_peered":"2023-11-28T11:09:15.398595+0000","last_clean":"2023-11-28T11:09:15.398595+0000","last_became_active":"2023-11-28T11:09:15.018356+0000","last_became_peered":"2023-11-28T11:09:15.018356+0000","last_unstale":"2023-11-28T11:09:15.398595+0000","last_undegraded":"2023-11-28T11:09:15.398595+0000","last_fullsized":"2023-11-28T11:09:15.398595+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:15.572173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.402323+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:09:15.402323+0000","last_peered":"2023-11-28T11:09:15.402323+0000","last_clean":"2023-11-28T11:09:15.402323+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:09:15.402323+0000","last_undegraded":"2023-11-28T11:09:15.402323+0000","last_fullsized":"2023-11-28T11:09:15.402323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.420195+0000","last_change":"2023-11-28T11:09:15.418111+0000","last_active":"2023-11-28T11:09:15.420195+0000","last_peered":"2023-11-28T11:09:15.420195+0000","last_clean":"2023-11-28T11:09:15.420195+0000","last_became_active":"2023-11-28T11:09:15.417782+0000","last_became_peered":"2023-11-28T11:09:15.417782+0000","last_unstale":"2023-11-28T11:09:15.420195+0000","last_undegraded":"2023-11-28T11:09:15.420195+0000","last_fullsized":"2023-11-28T11:09:15.420195+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:45.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401289+0000","last_change":"2023-11-28T11:09:15.012228+0000","last_active":"2023-11-28T11:09:16.401289+0000","last_peered":"2023-11-28T11:09:16.401289+0000","last_clean":"2023-11-28T11:09:16.401289+0000","last_became_active":"2023-11-28T11:09:15.011934+0000","last_became_peered":"2023-11-28T11:09:15.011934+0000","last_unstale":"2023-11-28T11:09:16.401289+0000","last_undegraded":"2023-11-28T11:09:16.401289+0000","last_fullsized":"2023-11-28T11:09:16.401289+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:58:15.998627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.405133+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:09:15.405133+0000","last_peered":"2023-11-28T11:09:15.405133+0000","last_clean":"2023-11-28T11:09:15.405133+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:09:15.405133+0000","last_undegraded":"2023-11-28T11:09:15.405133+0000","last_fullsized":"2023-11-28T11:09:15.405133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401830+0000","last_change":"2023-11-28T11:09:15.415879+0000","last_active":"2023-11-28T11:09:16.401830+0000","last_peered":"2023-11-28T11:09:16.401830+0000","last_clean":"2023-11-28T11:09:16.401830+0000","last_became_active":"2023-11-28T11:09:15.415655+0000","last_became_peered":"2023-11-28T11:09:15.415655+0000","last_unstale":"2023-11-28T11:09:16.401830+0000","last_undegraded":"2023-11-28T11:09:16.401830+0000","last_fullsized":"2023-11-28T11:09:16.401830+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:21:03.120132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.406908+0000","last_change":"2023-11-28T11:09:15.015486+0000","last_active":"2023-11-28T11:09:15.406908+0000","last_peered":"2023-11-28T11:09:15.406908+0000","last_clean":"2023-11-28T11:09:15.406908+0000","last_became_active":"2023-11-28T11:09:15.015210+0000","last_became_peered":"2023-11-28T11:09:15.015210+0000","last_unstale":"2023-11-28T11:09:15.406908+0000","last_undegraded":"2023-11-28T11:09:15.406908+0000","last_fullsized":"2023-11-28T11:09:15.406908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:08.798600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.402079+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:09:15.402079+0000","last_peered":"2023-11-28T11:09:15.402079+0000","last_clean":"2023-11-28T11:09:15.402079+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:09:15.402079+0000","last_undegraded":"2023-11-28T11:09:15.402079+0000","last_fullsized":"2023-11-28T11:09:15.402079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.005655+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:09:15.005655+0000","last_peered":"2023-11-28T11:09:15.005655+0000","last_clean":"2023-11-28T11:09:15.005655+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:09:15.005655+0000","last_undegraded":"2023-11-28T11:09:15.005655+0000","last_fullsized":"2023-11-28T11:09:15.005655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.418417+0000","last_change":"2023-11-28T11:09:15.417579+0000","last_active":"2023-11-28T11:09:15.418417+0000","last_peered":"2023-11-28T11:09:15.418417+0000","last_clean":"2023-11-28T11:09:15.418417+0000","last_became_active":"2023-11-28T11:09:15.417326+0000","last_became_peered":"2023-11-28T11:09:15.417326+0000","last_unstale":"2023-11-28T11:09:15.418417+0000","last_undegraded":"2023-11-28T11:09:15.418417+0000","last_fullsized":"2023-11-28T11:09:15.418417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:55:05.158077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401943+0000","last_change":"2023-11-28T11:09:15.014272+0000","last_active":"2023-11-28T11:09:16.401943+0000","last_peered":"2023-11-28T11:09:16.401943+0000","last_clean":"2023-11-28T11:09:16.401943+0000","last_became_active":"2023-11-28T11:09:15.014022+0000","last_became_peered":"2023-11-28T11:09:15.014022+0000","last_unstale":"2023-11-28T11:09:16.401943+0000","last_undegraded":"2023-11-28T11:09:16.401943+0000","last_fullsized":"2023-11-28T11:09:16.401943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:15.572173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.005742+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:09:15.005742+0000","last_peered":"2023-11-28T11:09:15.005742+0000","last_clean":"2023-11-28T11:09:15.005742+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:09:15.005742+0000","last_undegraded":"2023-11-28T11:09:15.005742+0000","last_fullsized":"2023-11-28T11:09:15.005742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.417784+0000","last_change":"2023-11-28T11:09:15.416594+0000","last_active":"2023-11-28T11:09:15.417784+0000","last_peered":"2023-11-28T11:09:15.417784+0000","last_clean":"2023-11-28T11:09:15.417784+0000","last_became_active":"2023-11-28T11:09:15.416340+0000","last_became_peered":"2023-11-28T11:09:15.416340+0000","last_unstale":"2023-11-28T11:09:15.417784+0000","last_undegraded":"2023-11-28T11:09:15.417784+0000","last_fullsized":"2023-11-28T11:09:15.417784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:33.228932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.401381+0000","last_change":"2023-11-28T11:09:15.010097+0000","last_active":"2023-11-28T11:09:15.401381+0000","last_peered":"2023-11-28T11:09:15.401381+0000","last_clean":"2023-11-28T11:09:15.401381+0000","last_became_active":"2023-11-28T11:09:15.009810+0000","last_became_peered":"2023-11-28T11:09:15.009810+0000","last_unstale":"2023-11-28T11:09:15.401381+0000","last_undegraded":"2023-11-28T11:09:15.401381+0000","last_fullsized":"2023-11-28T11:09:15.401381+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:35.910146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.006144+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:09:15.006144+0000","last_peered":"2023-11-28T11:09:15.006144+0000","last_clean":"2023-11-28T11:09:15.006144+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:09:15.006144+0000","last_undegraded":"2023-11-28T11:09:15.006144+0000","last_fullsized":"2023-11-28T11:09:15.006144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.424202+0000","last_change":"2023-11-28T11:09:15.421691+0000","last_active":"2023-11-28T11:09:15.424202+0000","last_peered":"2023-11-28T11:09:15.424202+0000","last_clean":"2023-11-28T11:09:15.424202+0000","last_became_active":"2023-11-28T11:09:15.421457+0000","last_became_peered":"2023-11-28T11:09:15.421457+0000","last_unstale":"2023-11-28T11:09:15.424202+0000","last_undegraded":"2023-11-28T11:09:15.424202+0000","last_fullsized":"2023-11-28T11:09:15.424202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:46.016400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.405899+0000","last_change":"2023-11-28T11:09:15.010103+0000","last_active":"2023-11-28T11:09:15.405899+0000","last_peered":"2023-11-28T11:09:15.405899+0000","last_clean":"2023-11-28T11:09:15.405899+0000","last_became_active":"2023-11-28T11:09:15.009846+0000","last_became_peered":"2023-11-28T11:09:15.009846+0000","last_unstale":"2023-11-28T11:09:15.405899+0000","last_undegraded":"2023-11-28T11:09:15.405899+0000","last_fullsized":"2023-11-28T11:09:15.405899+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:59.183814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.018878+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:09:15.018878+0000","last_peered":"2023-11-28T11:09:15.018878+0000","last_clean":"2023-11-28T11:09:15.018878+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:09:15.018878+0000","last_undegraded":"2023-11-28T11:09:15.018878+0000","last_fullsized":"2023-11-28T11:09:15.018878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.421555+0000","last_change":"2023-11-28T11:09:15.416581+0000","last_active":"2023-11-28T11:09:15.421555+0000","last_peered":"2023-11-28T11:09:15.421555+0000","last_clean":"2023-11-28T11:09:15.421555+0000","last_became_active":"2023-11-28T11:09:15.416310+0000","last_became_peered":"2023-11-28T11:09:15.416310+0000","last_unstale":"2023-11-28T11:09:15.421555+0000","last_undegraded":"2023-11-28T11:09:15.421555+0000","last_fullsized":"2023-11-28T11:09:15.421555+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:55:05.158077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.395944+0000","last_change":"2023-11-28T11:09:15.018841+0000","last_active":"2023-11-28T11:09:16.395944+0000","last_peered":"2023-11-28T11:09:16.395944+0000","last_clean":"2023-11-28T11:09:16.395944+0000","last_became_active":"2023-11-28T11:09:15.018409+0000","last_became_peered":"2023-11-28T11:09:15.018409+0000","last_unstale":"2023-11-28T11:09:16.395944+0000","last_undegraded":"2023-11-28T11:09:16.395944+0000","last_fullsized":"2023-11-28T11:09:16.395944+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:43.328002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.001282+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:09:15.001282+0000","last_peered":"2023-11-28T11:09:15.001282+0000","last_clean":"2023-11-28T11:09:15.001282+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:09:15.001282+0000","last_undegraded":"2023-11-28T11:09:15.001282+0000","last_fullsized":"2023-11-28T11:09:15.001282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9644,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362196,"statfs":{"total":96636764160,"available":96626888704,"internally_reserved":0,"allocated":1019904,"data_stored":760721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.503}]},{"osd":1,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71199999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.73399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8348,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363492,"statfs":{"total":96636764160,"available":96628215808,"internally_reserved":0,"allocated":413696,"data_stored":169301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":5.1239999999999997,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.0410000000000004,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.51000000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.0250000000000004,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":5.1509999999999998,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.752}]},{"osd":2,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":6.0620000000000003,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.9409999999999998,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.60699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.65700000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.84899999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.76800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.80400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8476,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363364,"statfs":{"total":96636764160,"available":96628084736,"internally_reserved":0,"allocated":413696,"data_stored":168825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.0519999999999996,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.32600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":7.0890000000000004,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.38}]},{"osd":3,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.307}]},{"osd":4,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":18.024999999999999,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":17.966999999999999,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":18.056999999999999,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.69499999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9504,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362336,"statfs":{"total":96636764160,"available":96627032064,"internally_reserved":0,"allocated":1007616,"data_stored":759137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.253},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.29099999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.307},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.37},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.56599999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.64500000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.67000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8348,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363492,"statfs":{"total":96636764160,"available":96628215808,"internally_reserved":0,"allocated":413696,"data_stored":168805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.308},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.33800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.375},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.29299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.35099999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.626}]},{"osd":6,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57399999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8488,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363352,"statfs":{"total":96636764160,"available":96628072448,"internally_reserved":0,"allocated":425984,"data_stored":170317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.28799999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.39500000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.35399999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.53700000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.45800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8608,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363232,"statfs":{"total":96636764160,"available":96627949568,"internally_reserved":0,"allocated":417792,"data_stored":169323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.496},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":4.9740000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":12.263,"5min":12.263,"15min":12.263},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.524},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.876}]},{"osd":3,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.54},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":1.5600000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":4.3710000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":5.0220000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":1.923}]},{"osd":7,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":1.6579999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":1.49}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8480,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363360,"statfs":{"total":96636764160,"available":96628080640,"internally_reserved":0,"allocated":417792,"data_stored":168847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.63900000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.47599999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.495}]},{"osd":3,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.61899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.58999999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.67500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:09:19.491 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:09:19.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.644+0000 7f4650366700 1 Processor -- start 2023-11-28T11:09:19.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.644+0000 7f4650366700 1 -- start start 2023-11-28T11:09:19.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.645+0000 7f4650366700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f464806ba90 0x7f46480af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.645+0000 7f4650366700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46480cc5f0 con 0x7f46480cbdf0 2023-11-28T11:09:19.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.645+0000 7f4650366700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464815d470 con 0x7f464815cef0 2023-11-28T11:09:19.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.645+0000 7f4650366700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464815d750 con 0x7f464806ba90 2023-11-28T11:09:19.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.645+0000 7f464e903700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f46480cbdf0 0x7f46480ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48686/0 (socket says 172.21.15.156:48686) 2023-11-28T11:09:19.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.645+0000 7f464e903700 1 -- 172.21.15.156:0/2712383505 learned_addr learned my addr 172.21.15.156:0/2712383505 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:19.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d901700 1 --2- 172.21.15.156:0/2712383505 >> v2:172.21.15.186:3300/0 conn(0x7f464806ba90 0x7f46480af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748483047 0 0) 0x7f46480cc5f0 con 0x7f46480cbdf0 2023-11-28T11:09:19.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46340035b0 con 0x7f46480cbdf0 2023-11-28T11:09:19.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2072981143 0 0) 0x7f46340035b0 con 0x7f46480cbdf0 2023-11-28T11:09:19.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 >> v2:172.21.15.186:3300/0 conn(0x7f464806ba90 msgr2=0x7f46480af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:19.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 --2- 172.21.15.156:0/2712383505 >> v2:172.21.15.186:3300/0 conn(0x7f464806ba90 0x7f46480af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 >> v1:172.21.15.186:6789/0 conn(0x7f464815cef0 legacy=0x7f464806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464815da30 con 0x7f46480cbdf0 2023-11-28T11:09:19.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f46400019f0 con 0x7f46480cbdf0 2023-11-28T11:09:19.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 >> v1:172.21.15.156:6789/0 conn(0x7f46480cbdf0 legacy=0x7f46480ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 --2- 172.21.15.156:0/2712383505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4634003a00 0x7f4634003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 --2- 172.21.15.156:0/2712383505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4634004850 0x7f4634004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 --2- 172.21.15.156:0/2712383505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4634005200 0x7f463400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464815d750 con 0x7f4634004850 2023-11-28T11:09:19.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463400c080 con 0x7f4634003a00 2023-11-28T11:09:19.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463400e450 con 0x7f4634005200 2023-11-28T11:09:19.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d901700 1 --2- 172.21.15.156:0/2712383505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4634004850 0x7f4634004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464e102700 1 --2- 172.21.15.156:0/2712383505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4634003a00 0x7f4634003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d901700 1 -- 172.21.15.156:0/2712383505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4634005200 msgr2=0x7f463400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:19.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d901700 1 --2- 172.21.15.156:0/2712383505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4634005200 0x7f463400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d901700 1 -- 172.21.15.156:0/2712383505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4634003a00 msgr2=0x7f4634003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:19.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d901700 1 --2- 172.21.15.156:0/2712383505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4634003a00 0x7f4634003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d901700 1 -- 172.21.15.156:0/2712383505 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340035b0 con 0x7f4634004850 2023-11-28T11:09:19.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464e903700 1 -- 172.21.15.156:0/2712383505 reap_dead start 2023-11-28T11:09:19.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.646+0000 7f464d901700 1 --2- 172.21.15.156:0/2712383505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4634004850 0x7f4634004cc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4644009b60 tx=0x7f4634005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1316243fd97594 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:19.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.647+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464400e6c0 con 0x7f4634004850 2023-11-28T11:09:19.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.647+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464400e820 con 0x7f4634004850 2023-11-28T11:09:19.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.647+0000 7f464d100700 1 -- 172.21.15.156:0/2712383505 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464400eb90 con 0x7f4634004850 2023-11-28T11:09:19.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.647+0000 7f4650366700 1 -- 172.21.15.156:0/2712383505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4634004850 msgr2=0x7f4634004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.647+0000 7f4650366700 1 --2- 172.21.15.156:0/2712383505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4634004850 0x7f4634004cc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4644009b60 tx=0x7f4634005880 comp rx=0 tx=0).stop 2023-11-28T11:09:19.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.647+0000 7f4650366700 1 -- 172.21.15.156:0/2712383505 shutdown_connections 2023-11-28T11:09:19.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.647+0000 7f4650366700 1 --2- 172.21.15.156:0/2712383505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4634004850 0x7f4634004cc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.647+0000 7f4650366700 1 -- 172.21.15.156:0/2712383505 >> 172.21.15.156:0/2712383505 conn(0x7f46480d0000 msgr2=0x7f46480d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:19.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.647+0000 7f4650366700 1 -- 172.21.15.156:0/2712383505 shutdown_connections 2023-11-28T11:09:19.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.647+0000 7f4650366700 1 -- 172.21.15.156:0/2712383505 wait complete. 2023-11-28T11:09:19.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.648+0000 7f4650366700 1 Processor -- start 2023-11-28T11:09:19.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.648+0000 7f4650366700 1 -- start start 2023-11-28T11:09:19.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f4650366700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4634004850 0x7f46481575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f4650366700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46480b55d0 0x7f4648158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f4650366700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46480b5a40 0x7f46481590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f4650366700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46481f23d0 con 0x7f46480b55d0 2023-11-28T11:09:19.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f4650366700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46481f26b0 con 0x7f4634004850 2023-11-28T11:09:19.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f4650366700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46481f2990 con 0x7f46480b5a40 2023-11-28T11:09:19.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f464d901700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46480b55d0 0x7f4648158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f464d901700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46480b55d0 0x7f4648158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33818/0 (socket says 172.21.15.156:33818) 2023-11-28T11:09:19.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f464d901700 1 -- 172.21.15.156:0/3652457102 learned_addr learned my addr 172.21.15.156:0/3652457102 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:19.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f464e903700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46480b5a40 0x7f46481590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f464e102700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4634004850 0x7f46481575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f464d901700 1 -- 172.21.15.156:0/3652457102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46480b5a40 msgr2=0x7f46481590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f464d901700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46480b5a40 0x7f46481590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f464d901700 1 -- 172.21.15.156:0/3652457102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4634004850 msgr2=0x7f46481575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.649+0000 7f464d901700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4634004850 0x7f46481575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.650+0000 7f464d901700 1 -- 172.21.15.156:0/3652457102 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464815dff0 con 0x7f46480b55d0 2023-11-28T11:09:19.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.650+0000 7f464e903700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46480b5a40 0x7f46481590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:19.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.650+0000 7f464e102700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4634004850 0x7f46481575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:19.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.650+0000 7f464d901700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46480b55d0 0x7f4648158b80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f463400c050 tx=0x7f46440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:19.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.650+0000 7f463effd700 1 -- 172.21.15.156:0/3652457102 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464400e6c0 con 0x7f46480b55d0 2023-11-28T11:09:19.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.650+0000 7f463effd700 1 -- 172.21.15.156:0/3652457102 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4644003ad0 con 0x7f46480b55d0 2023-11-28T11:09:19.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.650+0000 7f463effd700 1 -- 172.21.15.156:0/3652457102 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644003e80 con 0x7f46480b55d0 2023-11-28T11:09:19.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.650+0000 7f4650366700 1 -- 172.21.15.156:0/3652457102 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464815dd10 con 0x7f46480b55d0 2023-11-28T11:09:19.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.650+0000 7f4650366700 1 -- 172.21.15.156:0/3652457102 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46480af400 con 0x7f46480b55d0 2023-11-28T11:09:19.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.651+0000 7f4650366700 1 -- 172.21.15.156:0/3652457102 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46480aefd0 con 0x7f46480b55d0 2023-11-28T11:09:19.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.651+0000 7f463effd700 1 -- 172.21.15.156:0/3652457102 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4644019030 con 0x7f46480b55d0 2023-11-28T11:09:19.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.651+0000 7f463effd700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f462c0682f0 0x7f462c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:19.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.652+0000 7f463effd700 1 -- 172.21.15.156:0/3652457102 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f46440873f0 con 0x7f46480b55d0 2023-11-28T11:09:19.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.652+0000 7f464e102700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f462c0682f0 0x7f462c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:19.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.652+0000 7f464e102700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f462c0682f0 0x7f462c06a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4638000c00 tx=0x7f4638009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:19.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.656+0000 7f463effd700 1 -- 172.21.15.156:0/3652457102 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4644053c50 con 0x7f46480b55d0 2023-11-28T11:09:19.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.835+0000 7f4650366700 1 -- 172.21.15.156:0/3652457102 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46480d0e60 con 0x7f462c0682f0 2023-11-28T11:09:19.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.844+0000 7f463effd700 1 -- 172.21.15.156:0/3652457102 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311600 (secure 0 0 0) 0x7f46480d0e60 con 0x7f462c0682f0 2023-11-28T11:09:19.850 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:19.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.847+0000 7f4650366700 1 -- 172.21.15.156:0/3652457102 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f462c0682f0 msgr2=0x7f462c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.847+0000 7f4650366700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f462c0682f0 0x7f462c06a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4638000c00 tx=0x7f4638009040 comp rx=0 tx=0).stop 2023-11-28T11:09:19.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.847+0000 7f4650366700 1 -- 172.21.15.156:0/3652457102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46480b55d0 msgr2=0x7f4648158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:19.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.847+0000 7f4650366700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46480b55d0 0x7f4648158b80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f463400c050 tx=0x7f46440047c0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.847+0000 7f4650366700 1 -- 172.21.15.156:0/3652457102 shutdown_connections 2023-11-28T11:09:19.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.847+0000 7f4650366700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f462c0682f0 0x7f462c06a7a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.847+0000 7f4650366700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46480b5a40 0x7f46481590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.847+0000 7f4650366700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4634004850 0x7f46481575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.847+0000 7f4650366700 1 --2- 172.21.15.156:0/3652457102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46480b55d0 0x7f4648158b80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:19.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.847+0000 7f4650366700 1 -- 172.21.15.156:0/3652457102 >> 172.21.15.156:0/3652457102 conn(0x7f46480d0000 msgr2=0x7f464806d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:19.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.847+0000 7f4650366700 1 -- 172.21.15.156:0/3652457102 shutdown_connections 2023-11-28T11:09:19.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:19.847+0000 7f4650366700 1 -- 172.21.15.156:0/3652457102 wait complete. 2023-11-28T11:09:19.855 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:09:19.862 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2023-11-28T11:09:18.538194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":69896,"kb_used_data":4424,"kb_used_omap":0,"kb_used_meta":65408,"kb_avail":754904824,"statfs":{"total":773094113280,"available":773022539776,"internally_reserved":0,"allocated":4530176,"data_stored":2535276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66977792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.548800"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396608+0000","last_change":"2023-11-28T11:09:15.401196+0000","last_active":"2023-11-28T11:09:16.396608+0000","last_peered":"2023-11-28T11:09:16.396608+0000","last_clean":"2023-11-28T11:09:16.396608+0000","last_became_active":"2023-11-28T11:09:15.400945+0000","last_became_peered":"2023-11-28T11:09:15.400945+0000","last_unstale":"2023-11-28T11:09:16.396608+0000","last_undegraded":"2023-11-28T11:09:16.396608+0000","last_fullsized":"2023-11-28T11:09:16.396608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:22.625942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396148+0000","last_change":"2023-11-28T11:09:15.409873+0000","last_active":"2023-11-28T11:09:16.396148+0000","last_peered":"2023-11-28T11:09:16.396148+0000","last_clean":"2023-11-28T11:09:16.396148+0000","last_became_active":"2023-11-28T11:09:15.409593+0000","last_became_peered":"2023-11-28T11:09:15.409593+0000","last_unstale":"2023-11-28T11:09:16.396148+0000","last_undegraded":"2023-11-28T11:09:16.396148+0000","last_fullsized":"2023-11-28T11:09:16.396148+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:52.365078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.411151+0000","last_change":"2023-11-28T11:09:15.409034+0000","last_active":"2023-11-28T11:09:15.411151+0000","last_peered":"2023-11-28T11:09:15.411151+0000","last_clean":"2023-11-28T11:09:15.411151+0000","last_became_active":"2023-11-28T11:09:15.408740+0000","last_became_peered":"2023-11-28T11:09:15.408740+0000","last_unstale":"2023-11-28T11:09:15.411151+0000","last_undegraded":"2023-11-28T11:09:15.411151+0000","last_fullsized":"2023-11-28T11:09:15.411151+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:52.365078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401234+0000","last_change":"2023-11-28T11:09:15.415786+0000","last_active":"2023-11-28T11:09:16.401234+0000","last_peered":"2023-11-28T11:09:16.401234+0000","last_clean":"2023-11-28T11:09:16.401234+0000","last_became_active":"2023-11-28T11:09:15.415549+0000","last_became_peered":"2023-11-28T11:09:15.415549+0000","last_unstale":"2023-11-28T11:09:16.401234+0000","last_undegraded":"2023-11-28T11:09:16.401234+0000","last_fullsized":"2023-11-28T11:09:16.401234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:44:40.603431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401075+0000","last_change":"2023-11-28T11:09:15.423012+0000","last_active":"2023-11-28T11:09:16.401075+0000","last_peered":"2023-11-28T11:09:16.401075+0000","last_clean":"2023-11-28T11:09:16.401075+0000","last_became_active":"2023-11-28T11:09:15.422745+0000","last_became_peered":"2023-11-28T11:09:15.422745+0000","last_unstale":"2023-11-28T11:09:16.401075+0000","last_undegraded":"2023-11-28T11:09:16.401075+0000","last_fullsized":"2023-11-28T11:09:16.401075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:57.597840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.408155+0000","last_change":"2023-11-28T11:09:15.404365+0000","last_active":"2023-11-28T11:09:15.408155+0000","last_peered":"2023-11-28T11:09:15.408155+0000","last_clean":"2023-11-28T11:09:15.408155+0000","last_became_active":"2023-11-28T11:09:15.404126+0000","last_became_peered":"2023-11-28T11:09:15.404126+0000","last_unstale":"2023-11-28T11:09:15.408155+0000","last_undegraded":"2023-11-28T11:09:15.408155+0000","last_fullsized":"2023-11-28T11:09:15.408155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:40.096463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401325+0000","last_change":"2023-11-28T11:09:15.404108+0000","last_active":"2023-11-28T11:09:16.401325+0000","last_peered":"2023-11-28T11:09:16.401325+0000","last_clean":"2023-11-28T11:09:16.401325+0000","last_became_active":"2023-11-28T11:09:15.403803+0000","last_became_peered":"2023-11-28T11:09:15.403803+0000","last_unstale":"2023-11-28T11:09:16.401325+0000","last_undegraded":"2023-11-28T11:09:16.401325+0000","last_fullsized":"2023-11-28T11:09:16.401325+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:22.107839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.395874+0000","last_change":"2023-11-28T11:09:15.417326+0000","last_active":"2023-11-28T11:09:16.395874+0000","last_peered":"2023-11-28T11:09:16.395874+0000","last_clean":"2023-11-28T11:09:16.395874+0000","last_became_active":"2023-11-28T11:09:15.417081+0000","last_became_peered":"2023-11-28T11:09:15.417081+0000","last_unstale":"2023-11-28T11:09:16.395874+0000","last_undegraded":"2023-11-28T11:09:16.395874+0000","last_fullsized":"2023-11-28T11:09:16.395874+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:22.107839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419254+0000","last_change":"2023-11-28T11:09:15.416962+0000","last_active":"2023-11-28T11:09:15.419254+0000","last_peered":"2023-11-28T11:09:15.419254+0000","last_clean":"2023-11-28T11:09:15.419254+0000","last_became_active":"2023-11-28T11:09:15.416699+0000","last_became_peered":"2023-11-28T11:09:15.416699+0000","last_unstale":"2023-11-28T11:09:15.419254+0000","last_undegraded":"2023-11-28T11:09:15.419254+0000","last_fullsized":"2023-11-28T11:09:15.419254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:22.107839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401887+0000","last_change":"2023-11-28T11:09:15.417258+0000","last_active":"2023-11-28T11:09:16.401887+0000","last_peered":"2023-11-28T11:09:16.401887+0000","last_clean":"2023-11-28T11:09:16.401887+0000","last_became_active":"2023-11-28T11:09:15.416945+0000","last_became_peered":"2023-11-28T11:09:15.416945+0000","last_unstale":"2023-11-28T11:09:16.401887+0000","last_undegraded":"2023-11-28T11:09:16.401887+0000","last_fullsized":"2023-11-28T11:09:16.401887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:26:42.661150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401755+0000","last_change":"2023-11-28T11:09:15.417408+0000","last_active":"2023-11-28T11:09:16.401755+0000","last_peered":"2023-11-28T11:09:16.401755+0000","last_clean":"2023-11-28T11:09:16.401755+0000","last_became_active":"2023-11-28T11:09:15.416914+0000","last_became_peered":"2023-11-28T11:09:15.416914+0000","last_unstale":"2023-11-28T11:09:16.401755+0000","last_undegraded":"2023-11-28T11:09:16.401755+0000","last_fullsized":"2023-11-28T11:09:16.401755+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:01.481847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.413858+0000","last_change":"2023-11-28T11:09:15.410540+0000","last_active":"2023-11-28T11:09:15.413858+0000","last_peered":"2023-11-28T11:09:15.413858+0000","last_clean":"2023-11-28T11:09:15.413858+0000","last_became_active":"2023-11-28T11:09:15.410281+0000","last_became_peered":"2023-11-28T11:09:15.410281+0000","last_unstale":"2023-11-28T11:09:15.413858+0000","last_undegraded":"2023-11-28T11:09:15.413858+0000","last_fullsized":"2023-11-28T11:09:15.413858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:00.185455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401411+0000","last_change":"2023-11-28T11:09:15.417734+0000","last_active":"2023-11-28T11:09:16.401411+0000","last_peered":"2023-11-28T11:09:16.401411+0000","last_clean":"2023-11-28T11:09:16.401411+0000","last_became_active":"2023-11-28T11:09:15.417461+0000","last_became_peered":"2023-11-28T11:09:15.417461+0000","last_unstale":"2023-11-28T11:09:16.401411+0000","last_undegraded":"2023-11-28T11:09:16.401411+0000","last_fullsized":"2023-11-28T11:09:16.401411+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:52.365078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.423899+0000","last_change":"2023-11-28T11:09:15.419261+0000","last_active":"2023-11-28T11:09:15.423899+0000","last_peered":"2023-11-28T11:09:15.423899+0000","last_clean":"2023-11-28T11:09:15.423899+0000","last_became_active":"2023-11-28T11:09:15.418975+0000","last_became_peered":"2023-11-28T11:09:15.418975+0000","last_unstale":"2023-11-28T11:09:15.423899+0000","last_undegraded":"2023-11-28T11:09:15.423899+0000","last_fullsized":"2023-11-28T11:09:15.423899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:17:08.883199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401257+0000","last_change":"2023-11-28T11:09:15.411176+0000","last_active":"2023-11-28T11:09:16.401257+0000","last_peered":"2023-11-28T11:09:16.401257+0000","last_clean":"2023-11-28T11:09:16.401257+0000","last_became_active":"2023-11-28T11:09:15.410928+0000","last_became_peered":"2023-11-28T11:09:15.410928+0000","last_unstale":"2023-11-28T11:09:16.401257+0000","last_undegraded":"2023-11-28T11:09:16.401257+0000","last_fullsized":"2023-11-28T11:09:16.401257+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:00.185455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.426398+0000","last_change":"2023-11-28T11:09:15.424366+0000","last_active":"2023-11-28T11:09:15.426398+0000","last_peered":"2023-11-28T11:09:15.426398+0000","last_clean":"2023-11-28T11:09:15.426398+0000","last_became_active":"2023-11-28T11:09:15.424083+0000","last_became_peered":"2023-11-28T11:09:15.424083+0000","last_unstale":"2023-11-28T11:09:15.426398+0000","last_undegraded":"2023-11-28T11:09:15.426398+0000","last_fullsized":"2023-11-28T11:09:15.426398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:22.107839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.426486+0000","last_change":"2023-11-28T11:09:15.424388+0000","last_active":"2023-11-28T11:09:15.426486+0000","last_peered":"2023-11-28T11:09:15.426486+0000","last_clean":"2023-11-28T11:09:15.426486+0000","last_became_active":"2023-11-28T11:09:15.424031+0000","last_became_peered":"2023-11-28T11:09:15.424031+0000","last_unstale":"2023-11-28T11:09:15.426486+0000","last_undegraded":"2023-11-28T11:09:15.426486+0000","last_fullsized":"2023-11-28T11:09:15.426486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:17.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419519+0000","last_change":"2023-11-28T11:09:15.418609+0000","last_active":"2023-11-28T11:09:15.419519+0000","last_peered":"2023-11-28T11:09:15.419519+0000","last_clean":"2023-11-28T11:09:15.419519+0000","last_became_active":"2023-11-28T11:09:15.418325+0000","last_became_peered":"2023-11-28T11:09:15.418325+0000","last_unstale":"2023-11-28T11:09:15.419519+0000","last_undegraded":"2023-11-28T11:09:15.419519+0000","last_fullsized":"2023-11-28T11:09:15.419519+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:16.573814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401252+0000","last_change":"2023-11-28T11:09:15.408726+0000","last_active":"2023-11-28T11:09:16.401252+0000","last_peered":"2023-11-28T11:09:16.401252+0000","last_clean":"2023-11-28T11:09:16.401252+0000","last_became_active":"2023-11-28T11:09:15.408476+0000","last_became_peered":"2023-11-28T11:09:15.408476+0000","last_unstale":"2023-11-28T11:09:16.401252+0000","last_undegraded":"2023-11-28T11:09:16.401252+0000","last_fullsized":"2023-11-28T11:09:16.401252+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:00.535131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.418609+0000","last_change":"2023-11-28T11:09:15.413840+0000","last_active":"2023-11-28T11:09:15.418609+0000","last_peered":"2023-11-28T11:09:15.418609+0000","last_clean":"2023-11-28T11:09:15.418609+0000","last_became_active":"2023-11-28T11:09:15.413528+0000","last_became_peered":"2023-11-28T11:09:15.413528+0000","last_unstale":"2023-11-28T11:09:15.418609+0000","last_undegraded":"2023-11-28T11:09:15.418609+0000","last_fullsized":"2023-11-28T11:09:15.418609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:41.903553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401538+0000","last_change":"2023-11-28T11:09:15.407084+0000","last_active":"2023-11-28T11:09:16.401538+0000","last_peered":"2023-11-28T11:09:16.401538+0000","last_clean":"2023-11-28T11:09:16.401538+0000","last_became_active":"2023-11-28T11:09:15.406790+0000","last_became_peered":"2023-11-28T11:09:15.406790+0000","last_unstale":"2023-11-28T11:09:16.401538+0000","last_undegraded":"2023-11-28T11:09:16.401538+0000","last_fullsized":"2023-11-28T11:09:16.401538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:22.625942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.420417+0000","last_change":"2023-11-28T11:09:15.412311+0000","last_active":"2023-11-28T11:09:15.420417+0000","last_peered":"2023-11-28T11:09:15.420417+0000","last_clean":"2023-11-28T11:09:15.420417+0000","last_became_active":"2023-11-28T11:09:15.411968+0000","last_became_peered":"2023-11-28T11:09:15.411968+0000","last_unstale":"2023-11-28T11:09:15.420417+0000","last_undegraded":"2023-11-28T11:09:15.420417+0000","last_fullsized":"2023-11-28T11:09:15.420417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:58.303509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419139+0000","last_change":"2023-11-28T11:09:15.410052+0000","last_active":"2023-11-28T11:09:15.419139+0000","last_peered":"2023-11-28T11:09:15.419139+0000","last_clean":"2023-11-28T11:09:15.419139+0000","last_became_active":"2023-11-28T11:09:15.409756+0000","last_became_peered":"2023-11-28T11:09:15.409756+0000","last_unstale":"2023-11-28T11:09:15.419139+0000","last_undegraded":"2023-11-28T11:09:15.419139+0000","last_fullsized":"2023-11-28T11:09:15.419139+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:15.621788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.400987+0000","last_change":"2023-11-28T11:09:15.412130+0000","last_active":"2023-11-28T11:09:16.400987+0000","last_peered":"2023-11-28T11:09:16.400987+0000","last_clean":"2023-11-28T11:09:16.400987+0000","last_became_active":"2023-11-28T11:09:15.411857+0000","last_became_peered":"2023-11-28T11:09:15.411857+0000","last_unstale":"2023-11-28T11:09:16.400987+0000","last_undegraded":"2023-11-28T11:09:16.400987+0000","last_fullsized":"2023-11-28T11:09:16.400987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:40:23.343099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419066+0000","last_change":"2023-11-28T11:09:15.415566+0000","last_active":"2023-11-28T11:09:15.419066+0000","last_peered":"2023-11-28T11:09:15.419066+0000","last_clean":"2023-11-28T11:09:15.419066+0000","last_became_active":"2023-11-28T11:09:15.415238+0000","last_became_peered":"2023-11-28T11:09:15.415238+0000","last_unstale":"2023-11-28T11:09:15.419066+0000","last_undegraded":"2023-11-28T11:09:15.419066+0000","last_fullsized":"2023-11-28T11:09:15.419066+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:57.597840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419139+0000","last_change":"2023-11-28T11:09:15.417534+0000","last_active":"2023-11-28T11:09:15.419139+0000","last_peered":"2023-11-28T11:09:15.419139+0000","last_clean":"2023-11-28T11:09:15.419139+0000","last_became_active":"2023-11-28T11:09:15.417283+0000","last_became_peered":"2023-11-28T11:09:15.417283+0000","last_unstale":"2023-11-28T11:09:15.419139+0000","last_undegraded":"2023-11-28T11:09:15.419139+0000","last_fullsized":"2023-11-28T11:09:15.419139+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:45.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.418593+0000","last_change":"2023-11-28T11:09:15.413667+0000","last_active":"2023-11-28T11:09:15.418593+0000","last_peered":"2023-11-28T11:09:15.418593+0000","last_clean":"2023-11-28T11:09:15.418593+0000","last_became_active":"2023-11-28T11:09:15.413381+0000","last_became_peered":"2023-11-28T11:09:15.413381+0000","last_unstale":"2023-11-28T11:09:15.418593+0000","last_undegraded":"2023-11-28T11:09:15.418593+0000","last_fullsized":"2023-11-28T11:09:15.418593+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:21.992851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.402053+0000","last_change":"2023-11-28T11:09:15.410784+0000","last_active":"2023-11-28T11:09:16.402053+0000","last_peered":"2023-11-28T11:09:16.402053+0000","last_clean":"2023-11-28T11:09:16.402053+0000","last_became_active":"2023-11-28T11:09:15.409919+0000","last_became_peered":"2023-11-28T11:09:15.409919+0000","last_unstale":"2023-11-28T11:09:16.402053+0000","last_undegraded":"2023-11-28T11:09:16.402053+0000","last_fullsized":"2023-11-28T11:09:16.402053+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:17.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.418645+0000","last_change":"2023-11-28T11:09:15.413244+0000","last_active":"2023-11-28T11:09:15.418645+0000","last_peered":"2023-11-28T11:09:15.418645+0000","last_clean":"2023-11-28T11:09:15.418645+0000","last_became_active":"2023-11-28T11:09:15.412870+0000","last_became_peered":"2023-11-28T11:09:15.412870+0000","last_unstale":"2023-11-28T11:09:15.418645+0000","last_undegraded":"2023-11-28T11:09:15.418645+0000","last_fullsized":"2023-11-28T11:09:15.418645+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:25.569113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.418618+0000","last_change":"2023-11-28T11:09:15.413865+0000","last_active":"2023-11-28T11:09:15.418618+0000","last_peered":"2023-11-28T11:09:15.418618+0000","last_clean":"2023-11-28T11:09:15.418618+0000","last_became_active":"2023-11-28T11:09:15.413588+0000","last_became_peered":"2023-11-28T11:09:15.413588+0000","last_unstale":"2023-11-28T11:09:15.418618+0000","last_undegraded":"2023-11-28T11:09:15.418618+0000","last_fullsized":"2023-11-28T11:09:15.418618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:44.329643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419160+0000","last_change":"2023-11-28T11:09:15.410548+0000","last_active":"2023-11-28T11:09:15.419160+0000","last_peered":"2023-11-28T11:09:15.419160+0000","last_clean":"2023-11-28T11:09:15.419160+0000","last_became_active":"2023-11-28T11:09:15.410271+0000","last_became_peered":"2023-11-28T11:09:15.410271+0000","last_unstale":"2023-11-28T11:09:15.419160+0000","last_undegraded":"2023-11-28T11:09:15.419160+0000","last_fullsized":"2023-11-28T11:09:15.419160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:00.185455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.413944+0000","last_change":"2023-11-28T11:09:15.410581+0000","last_active":"2023-11-28T11:09:15.413944+0000","last_peered":"2023-11-28T11:09:15.413944+0000","last_clean":"2023-11-28T11:09:15.413944+0000","last_became_active":"2023-11-28T11:09:15.410318+0000","last_became_peered":"2023-11-28T11:09:15.410318+0000","last_unstale":"2023-11-28T11:09:15.413944+0000","last_undegraded":"2023-11-28T11:09:15.413944+0000","last_fullsized":"2023-11-28T11:09:15.413944+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:29:57.451363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.426506+0000","last_change":"2023-11-28T11:09:15.424426+0000","last_active":"2023-11-28T11:09:15.426506+0000","last_peered":"2023-11-28T11:09:15.426506+0000","last_clean":"2023-11-28T11:09:15.426506+0000","last_became_active":"2023-11-28T11:09:15.424132+0000","last_became_peered":"2023-11-28T11:09:15.424132+0000","last_unstale":"2023-11-28T11:09:15.426506+0000","last_undegraded":"2023-11-28T11:09:15.426506+0000","last_fullsized":"2023-11-28T11:09:15.426506+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:52.365078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.427696+0000","last_change":"2023-11-28T11:09:15.009274+0000","last_active":"2023-11-28T11:09:15.427696+0000","last_peered":"2023-11-28T11:09:15.427696+0000","last_clean":"2023-11-28T11:09:15.427696+0000","last_became_active":"2023-11-28T11:09:15.008992+0000","last_became_peered":"2023-11-28T11:09:15.008992+0000","last_unstale":"2023-11-28T11:09:15.427696+0000","last_undegraded":"2023-11-28T11:09:15.427696+0000","last_fullsized":"2023-11-28T11:09:15.427696+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:59:24.742836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.419738+0000","last_change":"2023-11-28T11:09:15.419090+0000","last_active":"2023-11-28T11:09:15.419738+0000","last_peered":"2023-11-28T11:09:15.419738+0000","last_clean":"2023-11-28T11:09:15.419738+0000","last_became_active":"2023-11-28T11:09:15.418036+0000","last_became_peered":"2023-11-28T11:09:15.418036+0000","last_unstale":"2023-11-28T11:09:15.419738+0000","last_undegraded":"2023-11-28T11:09:15.419738+0000","last_fullsized":"2023-11-28T11:09:15.419738+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:15.621788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.410212+0000","last_change":"2023-11-28T11:09:15.015521+0000","last_active":"2023-11-28T11:09:15.410212+0000","last_peered":"2023-11-28T11:09:15.410212+0000","last_clean":"2023-11-28T11:09:15.410212+0000","last_became_active":"2023-11-28T11:09:15.015247+0000","last_became_peered":"2023-11-28T11:09:15.015247+0000","last_unstale":"2023-11-28T11:09:15.410212+0000","last_undegraded":"2023-11-28T11:09:15.410212+0000","last_fullsized":"2023-11-28T11:09:15.410212+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:56.596199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.427179+0000","last_change":"2023-11-28T11:09:15.424585+0000","last_active":"2023-11-28T11:09:15.427179+0000","last_peered":"2023-11-28T11:09:15.427179+0000","last_clean":"2023-11-28T11:09:15.427179+0000","last_became_active":"2023-11-28T11:09:15.424315+0000","last_became_peered":"2023-11-28T11:09:15.424315+0000","last_unstale":"2023-11-28T11:09:15.427179+0000","last_undegraded":"2023-11-28T11:09:15.427179+0000","last_fullsized":"2023-11-28T11:09:15.427179+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:46:03.407983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.423406+0000","last_change":"2023-11-28T11:09:15.013936+0000","last_active":"2023-11-28T11:09:15.423406+0000","last_peered":"2023-11-28T11:09:15.423406+0000","last_clean":"2023-11-28T11:09:15.423406+0000","last_became_active":"2023-11-28T11:09:15.013533+0000","last_became_peered":"2023-11-28T11:09:15.013533+0000","last_unstale":"2023-11-28T11:09:15.423406+0000","last_undegraded":"2023-11-28T11:09:15.423406+0000","last_fullsized":"2023-11-28T11:09:15.423406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:24.567472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396626+0000","last_change":"2023-11-28T11:09:15.416355+0000","last_active":"2023-11-28T11:09:16.396626+0000","last_peered":"2023-11-28T11:09:16.396626+0000","last_clean":"2023-11-28T11:09:16.396626+0000","last_became_active":"2023-11-28T11:09:15.416076+0000","last_became_peered":"2023-11-28T11:09:15.416076+0000","last_unstale":"2023-11-28T11:09:16.396626+0000","last_undegraded":"2023-11-28T11:09:16.396626+0000","last_fullsized":"2023-11-28T11:09:16.396626+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:01.481847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.424141+0000","last_change":"2023-11-28T11:09:15.010421+0000","last_active":"2023-11-28T11:09:15.424141+0000","last_peered":"2023-11-28T11:09:15.424141+0000","last_clean":"2023-11-28T11:09:15.424141+0000","last_became_active":"2023-11-28T11:09:15.010125+0000","last_became_peered":"2023-11-28T11:09:15.010125+0000","last_unstale":"2023-11-28T11:09:15.424141+0000","last_undegraded":"2023-11-28T11:09:15.424141+0000","last_fullsized":"2023-11-28T11:09:15.424141+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:39.094822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.405913+0000","last_change":"2023-11-28T11:09:15.403018+0000","last_active":"2023-11-28T11:09:15.405913+0000","last_peered":"2023-11-28T11:09:15.405913+0000","last_clean":"2023-11-28T11:09:15.405913+0000","last_became_active":"2023-11-28T11:09:15.402753+0000","last_became_peered":"2023-11-28T11:09:15.402753+0000","last_unstale":"2023-11-28T11:09:15.405913+0000","last_undegraded":"2023-11-28T11:09:15.405913+0000","last_fullsized":"2023-11-28T11:09:15.405913+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:17.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401755+0000","last_change":"2023-11-28T11:09:15.016338+0000","last_active":"2023-11-28T11:09:16.401755+0000","last_peered":"2023-11-28T11:09:16.401755+0000","last_clean":"2023-11-28T11:09:16.401755+0000","last_became_active":"2023-11-28T11:09:15.016036+0000","last_became_peered":"2023-11-28T11:09:15.016036+0000","last_unstale":"2023-11-28T11:09:16.401755+0000","last_undegraded":"2023-11-28T11:09:16.401755+0000","last_fullsized":"2023-11-28T11:09:16.401755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:56.596199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401843+0000","last_change":"2023-11-28T11:09:15.404047+0000","last_active":"2023-11-28T11:09:16.401843+0000","last_peered":"2023-11-28T11:09:16.401843+0000","last_clean":"2023-11-28T11:09:16.401843+0000","last_became_active":"2023-11-28T11:09:15.403721+0000","last_became_peered":"2023-11-28T11:09:15.403721+0000","last_unstale":"2023-11-28T11:09:16.401843+0000","last_undegraded":"2023-11-28T11:09:16.401843+0000","last_fullsized":"2023-11-28T11:09:16.401843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:03.492691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.422752+0000","last_change":"2023-11-28T11:09:15.014947+0000","last_active":"2023-11-28T11:09:15.422752+0000","last_peered":"2023-11-28T11:09:15.422752+0000","last_clean":"2023-11-28T11:09:15.422752+0000","last_became_active":"2023-11-28T11:09:15.014635+0000","last_became_peered":"2023-11-28T11:09:15.014635+0000","last_unstale":"2023-11-28T11:09:15.422752+0000","last_undegraded":"2023-11-28T11:09:15.422752+0000","last_fullsized":"2023-11-28T11:09:15.422752+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:40.901913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.425689+0000","last_change":"2023-11-28T11:09:15.422433+0000","last_active":"2023-11-28T11:09:15.425689+0000","last_peered":"2023-11-28T11:09:15.425689+0000","last_clean":"2023-11-28T11:09:15.425689+0000","last_became_active":"2023-11-28T11:09:15.422028+0000","last_became_peered":"2023-11-28T11:09:15.422028+0000","last_unstale":"2023-11-28T11:09:15.425689+0000","last_undegraded":"2023-11-28T11:09:15.425689+0000","last_fullsized":"2023-11-28T11:09:15.425689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:16:12.091426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.400427+0000","last_change":"2023-11-28T11:09:15.014440+0000","last_active":"2023-11-28T11:09:15.400427+0000","last_peered":"2023-11-28T11:09:15.400427+0000","last_clean":"2023-11-28T11:09:15.400427+0000","last_became_active":"2023-11-28T11:09:15.014141+0000","last_became_peered":"2023-11-28T11:09:15.014141+0000","last_unstale":"2023-11-28T11:09:15.400427+0000","last_undegraded":"2023-11-28T11:09:15.400427+0000","last_fullsized":"2023-11-28T11:09:15.400427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:15.572173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396196+0000","last_change":"2023-11-28T11:09:15.412690+0000","last_active":"2023-11-28T11:09:16.396196+0000","last_peered":"2023-11-28T11:09:16.396196+0000","last_clean":"2023-11-28T11:09:16.396196+0000","last_became_active":"2023-11-28T11:09:15.412439+0000","last_became_peered":"2023-11-28T11:09:15.412439+0000","last_unstale":"2023-11-28T11:09:16.396196+0000","last_undegraded":"2023-11-28T11:09:16.396196+0000","last_fullsized":"2023-11-28T11:09:16.396196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:36.493873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.405540+0000","last_change":"2023-11-28T11:09:15.009687+0000","last_active":"2023-11-28T11:09:15.405540+0000","last_peered":"2023-11-28T11:09:15.405540+0000","last_clean":"2023-11-28T11:09:15.405540+0000","last_became_active":"2023-11-28T11:09:15.009406+0000","last_became_peered":"2023-11-28T11:09:15.009406+0000","last_unstale":"2023-11-28T11:09:15.405540+0000","last_undegraded":"2023-11-28T11:09:15.405540+0000","last_fullsized":"2023-11-28T11:09:15.405540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:30:48.725285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401257+0000","last_change":"2023-11-28T11:09:15.415648+0000","last_active":"2023-11-28T11:09:16.401257+0000","last_peered":"2023-11-28T11:09:16.401257+0000","last_clean":"2023-11-28T11:09:16.401257+0000","last_became_active":"2023-11-28T11:09:15.415385+0000","last_became_peered":"2023-11-28T11:09:15.415385+0000","last_unstale":"2023-11-28T11:09:16.401257+0000","last_undegraded":"2023-11-28T11:09:16.401257+0000","last_fullsized":"2023-11-28T11:09:16.401257+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:44.329643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396654+0000","last_change":"2023-11-28T11:09:15.018782+0000","last_active":"2023-11-28T11:09:16.396654+0000","last_peered":"2023-11-28T11:09:16.396654+0000","last_clean":"2023-11-28T11:09:16.396654+0000","last_became_active":"2023-11-28T11:09:15.018519+0000","last_became_peered":"2023-11-28T11:09:15.018519+0000","last_unstale":"2023-11-28T11:09:16.396654+0000","last_undegraded":"2023-11-28T11:09:16.396654+0000","last_fullsized":"2023-11-28T11:09:16.396654+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:44:39.601790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.420177+0000","last_change":"2023-11-28T11:09:15.416595+0000","last_active":"2023-11-28T11:09:15.420177+0000","last_peered":"2023-11-28T11:09:15.420177+0000","last_clean":"2023-11-28T11:09:15.420177+0000","last_became_active":"2023-11-28T11:09:15.416286+0000","last_became_peered":"2023-11-28T11:09:15.416286+0000","last_unstale":"2023-11-28T11:09:15.420177+0000","last_undegraded":"2023-11-28T11:09:15.420177+0000","last_fullsized":"2023-11-28T11:09:15.420177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:40.096463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401942+0000","last_change":"2023-11-28T11:09:15.011308+0000","last_active":"2023-11-28T11:09:16.401942+0000","last_peered":"2023-11-28T11:09:16.401942+0000","last_clean":"2023-11-28T11:09:16.401942+0000","last_became_active":"2023-11-28T11:09:15.010987+0000","last_became_peered":"2023-11-28T11:09:15.010987+0000","last_unstale":"2023-11-28T11:09:16.401942+0000","last_undegraded":"2023-11-28T11:09:16.401942+0000","last_fullsized":"2023-11-28T11:09:16.401942+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:43.328002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.414451+0000","last_change":"2023-11-28T11:09:15.411073+0000","last_active":"2023-11-28T11:09:15.414451+0000","last_peered":"2023-11-28T11:09:15.414451+0000","last_clean":"2023-11-28T11:09:15.414451+0000","last_became_active":"2023-11-28T11:09:15.410117+0000","last_became_peered":"2023-11-28T11:09:15.410117+0000","last_unstale":"2023-11-28T11:09:15.414451+0000","last_undegraded":"2023-11-28T11:09:15.414451+0000","last_fullsized":"2023-11-28T11:09:15.414451+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:15.621788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401890+0000","last_change":"2023-11-28T11:09:15.015564+0000","last_active":"2023-11-28T11:09:16.401890+0000","last_peered":"2023-11-28T11:09:16.401890+0000","last_clean":"2023-11-28T11:09:16.401890+0000","last_became_active":"2023-11-28T11:09:15.015293+0000","last_became_peered":"2023-11-28T11:09:15.015293+0000","last_unstale":"2023-11-28T11:09:16.401890+0000","last_undegraded":"2023-11-28T11:09:16.401890+0000","last_fullsized":"2023-11-28T11:09:16.401890+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:08.798600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.424649+0000","last_change":"2023-11-28T11:09:15.420136+0000","last_active":"2023-11-28T11:09:15.424649+0000","last_peered":"2023-11-28T11:09:15.424649+0000","last_clean":"2023-11-28T11:09:15.424649+0000","last_became_active":"2023-11-28T11:09:15.419884+0000","last_became_peered":"2023-11-28T11:09:15.419884+0000","last_unstale":"2023-11-28T11:09:15.424649+0000","last_undegraded":"2023-11-28T11:09:15.424649+0000","last_fullsized":"2023-11-28T11:09:15.424649+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:03.503649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396230+0000","last_change":"2023-11-28T11:09:15.014485+0000","last_active":"2023-11-28T11:09:16.396230+0000","last_peered":"2023-11-28T11:09:16.396230+0000","last_clean":"2023-11-28T11:09:16.396230+0000","last_became_active":"2023-11-28T11:09:15.014206+0000","last_became_peered":"2023-11-28T11:09:15.014206+0000","last_unstale":"2023-11-28T11:09:16.396230+0000","last_undegraded":"2023-11-28T11:09:16.396230+0000","last_fullsized":"2023-11-28T11:09:16.396230+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:40:22.341458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.411211+0000","last_change":"2023-11-28T11:09:15.409280+0000","last_active":"2023-11-28T11:09:15.411211+0000","last_peered":"2023-11-28T11:09:15.411211+0000","last_clean":"2023-11-28T11:09:15.411211+0000","last_became_active":"2023-11-28T11:09:15.409032+0000","last_became_peered":"2023-11-28T11:09:15.409032+0000","last_unstale":"2023-11-28T11:09:15.411211+0000","last_undegraded":"2023-11-28T11:09:15.411211+0000","last_fullsized":"2023-11-28T11:09:15.411211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:45.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396178+0000","last_change":"2023-11-28T11:09:15.009929+0000","last_active":"2023-11-28T11:09:16.396178+0000","last_peered":"2023-11-28T11:09:16.396178+0000","last_clean":"2023-11-28T11:09:16.396178+0000","last_became_active":"2023-11-28T11:09:15.009432+0000","last_became_peered":"2023-11-28T11:09:15.009432+0000","last_unstale":"2023-11-28T11:09:16.396178+0000","last_undegraded":"2023-11-28T11:09:16.396178+0000","last_fullsized":"2023-11-28T11:09:16.396178+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:27:59.533490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396007+0000","last_change":"2023-11-28T11:09:15.400798+0000","last_active":"2023-11-28T11:09:16.396007+0000","last_peered":"2023-11-28T11:09:16.396007+0000","last_clean":"2023-11-28T11:09:16.396007+0000","last_became_active":"2023-11-28T11:09:15.400537+0000","last_became_peered":"2023-11-28T11:09:15.400537+0000","last_unstale":"2023-11-28T11:09:16.396007+0000","last_undegraded":"2023-11-28T11:09:16.396007+0000","last_fullsized":"2023-11-28T11:09:16.396007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:26:42.661150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.400247+0000","last_change":"2023-11-28T11:09:15.014548+0000","last_active":"2023-11-28T11:09:15.400247+0000","last_peered":"2023-11-28T11:09:15.400247+0000","last_clean":"2023-11-28T11:09:15.400247+0000","last_became_active":"2023-11-28T11:09:15.014227+0000","last_became_peered":"2023-11-28T11:09:15.014227+0000","last_unstale":"2023-11-28T11:09:15.400247+0000","last_undegraded":"2023-11-28T11:09:15.400247+0000","last_fullsized":"2023-11-28T11:09:15.400247+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:40.901913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.418788+0000","last_change":"2023-11-28T11:09:15.409322+0000","last_active":"2023-11-28T11:09:15.418788+0000","last_peered":"2023-11-28T11:09:15.418788+0000","last_clean":"2023-11-28T11:09:15.418788+0000","last_became_active":"2023-11-28T11:09:15.408709+0000","last_became_peered":"2023-11-28T11:09:15.408709+0000","last_unstale":"2023-11-28T11:09:15.418788+0000","last_undegraded":"2023-11-28T11:09:15.418788+0000","last_fullsized":"2023-11-28T11:09:15.418788+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:30:49.726926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.403063+0000","last_change":"2023-11-28T11:09:15.009058+0000","last_active":"2023-11-28T11:09:15.403063+0000","last_peered":"2023-11-28T11:09:15.403063+0000","last_clean":"2023-11-28T11:09:15.403063+0000","last_became_active":"2023-11-28T11:09:15.008742+0000","last_became_peered":"2023-11-28T11:09:15.008742+0000","last_unstale":"2023-11-28T11:09:15.403063+0000","last_undegraded":"2023-11-28T11:09:15.403063+0000","last_fullsized":"2023-11-28T11:09:15.403063+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:41.576941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396042+0000","last_change":"2023-11-28T11:09:15.400595+0000","last_active":"2023-11-28T11:09:16.396042+0000","last_peered":"2023-11-28T11:09:16.396042+0000","last_clean":"2023-11-28T11:09:16.396042+0000","last_became_active":"2023-11-28T11:09:15.400320+0000","last_became_peered":"2023-11-28T11:09:15.400320+0000","last_unstale":"2023-11-28T11:09:16.396042+0000","last_undegraded":"2023-11-28T11:09:16.396042+0000","last_fullsized":"2023-11-28T11:09:16.396042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:03.492691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.426614+0000","last_change":"2023-11-28T11:09:15.014748+0000","last_active":"2023-11-28T11:09:15.426614+0000","last_peered":"2023-11-28T11:09:15.426614+0000","last_clean":"2023-11-28T11:09:15.426614+0000","last_became_active":"2023-11-28T11:09:15.014357+0000","last_became_peered":"2023-11-28T11:09:15.014357+0000","last_unstale":"2023-11-28T11:09:15.426614+0000","last_undegraded":"2023-11-28T11:09:15.426614+0000","last_fullsized":"2023-11-28T11:09:15.426614+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:20.991210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396145+0000","last_change":"2023-11-28T11:09:15.413732+0000","last_active":"2023-11-28T11:09:16.396145+0000","last_peered":"2023-11-28T11:09:16.396145+0000","last_clean":"2023-11-28T11:09:16.396145+0000","last_became_active":"2023-11-28T11:09:15.413494+0000","last_became_peered":"2023-11-28T11:09:15.413494+0000","last_unstale":"2023-11-28T11:09:16.396145+0000","last_undegraded":"2023-11-28T11:09:16.396145+0000","last_fullsized":"2023-11-28T11:09:16.396145+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:21:03.120132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401836+0000","last_change":"2023-11-28T11:09:15.010927+0000","last_active":"2023-11-28T11:09:16.401836+0000","last_peered":"2023-11-28T11:09:16.401836+0000","last_clean":"2023-11-28T11:09:16.401836+0000","last_became_active":"2023-11-28T11:09:15.010575+0000","last_became_peered":"2023-11-28T11:09:15.010575+0000","last_unstale":"2023-11-28T11:09:16.401836+0000","last_undegraded":"2023-11-28T11:09:16.401836+0000","last_fullsized":"2023-11-28T11:09:16.401836+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:24.567472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401643+0000","last_change":"2023-11-28T11:09:15.410705+0000","last_active":"2023-11-28T11:09:16.401643+0000","last_peered":"2023-11-28T11:09:16.401643+0000","last_clean":"2023-11-28T11:09:16.401643+0000","last_became_active":"2023-11-28T11:09:15.410443+0000","last_became_peered":"2023-11-28T11:09:15.410443+0000","last_unstale":"2023-11-28T11:09:16.401643+0000","last_undegraded":"2023-11-28T11:09:16.401643+0000","last_fullsized":"2023-11-28T11:09:16.401643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:03.503649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.427128+0000","last_change":"2023-11-28T11:09:15.014109+0000","last_active":"2023-11-28T11:09:15.427128+0000","last_peered":"2023-11-28T11:09:15.427128+0000","last_clean":"2023-11-28T11:09:15.427128+0000","last_became_active":"2023-11-28T11:09:15.013834+0000","last_became_peered":"2023-11-28T11:09:15.013834+0000","last_unstale":"2023-11-28T11:09:15.427128+0000","last_undegraded":"2023-11-28T11:09:15.427128+0000","last_fullsized":"2023-11-28T11:09:15.427128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:51.363437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.424160+0000","last_change":"2023-11-28T11:09:15.415149+0000","last_active":"2023-11-28T11:09:15.424160+0000","last_peered":"2023-11-28T11:09:15.424160+0000","last_clean":"2023-11-28T11:09:15.424160+0000","last_became_active":"2023-11-28T11:09:15.414602+0000","last_became_peered":"2023-11-28T11:09:15.414602+0000","last_unstale":"2023-11-28T11:09:15.424160+0000","last_undegraded":"2023-11-28T11:09:15.424160+0000","last_fullsized":"2023-11-28T11:09:15.424160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:47:10.507056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401177+0000","last_change":"2023-11-28T11:09:15.015937+0000","last_active":"2023-11-28T11:09:16.401177+0000","last_peered":"2023-11-28T11:09:16.401177+0000","last_clean":"2023-11-28T11:09:16.401177+0000","last_became_active":"2023-11-28T11:09:15.015651+0000","last_became_peered":"2023-11-28T11:09:15.015651+0000","last_unstale":"2023-11-28T11:09:16.401177+0000","last_undegraded":"2023-11-28T11:09:16.401177+0000","last_fullsized":"2023-11-28T11:09:16.401177+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:39.332888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401349+0000","last_change":"2023-11-28T11:09:15.414480+0000","last_active":"2023-11-28T11:09:16.401349+0000","last_peered":"2023-11-28T11:09:16.401349+0000","last_clean":"2023-11-28T11:09:16.401349+0000","last_became_active":"2023-11-28T11:09:15.414206+0000","last_became_peered":"2023-11-28T11:09:15.414206+0000","last_unstale":"2023-11-28T11:09:16.401349+0000","last_undegraded":"2023-11-28T11:09:16.401349+0000","last_fullsized":"2023-11-28T11:09:16.401349+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:52.222986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.400023+0000","last_change":"2023-11-28T11:09:15.014777+0000","last_active":"2023-11-28T11:09:15.400023+0000","last_peered":"2023-11-28T11:09:15.400023+0000","last_clean":"2023-11-28T11:09:15.400023+0000","last_became_active":"2023-11-28T11:09:15.014532+0000","last_became_peered":"2023-11-28T11:09:15.014532+0000","last_unstale":"2023-11-28T11:09:15.400023+0000","last_undegraded":"2023-11-28T11:09:15.400023+0000","last_fullsized":"2023-11-28T11:09:15.400023+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:56.596199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.420148+0000","last_change":"2023-11-28T11:09:15.417774+0000","last_active":"2023-11-28T11:09:15.420148+0000","last_peered":"2023-11-28T11:09:15.420148+0000","last_clean":"2023-11-28T11:09:15.420148+0000","last_became_active":"2023-11-28T11:09:15.413759+0000","last_became_peered":"2023-11-28T11:09:15.413759+0000","last_unstale":"2023-11-28T11:09:15.420148+0000","last_undegraded":"2023-11-28T11:09:15.420148+0000","last_fullsized":"2023-11-28T11:09:15.420148+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:33.228932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401146+0000","last_change":"2023-11-28T11:09:15.015872+0000","last_active":"2023-11-28T11:09:16.401146+0000","last_peered":"2023-11-28T11:09:16.401146+0000","last_clean":"2023-11-28T11:09:16.401146+0000","last_became_active":"2023-11-28T11:09:15.015605+0000","last_became_peered":"2023-11-28T11:09:15.015605+0000","last_unstale":"2023-11-28T11:09:16.401146+0000","last_undegraded":"2023-11-28T11:09:16.401146+0000","last_fullsized":"2023-11-28T11:09:16.401146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:53.361811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.414410+0000","last_change":"2023-11-28T11:09:15.411044+0000","last_active":"2023-11-28T11:09:15.414410+0000","last_peered":"2023-11-28T11:09:15.414410+0000","last_clean":"2023-11-28T11:09:15.414410+0000","last_became_active":"2023-11-28T11:09:15.410160+0000","last_became_peered":"2023-11-28T11:09:15.410160+0000","last_unstale":"2023-11-28T11:09:15.414410+0000","last_undegraded":"2023-11-28T11:09:15.414410+0000","last_fullsized":"2023-11-28T11:09:15.414410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:40.096463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401098+0000","last_change":"2023-11-28T11:09:15.011836+0000","last_active":"2023-11-28T11:09:16.401098+0000","last_peered":"2023-11-28T11:09:16.401098+0000","last_clean":"2023-11-28T11:09:16.401098+0000","last_became_active":"2023-11-28T11:09:15.011524+0000","last_became_peered":"2023-11-28T11:09:15.011524+0000","last_unstale":"2023-11-28T11:09:16.401098+0000","last_undegraded":"2023-11-28T11:09:16.401098+0000","last_fullsized":"2023-11-28T11:09:16.401098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:31:33.493593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401418+0000","last_change":"2023-11-28T11:09:15.416487+0000","last_active":"2023-11-28T11:09:16.401418+0000","last_peered":"2023-11-28T11:09:16.401418+0000","last_clean":"2023-11-28T11:09:16.401418+0000","last_became_active":"2023-11-28T11:09:15.416207+0000","last_became_peered":"2023-11-28T11:09:15.416207+0000","last_unstale":"2023-11-28T11:09:16.401418+0000","last_undegraded":"2023-11-28T11:09:16.401418+0000","last_fullsized":"2023-11-28T11:09:16.401418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:30:49.726926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.407603+0000","last_change":"2023-11-28T11:09:15.014756+0000","last_active":"2023-11-28T11:09:15.407603+0000","last_peered":"2023-11-28T11:09:15.407603+0000","last_clean":"2023-11-28T11:09:15.407603+0000","last_became_active":"2023-11-28T11:09:15.014480+0000","last_became_peered":"2023-11-28T11:09:15.014480+0000","last_unstale":"2023-11-28T11:09:15.407603+0000","last_undegraded":"2023-11-28T11:09:15.407603+0000","last_fullsized":"2023-11-28T11:09:15.407603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:02:27.855713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396077+0000","last_change":"2023-11-28T11:09:15.416362+0000","last_active":"2023-11-28T11:09:16.396077+0000","last_peered":"2023-11-28T11:09:16.396077+0000","last_clean":"2023-11-28T11:09:16.396077+0000","last_became_active":"2023-11-28T11:09:15.416111+0000","last_became_peered":"2023-11-28T11:09:15.416111+0000","last_unstale":"2023-11-28T11:09:16.396077+0000","last_undegraded":"2023-11-28T11:09:16.396077+0000","last_fullsized":"2023-11-28T11:09:16.396077+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:03.503649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.400976+0000","last_change":"2023-11-28T11:09:15.011238+0000","last_active":"2023-11-28T11:09:16.400976+0000","last_peered":"2023-11-28T11:09:16.400976+0000","last_clean":"2023-11-28T11:09:16.400976+0000","last_became_active":"2023-11-28T11:09:15.010828+0000","last_became_peered":"2023-11-28T11:09:15.010828+0000","last_unstale":"2023-11-28T11:09:16.400976+0000","last_undegraded":"2023-11-28T11:09:16.400976+0000","last_fullsized":"2023-11-28T11:09:16.400976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:59:24.742836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.401633+0000","last_change":"2023-11-28T11:09:15.399704+0000","last_active":"2023-11-28T11:09:15.401633+0000","last_peered":"2023-11-28T11:09:15.401633+0000","last_clean":"2023-11-28T11:09:15.401633+0000","last_became_active":"2023-11-28T11:09:15.399457+0000","last_became_peered":"2023-11-28T11:09:15.399457+0000","last_unstale":"2023-11-28T11:09:15.401633+0000","last_undegraded":"2023-11-28T11:09:15.401633+0000","last_fullsized":"2023-11-28T11:09:15.401633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:30:49.726926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.398569+0000","last_change":"2023-11-28T11:09:15.010635+0000","last_active":"2023-11-28T11:09:15.398569+0000","last_peered":"2023-11-28T11:09:15.398569+0000","last_clean":"2023-11-28T11:09:15.398569+0000","last_became_active":"2023-11-28T11:09:15.010334+0000","last_became_peered":"2023-11-28T11:09:15.010334+0000","last_unstale":"2023-11-28T11:09:15.398569+0000","last_undegraded":"2023-11-28T11:09:15.398569+0000","last_fullsized":"2023-11-28T11:09:15.398569+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:43.328002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.018564+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:09:15.018564+0000","last_peered":"2023-11-28T11:09:15.018564+0000","last_clean":"2023-11-28T11:09:15.018564+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:09:15.018564+0000","last_undegraded":"2023-11-28T11:09:15.018564+0000","last_fullsized":"2023-11-28T11:09:15.018564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.396236+0000","last_change":"2023-11-28T11:09:15.414033+0000","last_active":"2023-11-28T11:09:16.396236+0000","last_peered":"2023-11-28T11:09:16.396236+0000","last_clean":"2023-11-28T11:09:16.396236+0000","last_became_active":"2023-11-28T11:09:15.413794+0000","last_became_peered":"2023-11-28T11:09:15.413794+0000","last_unstale":"2023-11-28T11:09:16.396236+0000","last_undegraded":"2023-11-28T11:09:16.396236+0000","last_fullsized":"2023-11-28T11:09:16.396236+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:17.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.398595+0000","last_change":"2023-11-28T11:09:15.018622+0000","last_active":"2023-11-28T11:09:15.398595+0000","last_peered":"2023-11-28T11:09:15.398595+0000","last_clean":"2023-11-28T11:09:15.398595+0000","last_became_active":"2023-11-28T11:09:15.018356+0000","last_became_peered":"2023-11-28T11:09:15.018356+0000","last_unstale":"2023-11-28T11:09:15.398595+0000","last_undegraded":"2023-11-28T11:09:15.398595+0000","last_fullsized":"2023-11-28T11:09:15.398595+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:15.572173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.402323+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:09:15.402323+0000","last_peered":"2023-11-28T11:09:15.402323+0000","last_clean":"2023-11-28T11:09:15.402323+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:09:15.402323+0000","last_undegraded":"2023-11-28T11:09:15.402323+0000","last_fullsized":"2023-11-28T11:09:15.402323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.420195+0000","last_change":"2023-11-28T11:09:15.418111+0000","last_active":"2023-11-28T11:09:15.420195+0000","last_peered":"2023-11-28T11:09:15.420195+0000","last_clean":"2023-11-28T11:09:15.420195+0000","last_became_active":"2023-11-28T11:09:15.417782+0000","last_became_peered":"2023-11-28T11:09:15.417782+0000","last_unstale":"2023-11-28T11:09:15.420195+0000","last_undegraded":"2023-11-28T11:09:15.420195+0000","last_fullsized":"2023-11-28T11:09:15.420195+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:45.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401289+0000","last_change":"2023-11-28T11:09:15.012228+0000","last_active":"2023-11-28T11:09:16.401289+0000","last_peered":"2023-11-28T11:09:16.401289+0000","last_clean":"2023-11-28T11:09:16.401289+0000","last_became_active":"2023-11-28T11:09:15.011934+0000","last_became_peered":"2023-11-28T11:09:15.011934+0000","last_unstale":"2023-11-28T11:09:16.401289+0000","last_undegraded":"2023-11-28T11:09:16.401289+0000","last_fullsized":"2023-11-28T11:09:16.401289+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:58:15.998627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.405133+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:09:15.405133+0000","last_peered":"2023-11-28T11:09:15.405133+0000","last_clean":"2023-11-28T11:09:15.405133+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:09:15.405133+0000","last_undegraded":"2023-11-28T11:09:15.405133+0000","last_fullsized":"2023-11-28T11:09:15.405133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401830+0000","last_change":"2023-11-28T11:09:15.415879+0000","last_active":"2023-11-28T11:09:16.401830+0000","last_peered":"2023-11-28T11:09:16.401830+0000","last_clean":"2023-11-28T11:09:16.401830+0000","last_became_active":"2023-11-28T11:09:15.415655+0000","last_became_peered":"2023-11-28T11:09:15.415655+0000","last_unstale":"2023-11-28T11:09:16.401830+0000","last_undegraded":"2023-11-28T11:09:16.401830+0000","last_fullsized":"2023-11-28T11:09:16.401830+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:21:03.120132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.406908+0000","last_change":"2023-11-28T11:09:15.015486+0000","last_active":"2023-11-28T11:09:15.406908+0000","last_peered":"2023-11-28T11:09:15.406908+0000","last_clean":"2023-11-28T11:09:15.406908+0000","last_became_active":"2023-11-28T11:09:15.015210+0000","last_became_peered":"2023-11-28T11:09:15.015210+0000","last_unstale":"2023-11-28T11:09:15.406908+0000","last_undegraded":"2023-11-28T11:09:15.406908+0000","last_fullsized":"2023-11-28T11:09:15.406908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:08.798600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.402079+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:09:15.402079+0000","last_peered":"2023-11-28T11:09:15.402079+0000","last_clean":"2023-11-28T11:09:15.402079+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:09:15.402079+0000","last_undegraded":"2023-11-28T11:09:15.402079+0000","last_fullsized":"2023-11-28T11:09:15.402079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.005655+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:09:15.005655+0000","last_peered":"2023-11-28T11:09:15.005655+0000","last_clean":"2023-11-28T11:09:15.005655+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:09:15.005655+0000","last_undegraded":"2023-11-28T11:09:15.005655+0000","last_fullsized":"2023-11-28T11:09:15.005655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.418417+0000","last_change":"2023-11-28T11:09:15.417579+0000","last_active":"2023-11-28T11:09:15.418417+0000","last_peered":"2023-11-28T11:09:15.418417+0000","last_clean":"2023-11-28T11:09:15.418417+0000","last_became_active":"2023-11-28T11:09:15.417326+0000","last_became_peered":"2023-11-28T11:09:15.417326+0000","last_unstale":"2023-11-28T11:09:15.418417+0000","last_undegraded":"2023-11-28T11:09:15.418417+0000","last_fullsized":"2023-11-28T11:09:15.418417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:55:05.158077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.401943+0000","last_change":"2023-11-28T11:09:15.014272+0000","last_active":"2023-11-28T11:09:16.401943+0000","last_peered":"2023-11-28T11:09:16.401943+0000","last_clean":"2023-11-28T11:09:16.401943+0000","last_became_active":"2023-11-28T11:09:15.014022+0000","last_became_peered":"2023-11-28T11:09:15.014022+0000","last_unstale":"2023-11-28T11:09:16.401943+0000","last_undegraded":"2023-11-28T11:09:16.401943+0000","last_fullsized":"2023-11-28T11:09:16.401943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:15.572173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.005742+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:09:15.005742+0000","last_peered":"2023-11-28T11:09:15.005742+0000","last_clean":"2023-11-28T11:09:15.005742+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:09:15.005742+0000","last_undegraded":"2023-11-28T11:09:15.005742+0000","last_fullsized":"2023-11-28T11:09:15.005742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.417784+0000","last_change":"2023-11-28T11:09:15.416594+0000","last_active":"2023-11-28T11:09:15.417784+0000","last_peered":"2023-11-28T11:09:15.417784+0000","last_clean":"2023-11-28T11:09:15.417784+0000","last_became_active":"2023-11-28T11:09:15.416340+0000","last_became_peered":"2023-11-28T11:09:15.416340+0000","last_unstale":"2023-11-28T11:09:15.417784+0000","last_undegraded":"2023-11-28T11:09:15.417784+0000","last_fullsized":"2023-11-28T11:09:15.417784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:33.228932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.401381+0000","last_change":"2023-11-28T11:09:15.010097+0000","last_active":"2023-11-28T11:09:15.401381+0000","last_peered":"2023-11-28T11:09:15.401381+0000","last_clean":"2023-11-28T11:09:15.401381+0000","last_became_active":"2023-11-28T11:09:15.009810+0000","last_became_peered":"2023-11-28T11:09:15.009810+0000","last_unstale":"2023-11-28T11:09:15.401381+0000","last_undegraded":"2023-11-28T11:09:15.401381+0000","last_fullsized":"2023-11-28T11:09:15.401381+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:35.910146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.006144+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:09:15.006144+0000","last_peered":"2023-11-28T11:09:15.006144+0000","last_clean":"2023-11-28T11:09:15.006144+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:09:15.006144+0000","last_undegraded":"2023-11-28T11:09:15.006144+0000","last_fullsized":"2023-11-28T11:09:15.006144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.424202+0000","last_change":"2023-11-28T11:09:15.421691+0000","last_active":"2023-11-28T11:09:15.424202+0000","last_peered":"2023-11-28T11:09:15.424202+0000","last_clean":"2023-11-28T11:09:15.424202+0000","last_became_active":"2023-11-28T11:09:15.421457+0000","last_became_peered":"2023-11-28T11:09:15.421457+0000","last_unstale":"2023-11-28T11:09:15.424202+0000","last_undegraded":"2023-11-28T11:09:15.424202+0000","last_fullsized":"2023-11-28T11:09:15.424202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:46.016400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.405899+0000","last_change":"2023-11-28T11:09:15.010103+0000","last_active":"2023-11-28T11:09:15.405899+0000","last_peered":"2023-11-28T11:09:15.405899+0000","last_clean":"2023-11-28T11:09:15.405899+0000","last_became_active":"2023-11-28T11:09:15.009846+0000","last_became_peered":"2023-11-28T11:09:15.009846+0000","last_unstale":"2023-11-28T11:09:15.405899+0000","last_undegraded":"2023-11-28T11:09:15.405899+0000","last_fullsized":"2023-11-28T11:09:15.405899+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:59.183814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.018878+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:09:15.018878+0000","last_peered":"2023-11-28T11:09:15.018878+0000","last_clean":"2023-11-28T11:09:15.018878+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:09:15.018878+0000","last_undegraded":"2023-11-28T11:09:15.018878+0000","last_fullsized":"2023-11-28T11:09:15.018878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.421555+0000","last_change":"2023-11-28T11:09:15.416581+0000","last_active":"2023-11-28T11:09:15.421555+0000","last_peered":"2023-11-28T11:09:15.421555+0000","last_clean":"2023-11-28T11:09:15.421555+0000","last_became_active":"2023-11-28T11:09:15.416310+0000","last_became_peered":"2023-11-28T11:09:15.416310+0000","last_unstale":"2023-11-28T11:09:15.421555+0000","last_undegraded":"2023-11-28T11:09:15.421555+0000","last_fullsized":"2023-11-28T11:09:15.421555+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:14.989423+0000","last_clean_scrub_stamp":"2023-11-28T11:09:14.989423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:55:05.158077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T11:09:16.395944+0000","last_change":"2023-11-28T11:09:15.018841+0000","last_active":"2023-11-28T11:09:16.395944+0000","last_peered":"2023-11-28T11:09:16.395944+0000","last_clean":"2023-11-28T11:09:16.395944+0000","last_became_active":"2023-11-28T11:09:15.018409+0000","last_became_peered":"2023-11-28T11:09:15.018409+0000","last_unstale":"2023-11-28T11:09:16.395944+0000","last_undegraded":"2023-11-28T11:09:16.395944+0000","last_fullsized":"2023-11-28T11:09:16.395944+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:13.987782+0000","last_clean_scrub_stamp":"2023-11-28T11:09:13.987782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:56:43.328002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:09:15.001282+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:09:15.001282+0000","last_peered":"2023-11-28T11:09:15.001282+0000","last_clean":"2023-11-28T11:09:15.001282+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:09:15.001282+0000","last_undegraded":"2023-11-28T11:09:15.001282+0000","last_fullsized":"2023-11-28T11:09:15.001282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9644,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362196,"statfs":{"total":96636764160,"available":96626888704,"internally_reserved":0,"allocated":1019904,"data_stored":760721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.503}]},{"osd":1,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71199999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.73399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8348,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363492,"statfs":{"total":96636764160,"available":96628215808,"internally_reserved":0,"allocated":413696,"data_stored":169301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":5.1239999999999997,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.0410000000000004,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.51000000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.0250000000000004,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":5.1509999999999998,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.752}]},{"osd":2,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":6.0620000000000003,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.9409999999999998,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.60699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.65700000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.84899999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.76800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.80400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8476,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363364,"statfs":{"total":96636764160,"available":96628084736,"internally_reserved":0,"allocated":413696,"data_stored":168825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.0519999999999996,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.32600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":7.0890000000000004,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.38}]},{"osd":3,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.307}]},{"osd":4,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":18.024999999999999,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":17.966999999999999,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":18.056999999999999,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.69499999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9504,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362336,"statfs":{"total":96636764160,"available":96627032064,"internally_reserved":0,"allocated":1007616,"data_stored":759137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.253},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.29099999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.307},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.37},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.56599999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.64500000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.67000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8348,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363492,"statfs":{"total":96636764160,"available":96628215808,"internally_reserved":0,"allocated":413696,"data_stored":168805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.308},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.33800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.375},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.29299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.35099999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.626}]},{"osd":6,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57399999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8488,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363352,"statfs":{"total":96636764160,"available":96628072448,"internally_reserved":0,"allocated":425984,"data_stored":170317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.28799999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.39500000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.35399999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.53700000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.45800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8608,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363232,"statfs":{"total":96636764160,"available":96627949568,"internally_reserved":0,"allocated":417792,"data_stored":169323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.496},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":4.9740000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":12.263,"5min":12.263,"15min":12.263},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.524},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.876}]},{"osd":3,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.54},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":1.5600000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":4.3710000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":5.0220000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":1.923}]},{"osd":7,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":1.6579999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":1.49}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8480,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363360,"statfs":{"total":96636764160,"available":96628080640,"internally_reserved":0,"allocated":417792,"data_stored":168847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.63900000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.47599999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.495}]},{"osd":3,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.61899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.58999999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.67500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:09:19.869 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:09:19.869 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:09:20.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.028+0000 7f9a55a2c700 1 Processor -- start 2023-11-28T11:09:20.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.028+0000 7f9a55a2c700 1 -- start start 2023-11-28T11:09:20.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.029+0000 7f9a55a2c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9a50161340 0x7f9a500d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.029+0000 7f9a55a2c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a500d1950 con 0x7f9a5015c9b0 2023-11-28T11:09:20.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.029+0000 7f9a55a2c700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a500b5520 con 0x7f9a500d2320 2023-11-28T11:09:20.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.029+0000 7f9a55a2c700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a500b5800 con 0x7f9a50161340 2023-11-28T11:09:20.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.029+0000 7f9a4effd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f9a5015c9b0 0x7f9a500b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48702/0 (socket says 172.21.15.156:48702) 2023-11-28T11:09:20.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.029+0000 7f9a4effd700 1 -- 172.21.15.156:0/3074023030 learned_addr learned my addr 172.21.15.156:0/3074023030 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:20.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.029+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625820262 0 0) 0x7f9a500d1950 con 0x7f9a5015c9b0 2023-11-28T11:09:20.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.029+0000 7f9a4e7fc700 1 --2- 172.21.15.156:0/3074023030 >> v2:172.21.15.186:3300/0 conn(0x7f9a50161340 0x7f9a500d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.029+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a380035b0 con 0x7f9a5015c9b0 2023-11-28T11:09:20.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.029+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 749520874 0 0) 0x7f9a380035b0 con 0x7f9a5015c9b0 2023-11-28T11:09:20.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 >> v2:172.21.15.186:3300/0 conn(0x7f9a50161340 msgr2=0x7f9a500d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:20.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 --2- 172.21.15.156:0/3074023030 >> v2:172.21.15.186:3300/0 conn(0x7f9a50161340 0x7f9a500d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 >> v1:172.21.15.186:6789/0 conn(0x7f9a500d2320 legacy=0x7f9a500cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500d1300 con 0x7f9a5015c9b0 2023-11-28T11:09:20.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f9a44001670 con 0x7f9a5015c9b0 2023-11-28T11:09:20.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 >> v1:172.21.15.156:6789/0 conn(0x7f9a5015c9b0 legacy=0x7f9a500b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a38003a00 0x7f9a38003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 0x7f9a38004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38005200 0x7f9a3800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500b5800 con 0x7f9a38004850 2023-11-28T11:09:20.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3800c110 con 0x7f9a38005200 2023-11-28T11:09:20.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3800c2d0 con 0x7f9a38003a00 2023-11-28T11:09:20.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4e7fc700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 0x7f9a38004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4effd700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a38003a00 0x7f9a38003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4f7fe700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38005200 0x7f9a3800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4e7fc700 1 -- 172.21.15.156:0/3074023030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a38003a00 msgr2=0x7f9a38003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:20.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4e7fc700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a38003a00 0x7f9a38003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4e7fc700 1 -- 172.21.15.156:0/3074023030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38005200 msgr2=0x7f9a3800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:20.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4e7fc700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38005200 0x7f9a3800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4e7fc700 1 -- 172.21.15.156:0/3074023030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a380035b0 con 0x7f9a38004850 2023-11-28T11:09:20.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4effd700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a38003a00 0x7f9a38003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:20.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4f7fe700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38005200 0x7f9a3800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:20.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4f7fe700 1 -- 172.21.15.156:0/3074023030 reap_dead start 2023-11-28T11:09:20.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4e7fc700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 0x7f9a38004cc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009b60 tx=0x7f9a38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8629c73a2d0162 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000e6c0 con 0x7f9a38004850 2023-11-28T11:09:20.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4000e820 con 0x7f9a38004850 2023-11-28T11:09:20.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.030+0000 7f9a4dffb700 1 -- 172.21.15.156:0/3074023030 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000eb90 con 0x7f9a38004850 2023-11-28T11:09:20.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.031+0000 7f9a55a2c700 1 -- 172.21.15.156:0/3074023030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 msgr2=0x7f9a38004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.031+0000 7f9a55a2c700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 0x7f9a38004cc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009b60 tx=0x7f9a38005880 comp rx=0 tx=0).stop 2023-11-28T11:09:20.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.031+0000 7f9a55a2c700 1 -- 172.21.15.156:0/3074023030 shutdown_connections 2023-11-28T11:09:20.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.031+0000 7f9a55a2c700 1 --2- 172.21.15.156:0/3074023030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 0x7f9a38004cc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.031+0000 7f9a55a2c700 1 -- 172.21.15.156:0/3074023030 >> 172.21.15.156:0/3074023030 conn(0x7f9a50072600 msgr2=0x7f9a500cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:20.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.031+0000 7f9a55a2c700 1 -- 172.21.15.156:0/3074023030 shutdown_connections 2023-11-28T11:09:20.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.031+0000 7f9a55a2c700 1 -- 172.21.15.156:0/3074023030 wait complete. 2023-11-28T11:09:20.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.032+0000 7f9a55a2c700 1 Processor -- start 2023-11-28T11:09:20.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.032+0000 7f9a55a2c700 1 -- start start 2023-11-28T11:09:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a55a2c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 0x7f9a501f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a55a2c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a50152c00 0x7f9a50156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a55a2c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a501f5060 0x7f9a500b8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a55a2c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a50152460 con 0x7f9a38004850 2023-11-28T11:09:20.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a55a2c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a501517e0 con 0x7f9a501f5060 2023-11-28T11:09:20.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a55a2c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a50151ac0 con 0x7f9a50152c00 2023-11-28T11:09:20.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a4effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 0x7f9a501f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a4effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 0x7f9a501f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33836/0 (socket says 172.21.15.156:33836) 2023-11-28T11:09:20.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a4effd700 1 -- 172.21.15.156:0/754903767 learned_addr learned my addr 172.21.15.156:0/754903767 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:20.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a4f7fe700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a501f5060 0x7f9a500b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a4e7fc700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a50152c00 0x7f9a50156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a4effd700 1 -- 172.21.15.156:0/754903767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a50152c00 msgr2=0x7f9a50156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a4effd700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a50152c00 0x7f9a50156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a4effd700 1 -- 172.21.15.156:0/754903767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a501f5060 msgr2=0x7f9a500b8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a4effd700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a501f5060 0x7f9a500b8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.033+0000 7f9a4effd700 1 -- 172.21.15.156:0/754903767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5015cd80 con 0x7f9a38004850 2023-11-28T11:09:20.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.034+0000 7f9a4effd700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 0x7f9a501f23b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9a44000fb0 tx=0x7f9a4400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.034+0000 7f9a37fff700 1 -- 172.21.15.156:0/754903767 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a44005e40 con 0x7f9a38004850 2023-11-28T11:09:20.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.034+0000 7f9a37fff700 1 -- 172.21.15.156:0/754903767 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4400c7e0 con 0x7f9a38004850 2023-11-28T11:09:20.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.034+0000 7f9a37fff700 1 -- 172.21.15.156:0/754903767 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400cb50 con 0x7f9a38004850 2023-11-28T11:09:20.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.034+0000 7f9a55a2c700 1 -- 172.21.15.156:0/754903767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a500d15e0 con 0x7f9a38004850 2023-11-28T11:09:20.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.034+0000 7f9a55a2c700 1 -- 172.21.15.156:0/754903767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5015cf10 con 0x7f9a38004850 2023-11-28T11:09:20.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.035+0000 7f9a55a2c700 1 -- 172.21.15.156:0/754903767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a50153380 con 0x7f9a38004850 2023-11-28T11:09:20.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.035+0000 7f9a37fff700 1 -- 172.21.15.156:0/754903767 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9a4400ccb0 con 0x7f9a38004850 2023-11-28T11:09:20.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.035+0000 7f9a37fff700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9a3c0682f0 0x7f9a3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.036+0000 7f9a37fff700 1 -- 172.21.15.156:0/754903767 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a44085c40 con 0x7f9a38004850 2023-11-28T11:09:20.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.036+0000 7f9a4e7fc700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9a3c0682f0 0x7f9a3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.036+0000 7f9a4e7fc700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9a3c0682f0 0x7f9a3c06a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9a400090b0 tx=0x7f9a4001f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.040+0000 7f9a37fff700 1 -- 172.21.15.156:0/754903767 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9a50153380 con 0x7f9a38004850 2023-11-28T11:09:20.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.299+0000 7f9a55a2c700 1 -- 172.21.15.156:0/754903767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f9a50153380 con 0x7f9a38004850 2023-11-28T11:09:20.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.302+0000 7f9a37fff700 1 -- 172.21.15.156:0/754903767 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9a50153380 con 0x7f9a38004850 2023-11-28T11:09:20.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.305+0000 7f9a55a2c700 1 -- 172.21.15.156:0/754903767 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9a3c0682f0 msgr2=0x7f9a3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.305+0000 7f9a55a2c700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9a3c0682f0 0x7f9a3c06a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9a400090b0 tx=0x7f9a4001f040 comp rx=0 tx=0).stop 2023-11-28T11:09:20.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.305+0000 7f9a55a2c700 1 -- 172.21.15.156:0/754903767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 msgr2=0x7f9a501f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.305+0000 7f9a55a2c700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 0x7f9a501f23b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9a44000fb0 tx=0x7f9a4400edd0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.305+0000 7f9a55a2c700 1 -- 172.21.15.156:0/754903767 shutdown_connections 2023-11-28T11:09:20.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.305+0000 7f9a55a2c700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a501f5060 0x7f9a500b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.305+0000 7f9a55a2c700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a38004850 0x7f9a501f23b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.305+0000 7f9a55a2c700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9a3c0682f0 0x7f9a3c06a7a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.305+0000 7f9a55a2c700 1 --2- 172.21.15.156:0/754903767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a50152c00 0x7f9a50156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.305+0000 7f9a55a2c700 1 -- 172.21.15.156:0/754903767 >> 172.21.15.156:0/754903767 conn(0x7f9a50072600 msgr2=0x7f9a5015fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:20.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.305+0000 7f9a55a2c700 1 -- 172.21.15.156:0/754903767 shutdown_connections 2023-11-28T11:09:20.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.305+0000 7f9a55a2c700 1 -- 172.21.15.156:0/754903767 wait complete. 2023-11-28T11:09:20.307 INFO:teuthology.orchestra.run.smithi156.stderr:updated caps for client.0 2023-11-28T11:09:20.318 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:20.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.438+0000 7fdfcead0700 1 Processor -- start 2023-11-28T11:09:20.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.438+0000 7fdfcead0700 1 -- start start 2023-11-28T11:09:20.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.439+0000 7fdfcead0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdfc8071040 0x7fdfc8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.439+0000 7fdfcead0700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdfc806dee0 0x7fdfc806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.439+0000 7fdfcead0700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc8071950 con 0x7fdfc806cdc0 2023-11-28T11:09:20.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.439+0000 7fdfcead0700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc8161550 con 0x7fdfc8071040 2023-11-28T11:09:20.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.439+0000 7fdfcead0700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc815d060 con 0x7fdfc806dee0 2023-11-28T11:09:20.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.439+0000 7fdfcc86c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdfc8071040 0x7fdfc8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.439+0000 7fdfcc86c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdfc8071040 0x7fdfc8071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33844/0 (socket says 172.21.15.156:33844) 2023-11-28T11:09:20.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.439+0000 7fdfcc86c700 1 -- 172.21.15.156:0/1535382980 learned_addr learned my addr 172.21.15.156:0/1535382980 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:20.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.440+0000 7fdfc7fff700 1 --2- 172.21.15.156:0/1535382980 >> v2:172.21.15.186:3300/0 conn(0x7fdfc806dee0 0x7fdfc806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.440+0000 7fdfcc86c700 1 -- 172.21.15.156:0/1535382980 >> v2:172.21.15.186:3300/0 conn(0x7fdfc806dee0 msgr2=0x7fdfc806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:20.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.440+0000 7fdfcc86c700 1 --2- 172.21.15.156:0/1535382980 >> v2:172.21.15.186:3300/0 conn(0x7fdfc806dee0 0x7fdfc806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.440+0000 7fdfcc86c700 1 -- 172.21.15.156:0/1535382980 >> v1:172.21.15.186:6789/0 conn(0x7fdfc806cdc0 legacy=0x7fdfc806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:20.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.440+0000 7fdfcc86c700 1 -- 172.21.15.156:0/1535382980 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc815d340 con 0x7fdfc8071040 2023-11-28T11:09:20.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.440+0000 7fdfc7fff700 1 --2- 172.21.15.156:0/1535382980 >> v2:172.21.15.186:3300/0 conn(0x7fdfc806dee0 0x7fdfc806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:20.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.440+0000 7fdfcc86c700 1 --2- 172.21.15.156:0/1535382980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 0x7fdfc8071410 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdfc806fb00 tx=0x7fdfbc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d35d4a65d92bab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.440+0000 7fdfc77fe700 1 -- 172.21.15.156:0/1535382980 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc004230 con 0x7fdfc8071040 2023-11-28T11:09:20.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.440+0000 7fdfc77fe700 1 -- 172.21.15.156:0/1535382980 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc004390 con 0x7fdfc8071040 2023-11-28T11:09:20.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.440+0000 7fdfc77fe700 1 -- 172.21.15.156:0/1535382980 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0036a0 con 0x7fdfc8071040 2023-11-28T11:09:20.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.440+0000 7fdfcead0700 1 -- 172.21.15.156:0/1535382980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 msgr2=0x7fdfc8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.440+0000 7fdfcead0700 1 --2- 172.21.15.156:0/1535382980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 0x7fdfc8071410 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdfc806fb00 tx=0x7fdfbc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.441+0000 7fdfcead0700 1 -- 172.21.15.156:0/1535382980 shutdown_connections 2023-11-28T11:09:20.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.441+0000 7fdfcead0700 1 --2- 172.21.15.156:0/1535382980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 0x7fdfc8071410 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.441+0000 7fdfcead0700 1 --2- 172.21.15.156:0/1535382980 >> v2:172.21.15.186:3300/0 conn(0x7fdfc806dee0 0x7fdfc806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.441+0000 7fdfcead0700 1 -- 172.21.15.156:0/1535382980 >> 172.21.15.156:0/1535382980 conn(0x7fdfc80d0000 msgr2=0x7fdfc80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:20.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.441+0000 7fdfcead0700 1 -- 172.21.15.156:0/1535382980 shutdown_connections 2023-11-28T11:09:20.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.441+0000 7fdfcead0700 1 -- 172.21.15.156:0/1535382980 wait complete. 2023-11-28T11:09:20.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.442+0000 7fdfcead0700 1 Processor -- start 2023-11-28T11:09:20.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.442+0000 7fdfcead0700 1 -- start start 2023-11-28T11:09:20.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcead0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc806cdc0 0x7fdfc80ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcead0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc806dee0 0x7fdfc8154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcead0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 0x7fdfc8153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcead0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80b3530 con 0x7fdfc8071040 2023-11-28T11:09:20.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcead0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80acdf0 con 0x7fdfc806cdc0 2023-11-28T11:09:20.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcead0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80ad0b0 con 0x7fdfc806dee0 2023-11-28T11:09:20.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcd06d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 0x7fdfc8153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcd06d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 0x7fdfc8153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33852/0 (socket says 172.21.15.156:33852) 2023-11-28T11:09:20.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcd06d700 1 -- 172.21.15.156:0/3687649788 learned_addr learned my addr 172.21.15.156:0/3687649788 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:20.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcd06d700 1 -- 172.21.15.156:0/3687649788 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc806dee0 msgr2=0x7fdfc8154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcd06d700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc806dee0 0x7fdfc8154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcc86c700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc806cdc0 0x7fdfc80ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcd06d700 1 -- 172.21.15.156:0/3687649788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc806cdc0 msgr2=0x7fdfc80ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcd06d700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc806cdc0 0x7fdfc80ae420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcd06d700 1 -- 172.21.15.156:0/3687649788 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc009550 con 0x7fdfc8071040 2023-11-28T11:09:20.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcd06d700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 0x7fdfc8153e60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000b970 tx=0x7fdfc000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfc57fa700 1 -- 172.21.15.156:0/3687649788 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000c9b0 con 0x7fdfc8071040 2023-11-28T11:09:20.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfc57fa700 1 -- 172.21.15.156:0/3687649788 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc000cb10 con 0x7fdfc8071040 2023-11-28T11:09:20.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfc57fa700 1 -- 172.21.15.156:0/3687649788 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0014440 con 0x7fdfc8071040 2023-11-28T11:09:20.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcead0700 1 -- 172.21.15.156:0/3687649788 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc815d900 con 0x7fdfc8071040 2023-11-28T11:09:20.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.443+0000 7fdfcead0700 1 -- 172.21.15.156:0/3687649788 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc815d620 con 0x7fdfc8071040 2023-11-28T11:09:20.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.444+0000 7fdfcead0700 1 -- 172.21.15.156:0/3687649788 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc806e870 con 0x7fdfc8071040 2023-11-28T11:09:20.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.445+0000 7fdfc57fa700 1 -- 172.21.15.156:0/3687649788 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdfc001b030 con 0x7fdfc8071040 2023-11-28T11:09:20.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.445+0000 7fdfc57fa700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdfac0682f0 0x7fdfac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.445+0000 7fdfc57fa700 1 -- 172.21.15.156:0/3687649788 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdfc0085850 con 0x7fdfc8071040 2023-11-28T11:09:20.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.445+0000 7fdfcc86c700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdfac0682f0 0x7fdfac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.446+0000 7fdfcc86c700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdfac0682f0 0x7fdfac06a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.449+0000 7fdfc57fa700 1 -- 172.21.15.156:0/3687649788 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdfc0052030 con 0x7fdfc8071040 2023-11-28T11:09:20.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.677+0000 7fdfcead0700 1 -- 172.21.15.156:0/3687649788 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfc80720a0 con 0x7fdfc8071040 2023-11-28T11:09:20.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.677+0000 7fdfc57fa700 1 -- 172.21.15.156:0/3687649788 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fdfc0055650 con 0x7fdfc8071040 2023-11-28T11:09:20.679 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:20.679 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1085440106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1085440106},{"type":"v1","addr":"172.21.15.186:6834","nonce":1085440106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1548245207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1548245207},{"type":"v1","addr":"172.21.15.156:6836","nonce":1548245207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4882,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3102077957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":3102077957},{"type":"v1","addr":"172.21.15.156:6837","nonce":3102077957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:15.391255+0000","modified":"2023-11-28T11:09:18.414744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3897746301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3897746301},{"type":"v1","addr":"172.21.15.186:6835","nonce":3897746301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:09:20.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.679+0000 7fdfcead0700 1 -- 172.21.15.156:0/3687649788 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdfac0682f0 msgr2=0x7fdfac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.679+0000 7fdfcead0700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdfac0682f0 0x7fdfac06a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc008030 comp rx=0 tx=0).stop 2023-11-28T11:09:20.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.679+0000 7fdfcead0700 1 -- 172.21.15.156:0/3687649788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 msgr2=0x7fdfc8153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.679+0000 7fdfcead0700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 0x7fdfc8153e60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000b970 tx=0x7fdfc000bd30 comp rx=0 tx=0).stop 2023-11-28T11:09:20.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.679+0000 7fdfcead0700 1 -- 172.21.15.156:0/3687649788 shutdown_connections 2023-11-28T11:09:20.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.679+0000 7fdfcead0700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfc8071040 0x7fdfc8153e60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.679+0000 7fdfcead0700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc806cdc0 0x7fdfc80ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.679+0000 7fdfcead0700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdfac0682f0 0x7fdfac06a7a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.679+0000 7fdfcead0700 1 --2- 172.21.15.156:0/3687649788 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfc806dee0 0x7fdfc8154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.679+0000 7fdfcead0700 1 -- 172.21.15.156:0/3687649788 >> 172.21.15.156:0/3687649788 conn(0x7fdfc80d0000 msgr2=0x7fdfc806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:20.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.679+0000 7fdfcead0700 1 -- 172.21.15.156:0/3687649788 shutdown_connections 2023-11-28T11:09:20.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:20.679+0000 7fdfcead0700 1 -- 172.21.15.156:0/3687649788 wait complete. 2023-11-28T11:09:20.682 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T11:09:20.825 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:09:20.828 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:09:21.095 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:09:15.391255+0000', 'modified': '2023-11-28T11:09:18.414744+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4894}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4894': {'gid': 4894, 'name': 'd', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6835/3897746301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6833', 'nonce': 3897746301}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 3897746301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:09:21.095 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:09:21.095 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T11:09:21.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.217+0000 7f6a1a555700 1 Processor -- start 2023-11-28T11:09:21.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.217+0000 7f6a1a555700 1 -- start start 2023-11-28T11:09:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a1a555700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6a140cd8c0 0x7f6a14140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a1a555700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6a1406f850 0x7f6a1406fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a1a555700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1413ff00 con 0x7f6a140cdc90 2023-11-28T11:09:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a1a555700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a141404c0 con 0x7f6a140cd8c0 2023-11-28T11:09:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a1a555700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1415aa80 con 0x7f6a1406f850 2023-11-28T11:09:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a13fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6a140cd8c0 0x7f6a14140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a13fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6a140cd8c0 0x7f6a14140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33856/0 (socket says 172.21.15.156:33856) 2023-11-28T11:09:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a13fff700 1 -- 172.21.15.156:0/2276138463 learned_addr learned my addr 172.21.15.156:0/2276138463 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a18af2700 1 --2- 172.21.15.156:0/2276138463 >> v2:172.21.15.186:3300/0 conn(0x7f6a1406f850 0x7f6a1406fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a13fff700 1 -- 172.21.15.156:0/2276138463 >> v2:172.21.15.186:3300/0 conn(0x7f6a1406f850 msgr2=0x7f6a1406fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a13fff700 1 --2- 172.21.15.156:0/2276138463 >> v2:172.21.15.186:3300/0 conn(0x7f6a1406f850 0x7f6a1406fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a13fff700 1 -- 172.21.15.156:0/2276138463 >> v1:172.21.15.186:6789/0 conn(0x7f6a140cdc90 legacy=0x7f6a140b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a13fff700 1 -- 172.21.15.156:0/2276138463 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140b3040 con 0x7f6a140cd8c0 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.218+0000 7f6a13fff700 1 --2- 172.21.15.156:0/2276138463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a140cd8c0 0x7f6a14140950 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6a1406ea90 tx=0x7f6a08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b741bc099c48913 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.219+0000 7f6a12ffd700 1 -- 172.21.15.156:0/2276138463 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a08005e30 con 0x7f6a140cd8c0 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.219+0000 7f6a12ffd700 1 -- 172.21.15.156:0/2276138463 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a08004030 con 0x7f6a140cd8c0 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.219+0000 7f6a12ffd700 1 -- 172.21.15.156:0/2276138463 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a080043e0 con 0x7f6a140cd8c0 2023-11-28T11:09:21.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.219+0000 7f6a1a555700 1 -- 172.21.15.156:0/2276138463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a140cd8c0 msgr2=0x7f6a14140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.219+0000 7f6a1a555700 1 --2- 172.21.15.156:0/2276138463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a140cd8c0 0x7f6a14140950 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6a1406ea90 tx=0x7f6a08009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.219+0000 7f6a1a555700 1 -- 172.21.15.156:0/2276138463 shutdown_connections 2023-11-28T11:09:21.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.219+0000 7f6a1a555700 1 --2- 172.21.15.156:0/2276138463 >> v2:172.21.15.186:3300/0 conn(0x7f6a1406f850 0x7f6a1406fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.219+0000 7f6a1a555700 1 --2- 172.21.15.156:0/2276138463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a140cd8c0 0x7f6a14140950 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.219+0000 7f6a1a555700 1 -- 172.21.15.156:0/2276138463 >> 172.21.15.156:0/2276138463 conn(0x7f6a1406aed0 msgr2=0x7f6a1406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.219+0000 7f6a1a555700 1 -- 172.21.15.156:0/2276138463 shutdown_connections 2023-11-28T11:09:21.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.219+0000 7f6a1a555700 1 -- 172.21.15.156:0/2276138463 wait complete. 2023-11-28T11:09:21.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.220+0000 7f6a1a555700 1 Processor -- start 2023-11-28T11:09:21.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.221+0000 7f6a1a555700 1 -- start start 2023-11-28T11:09:21.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.221+0000 7f6a1a555700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a1406f850 0x7f6a140c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.221+0000 7f6a1a555700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a140cd8c0 0x7f6a141f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.221+0000 7f6a1a555700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a140cdc90 0x7f6a141f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.221+0000 7f6a1a555700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140b5f10 con 0x7f6a140cdc90 2023-11-28T11:09:21.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.221+0000 7f6a1a555700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140b61f0 con 0x7f6a1406f850 2023-11-28T11:09:21.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.221+0000 7f6a1a555700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140b64d0 con 0x7f6a140cd8c0 2023-11-28T11:09:21.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.221+0000 7f6a18af2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a140cdc90 0x7f6a141f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.221+0000 7f6a18af2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a140cdc90 0x7f6a141f32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33864/0 (socket says 172.21.15.156:33864) 2023-11-28T11:09:21.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.221+0000 7f6a18af2700 1 -- 172.21.15.156:0/3095274758 learned_addr learned my addr 172.21.15.156:0/3095274758 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:21.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a13fff700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a1406f850 0x7f6a140c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a18af2700 1 -- 172.21.15.156:0/3095274758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a140cd8c0 msgr2=0x7f6a141f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a18af2700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a140cd8c0 0x7f6a141f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a18af2700 1 -- 172.21.15.156:0/3095274758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a1406f850 msgr2=0x7f6a140c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a18af2700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a1406f850 0x7f6a140c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a18af2700 1 -- 172.21.15.156:0/3095274758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a08009500 con 0x7f6a140cdc90 2023-11-28T11:09:21.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a13fff700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a1406f850 0x7f6a140c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:21.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a18af2700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a140cdc90 0x7f6a141f32b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c007e40 tx=0x7f6a0c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a10ff9700 1 -- 172.21.15.156:0/3095274758 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00eea0 con 0x7f6a140cdc90 2023-11-28T11:09:21.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a10ff9700 1 -- 172.21.15.156:0/3095274758 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c014420 con 0x7f6a140cdc90 2023-11-28T11:09:21.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a1a555700 1 -- 172.21.15.156:0/3095274758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a140b35f0 con 0x7f6a140cdc90 2023-11-28T11:09:21.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a10ff9700 1 -- 172.21.15.156:0/3095274758 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c0147b0 con 0x7f6a140cdc90 2023-11-28T11:09:21.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.222+0000 7f6a1a555700 1 -- 172.21.15.156:0/3095274758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a140b3310 con 0x7f6a140cdc90 2023-11-28T11:09:21.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.223+0000 7f6a10ff9700 1 -- 172.21.15.156:0/3095274758 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a0c012050 con 0x7f6a140cdc90 2023-11-28T11:09:21.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.224+0000 7f6a10ff9700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6a000682f0 0x7f6a0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.224+0000 7f6a10ff9700 1 -- 172.21.15.156:0/3095274758 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a0c086310 con 0x7f6a140cdc90 2023-11-28T11:09:21.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.224+0000 7f6a13fff700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6a000682f0 0x7f6a0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.224+0000 7f6a13fff700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6a000682f0 0x7f6a0006a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a1406ea90 tx=0x7f6a08008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.226+0000 7f6a266deb80 1 Processor -- start 2023-11-28T11:09:21.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.226+0000 7f6a266deb80 1 -- start start 2023-11-28T11:09:21.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a266deb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab83810 0x56439ab83be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a266deb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab581c0 0x56439ab84120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a266deb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab7cce0 0x56439ab7d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a266deb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439aa43bc0 con 0x56439ab581c0 2023-11-28T11:09:21.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a266deb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439aab2350 con 0x56439ab83810 2023-11-28T11:09:21.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a266deb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56439aaab880 con 0x56439ab7cce0 2023-11-28T11:09:21.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab581c0 0x56439ab84120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab581c0 0x56439ab84120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33870/0 (socket says 172.21.15.156:33870) 2023-11-28T11:09:21.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a137fe700 1 -- 172.21.15.156:0/1463773151 learned_addr learned my addr 172.21.15.156:0/1463773151 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:21.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a13fff700 1 --2- 172.21.15.156:0/1463773151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab83810 0x56439ab83be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a18af2700 1 --2- 172.21.15.156:0/1463773151 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab7cce0 0x56439ab7d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a137fe700 1 -- 172.21.15.156:0/1463773151 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab7cce0 msgr2=0x56439ab7d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a137fe700 1 --2- 172.21.15.156:0/1463773151 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab7cce0 0x56439ab7d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a137fe700 1 -- 172.21.15.156:0/1463773151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab83810 msgr2=0x56439ab83be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a137fe700 1 --2- 172.21.15.156:0/1463773151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab83810 0x56439ab83be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a137fe700 1 -- 172.21.15.156:0/1463773151 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56439a92d5e0 con 0x56439ab581c0 2023-11-28T11:09:21.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a13fff700 1 --2- 172.21.15.156:0/1463773151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab83810 0x56439ab83be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:21.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f6a137fe700 1 --2- 172.21.15.156:0/1463773151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab581c0 0x56439ab84120 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6a040014a0 tx=0x7f6a040017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10a9622f7e5bfd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f69f9ffb700 1 -- 172.21.15.156:0/1463773151 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a040052d0 con 0x56439ab581c0 2023-11-28T11:09:21.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f69f9ffb700 1 -- 172.21.15.156:0/1463773151 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a04005450 con 0x56439ab581c0 2023-11-28T11:09:21.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.227+0000 7f69f9ffb700 1 -- 172.21.15.156:0/1463773151 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a040038c0 con 0x56439ab581c0 2023-11-28T11:09:21.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.228+0000 7f6a266deb80 1 -- 172.21.15.156:0/1463773151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab581c0 msgr2=0x56439ab84120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.228+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1463773151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab581c0 0x56439ab84120 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6a040014a0 tx=0x7f6a040017b0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.228+0000 7f6a266deb80 1 -- 172.21.15.156:0/1463773151 shutdown_connections 2023-11-28T11:09:21.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.228+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1463773151 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab7cce0 0x56439ab7d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.228+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1463773151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab83810 0x56439ab83be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.228+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1463773151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab581c0 0x56439ab84120 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.228+0000 7f6a266deb80 1 -- 172.21.15.156:0/1463773151 >> 172.21.15.156:0/1463773151 conn(0x56439ab56d60 msgr2=0x56439ab576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.228+0000 7f6a266deb80 1 -- 172.21.15.156:0/1463773151 shutdown_connections 2023-11-28T11:09:21.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.228+0000 7f6a266deb80 1 -- 172.21.15.156:0/1463773151 wait complete. 2023-11-28T11:09:21.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.229+0000 7f6a266deb80 1 -- start start 2023-11-28T11:09:21.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.229+0000 7f6a266deb80 1 Processor -- start 2023-11-28T11:09:21.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.230+0000 7f6a266deb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab581c0 0x56439ab7b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.230+0000 7f6a266deb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab7cce0 0x56439aba91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.230+0000 7f6a266deb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 0x56439ab86e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.230+0000 7f6a266deb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439a92f8d0 con 0x56439ab83810 2023-11-28T11:09:21.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.230+0000 7f6a266deb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439ab7bc60 con 0x56439ab7cce0 2023-11-28T11:09:21.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.230+0000 7f6a266deb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56439aba9bd0 con 0x56439ab581c0 2023-11-28T11:09:21.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.230+0000 7f6a266deb80 10 client.0 _ll_register_callbacks cb 0x56439a951240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:09:21.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.230+0000 7f6a18af2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 0x56439ab86e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.230+0000 7f6a18af2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 0x56439ab86e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33878/0 (socket says 172.21.15.156:33878) 2023-11-28T11:09:21.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.230+0000 7f6a18af2700 1 -- 172.21.15.156:0/102465449 learned_addr learned my addr 172.21.15.156:0/102465449 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:21.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.230+0000 7f6a13fff700 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab581c0 0x56439ab7b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.230+0000 7f6a137fe700 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab7cce0 0x56439aba91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a18af2700 1 -- 172.21.15.156:0/102465449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab581c0 msgr2=0x56439ab7b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a18af2700 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab581c0 0x56439ab7b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a18af2700 1 -- 172.21.15.156:0/102465449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab7cce0 msgr2=0x56439aba91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a18af2700 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab7cce0 0x56439aba91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a18af2700 1 -- 172.21.15.156:0/102465449 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1413ff00 con 0x56439ab83810 2023-11-28T11:09:21.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a18af2700 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 0x56439ab86e50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c05e390 tx=0x7f6a0c05ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9759d022b604b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f69f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:09:21.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f69f2ffd700 1 -- 172.21.15.156:0/102465449 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c057bc0 con 0x56439ab83810 2023-11-28T11:09:21.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f69f2ffd700 1 -- 172.21.15.156:0/102465449 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a0c057d20 con 0x56439ab83810 2023-11-28T11:09:21.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a266deb80 1 -- 172.21.15.156:0/102465449 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56439aa79140 con 0x56439ab83810 2023-11-28T11:09:21.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f69f2ffd700 1 -- 172.21.15.156:0/102465449 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a0c05c0b0 con 0x56439ab83810 2023-11-28T11:09:21.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a266deb80 10 client.5280 fetch_fsmap learned FSMap version 16 2023-11-28T11:09:21.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a266deb80 1 -- 172.21.15.156:0/102465449 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x56439a8d9d90 con 0x56439ab83810 2023-11-28T11:09:21.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f69f2ffd700 1 -- 172.21.15.156:0/102465449 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c062070 con 0x56439ab83810 2023-11-28T11:09:21.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f69f2ffd700 1 -- 172.21.15.156:0/102465449 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6a0c05c210 con 0x56439ab83810 2023-11-28T11:09:21.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a266deb80 10 client.5280 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T11:09:21.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a266deb80 10 client.5280 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:09:21.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a266deb80 20 client.5280 populate_metadata read hostname 'smithi156' 2023-11-28T11:09:21.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a266deb80 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.186:6833/3897746301,v1:172.21.15.186:6835/3897746301] conn(0x56439ab97710 0x56439ab956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a266deb80 4 client.5280 mds_command: new command op to 4894 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:09:21.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.231+0000 7f6a266deb80 1 -- 172.21.15.156:0/102465449 --> [v2:172.21.15.186:6833/3897746301,v1:172.21.15.186:6835/3897746301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56439a92ff10 con 0x56439ab97710 2023-11-28T11:09:21.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.232+0000 7f6a13fff700 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.186:6833/3897746301,v1:172.21.15.186:6835/3897746301] conn(0x56439ab97710 0x56439ab956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.232+0000 7f6a13fff700 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.186:6833/3897746301,v1:172.21.15.186:6835/3897746301] conn(0x56439ab97710 0x56439ab956d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde978aade403904 server_cookie=7628aac6f726f3f8 in_seq=0 out_seq=0 2023-11-28T11:09:21.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.232+0000 7f69f2ffd700 10 client.5280 ms_handle_connect on v2:172.21.15.186:6833/3897746301 2023-11-28T11:09:21.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f69f2ffd700 1 -- 172.21.15.156:0/102465449 <== mds.0 v2:172.21.15.186:6833/3897746301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x56439a92ff10 con 0x56439ab97710 2023-11-28T11:09:21.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f69f2ffd700 10 client.5280 handle_command_reply: tid=0 2023-11-28T11:09:21.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f6a266deb80 1 client.5280 shutdown 2023-11-28T11:09:21.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f6a266deb80 1 -- 172.21.15.156:0/102465449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 msgr2=0x56439ab86e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f6a266deb80 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 0x56439ab86e50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c05e390 tx=0x7f6a0c05ea10 comp rx=0 tx=0).stop 2023-11-28T11:09:21.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f6a266deb80 1 -- 172.21.15.156:0/102465449 shutdown_connections 2023-11-28T11:09:21.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f6a266deb80 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 0x56439ab86e50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f6a266deb80 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab581c0 0x56439ab7b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f6a266deb80 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.186:6833/3897746301,v1:172.21.15.186:6835/3897746301] conn(0x56439ab97710 0x56439ab956d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f6a266deb80 1 --2- 172.21.15.156:0/102465449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab7cce0 0x56439aba91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f69f2ffd700 0 client.5280 ms_handle_reset on v2:172.21.15.186:6833/3897746301 2023-11-28T11:09:21.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f6a266deb80 1 -- 172.21.15.156:0/102465449 >> 172.21.15.156:0/102465449 conn(0x56439ab56d60 msgr2=0x56439ab58590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f6a266deb80 1 -- 172.21.15.156:0/102465449 shutdown_connections 2023-11-28T11:09:21.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f6a266deb80 1 -- 172.21.15.156:0/102465449 wait complete. 2023-11-28T11:09:21.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.233+0000 7f6a266deb80 20 client.5280 trim_cache size 0 max 16384 2023-11-28T11:09:21.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.250+0000 7f6a266deb80 1 Processor -- start 2023-11-28T11:09:21.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.250+0000 7f6a266deb80 1 -- start start 2023-11-28T11:09:21.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a266deb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab581c0 0x56439aba2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a266deb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab7cce0 0x56439aba26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a266deb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab83810 0x56439ab96610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a266deb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439ab569b0 con 0x56439ab7cce0 2023-11-28T11:09:21.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a266deb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439ab58b10 con 0x56439ab581c0 2023-11-28T11:09:21.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a266deb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56439ab52cf0 con 0x56439ab83810 2023-11-28T11:09:21.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab7cce0 0x56439aba26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab7cce0 0x56439aba26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33884/0 (socket says 172.21.15.156:33884) 2023-11-28T11:09:21.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a137fe700 1 -- 172.21.15.156:0/1048459333 learned_addr learned my addr 172.21.15.156:0/1048459333 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:21.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a13fff700 1 --2- 172.21.15.156:0/1048459333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab581c0 0x56439aba2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a18af2700 1 --2- 172.21.15.156:0/1048459333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab83810 0x56439ab96610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a137fe700 1 -- 172.21.15.156:0/1048459333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab83810 msgr2=0x56439ab96610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a137fe700 1 --2- 172.21.15.156:0/1048459333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab83810 0x56439ab96610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a137fe700 1 -- 172.21.15.156:0/1048459333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab581c0 msgr2=0x56439aba2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a137fe700 1 --2- 172.21.15.156:0/1048459333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab581c0 0x56439aba2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a137fe700 1 -- 172.21.15.156:0/1048459333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56439ab7c920 con 0x56439ab7cce0 2023-11-28T11:09:21.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.251+0000 7f6a137fe700 1 --2- 172.21.15.156:0/1048459333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab7cce0 0x56439aba26b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6a040017f0 tx=0x7f6a04010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a46f001a5542c40 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.252+0000 7f69fa7fc700 1 -- 172.21.15.156:0/1048459333 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04005390 con 0x56439ab7cce0 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.252+0000 7f69fa7fc700 1 -- 172.21.15.156:0/1048459333 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0400b070 con 0x56439ab7cce0 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.252+0000 7f69fa7fc700 1 -- 172.21.15.156:0/1048459333 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04006aa0 con 0x56439ab7cce0 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.252+0000 7f6a266deb80 1 -- 172.21.15.156:0/1048459333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab7cce0 msgr2=0x56439aba26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.252+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1048459333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab7cce0 0x56439aba26b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6a040017f0 tx=0x7f6a04010000 comp rx=0 tx=0).stop 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.252+0000 7f6a266deb80 1 -- 172.21.15.156:0/1048459333 shutdown_connections 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.252+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1048459333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab83810 0x56439ab96610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.252+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1048459333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab581c0 0x56439aba2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.252+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1048459333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab7cce0 0x56439aba26b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.252+0000 7f6a266deb80 1 -- 172.21.15.156:0/1048459333 >> 172.21.15.156:0/1048459333 conn(0x56439ab56d60 msgr2=0x56439ab7b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.252+0000 7f6a266deb80 1 -- 172.21.15.156:0/1048459333 shutdown_connections 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.252+0000 7f6a266deb80 1 -- 172.21.15.156:0/1048459333 wait complete. 2023-11-28T11:09:21.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.253+0000 7f6a266deb80 1 -- start start 2023-11-28T11:09:21.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.253+0000 7f6a266deb80 1 Processor -- start 2023-11-28T11:09:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a266deb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab581c0 0x56439ac07cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a266deb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab7cce0 0x56439aba2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a266deb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 0x56439aba26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a266deb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439ac08580 con 0x56439ab83810 2023-11-28T11:09:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a266deb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439ab57e80 con 0x56439ab581c0 2023-11-28T11:09:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a266deb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56439ab79ef0 con 0x56439ab7cce0 2023-11-28T11:09:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a266deb80 10 client.0 _ll_register_callbacks cb 0x56439a951240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:09:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a18af2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 0x56439aba26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a18af2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 0x56439aba26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33892/0 (socket says 172.21.15.156:33892) 2023-11-28T11:09:21.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a18af2700 1 -- 172.21.15.156:0/1258903668 learned_addr learned my addr 172.21.15.156:0/1258903668 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a13fff700 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab581c0 0x56439ac07cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a137fe700 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab7cce0 0x56439aba2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a18af2700 1 -- 172.21.15.156:0/1258903668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab7cce0 msgr2=0x56439aba2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a18af2700 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab7cce0 0x56439aba2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a18af2700 1 -- 172.21.15.156:0/1258903668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab581c0 msgr2=0x56439ac07cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a18af2700 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab581c0 0x56439ac07cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a18af2700 1 -- 172.21.15.156:0/1258903668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56439a92d5e0 con 0x56439ab83810 2023-11-28T11:09:21.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a18af2700 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 0x56439aba26b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0848f0 tx=0x7f6a0c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1455bb8fbfc33e23 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f69f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:09:21.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f69f2ffd700 1 -- 172.21.15.156:0/1258903668 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c05c0b0 con 0x56439ab83810 2023-11-28T11:09:21.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f69f2ffd700 1 -- 172.21.15.156:0/1258903668 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a0c05c210 con 0x56439ab83810 2023-11-28T11:09:21.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.254+0000 7f6a266deb80 1 -- 172.21.15.156:0/1258903668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56439aa69160 con 0x56439ab83810 2023-11-28T11:09:21.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f69f2ffd700 1 -- 172.21.15.156:0/1258903668 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a0c06f080 con 0x56439ab83810 2023-11-28T11:09:21.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f6a266deb80 10 client.5286 fetch_fsmap learned FSMap version 16 2023-11-28T11:09:21.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f6a266deb80 1 -- 172.21.15.156:0/1258903668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x56439a8c2c10 con 0x56439ab83810 2023-11-28T11:09:21.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f69f2ffd700 1 -- 172.21.15.156:0/1258903668 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56439aa69160 con 0x56439ab83810 2023-11-28T11:09:21.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f69f2ffd700 1 -- 172.21.15.156:0/1258903668 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6a0c052620 con 0x56439ab83810 2023-11-28T11:09:21.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f6a266deb80 10 client.5286 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T11:09:21.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f6a266deb80 10 client.5286 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:09:21.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f6a266deb80 20 client.5286 populate_metadata read hostname 'smithi156' 2023-11-28T11:09:21.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.186:6833/3897746301,v1:172.21.15.186:6835/3897746301] conn(0x56439ac0b770 0x56439ac0bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f6a266deb80 4 client.5286 mds_command: new command op to 4894 tid=0[{"prefix": "status"}] 2023-11-28T11:09:21.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f6a266deb80 1 -- 172.21.15.156:0/1258903668 --> [v2:172.21.15.186:6833/3897746301,v1:172.21.15.186:6835/3897746301] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56439aa53170 con 0x56439ac0b770 2023-11-28T11:09:21.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f6a13fff700 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.186:6833/3897746301,v1:172.21.15.186:6835/3897746301] conn(0x56439ac0b770 0x56439ac0bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f69f2ffd700 10 client.5286 ms_handle_connect on v2:172.21.15.186:6833/3897746301 2023-11-28T11:09:21.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.255+0000 7f6a13fff700 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.186:6833/3897746301,v1:172.21.15.186:6835/3897746301] conn(0x56439ac0b770 0x56439ac0bb40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1025a83a6ff8e2a0 server_cookie=ba8ab492e6823807 in_seq=0 out_seq=0 2023-11-28T11:09:21.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f69f2ffd700 1 -- 172.21.15.156:0/1258903668 <== mds.0 v2:172.21.15.186:6833/3897746301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56439aa53170 con 0x56439ac0b770 2023-11-28T11:09:21.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f69f2ffd700 10 client.5286 handle_command_reply: tid=0 2023-11-28T11:09:21.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a266deb80 1 client.5286 shutdown 2023-11-28T11:09:21.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a266deb80 1 -- 172.21.15.156:0/1258903668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 msgr2=0x56439aba26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 0x56439aba26b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0848f0 tx=0x7f6a0c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a266deb80 1 -- 172.21.15.156:0/1258903668 shutdown_connections 2023-11-28T11:09:21.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56439ab83810 0x56439aba26b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.186:6833/3897746301,v1:172.21.15.186:6835/3897746301] conn(0x56439ac0b770 0x56439ac0bb40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56439ab581c0 0x56439ac07cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f69f2ffd700 0 client.5286 ms_handle_reset on v2:172.21.15.186:6833/3897746301 2023-11-28T11:09:21.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a266deb80 1 --2- 172.21.15.156:0/1258903668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56439ab7cce0 0x56439aba2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a266deb80 1 -- 172.21.15.156:0/1258903668 >> 172.21.15.156:0/1258903668 conn(0x56439ab56d60 msgr2=0x56439ab52450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a266deb80 1 -- 172.21.15.156:0/1258903668 shutdown_connections 2023-11-28T11:09:21.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a266deb80 1 -- 172.21.15.156:0/1258903668 wait complete. 2023-11-28T11:09:21.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a266deb80 20 client.5286 trim_cache size 0 max 16384 2023-11-28T11:09:21.273 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:09:21.273 INFO:teuthology.orchestra.run.smithi156.stdout: "cluster_fsid": "351da896-8dbf-473b-9472-c3454d9368bf", 2023-11-28T11:09:21.273 INFO:teuthology.orchestra.run.smithi156.stdout: "whoami": 0, 2023-11-28T11:09:21.273 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 4894, 2023-11-28T11:09:21.273 INFO:teuthology.orchestra.run.smithi156.stdout: "want_state": "up:active", 2023-11-28T11:09:21.273 INFO:teuthology.orchestra.run.smithi156.stdout: "state": "up:active", 2023-11-28T11:09:21.273 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs", 2023-11-28T11:09:21.273 INFO:teuthology.orchestra.run.smithi156.stdout: "rank_uptime": 5.8630709149999998, 2023-11-28T11:09:21.273 INFO:teuthology.orchestra.run.smithi156.stdout: "mdsmap_epoch": 16, 2023-11-28T11:09:21.274 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch": 37, 2023-11-28T11:09:21.274 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch_barrier": 37, 2023-11-28T11:09:21.274 INFO:teuthology.orchestra.run.smithi156.stdout: "uptime": 13.180164453 2023-11-28T11:09:21.274 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:09:21.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a1a555700 1 -- 172.21.15.156:0/3095274758 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6a000682f0 msgr2=0x7f6a0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a1a555700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6a000682f0 0x7f6a0006a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a1406ea90 tx=0x7f6a08008070 comp rx=0 tx=0).stop 2023-11-28T11:09:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a1a555700 1 -- 172.21.15.156:0/3095274758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a140cdc90 msgr2=0x7f6a141f32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.256+0000 7f6a1a555700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a140cdc90 0x7f6a141f32b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c007e40 tx=0x7f6a0c00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.257+0000 7f6a1a555700 1 -- 172.21.15.156:0/3095274758 shutdown_connections 2023-11-28T11:09:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.257+0000 7f6a1a555700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a140cdc90 0x7f6a141f32b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.257+0000 7f6a1a555700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a1406f850 0x7f6a140c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.257+0000 7f6a1a555700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6a000682f0 0x7f6a0006a7a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.257+0000 7f6a1a555700 1 --2- 172.21.15.156:0/3095274758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a140cd8c0 0x7f6a141f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.257+0000 7f6a1a555700 1 -- 172.21.15.156:0/3095274758 >> 172.21.15.156:0/3095274758 conn(0x7f6a1406aed0 msgr2=0x7f6a140cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.257+0000 7f6a1a555700 1 -- 172.21.15.156:0/3095274758 shutdown_connections 2023-11-28T11:09:21.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.257+0000 7f6a1a555700 1 -- 172.21.15.156:0/3095274758 wait complete. 2023-11-28T11:09:21.277 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:09:21.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.396+0000 7f52e3922700 1 Processor -- start 2023-11-28T11:09:21.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.396+0000 7f52e3922700 1 -- start start 2023-11-28T11:09:21.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e3922700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52dc140950 0x7f52dc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e3922700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f52dc0b2e70 0x7f52dc06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e3922700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52dc13ff00 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e3922700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52dc1404c0 con 0x7f52dc0b2e70 2023-11-28T11:09:21.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e3922700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc140dc0 con 0x7f52dc140950 2023-11-28T11:09:21.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e1ebf700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f52dc0b2e70 0x7f52dc06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e1ebf700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f52dc0b2e70 0x7f52dc06f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33900/0 (socket says 172.21.15.156:33900) 2023-11-28T11:09:21.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e1ebf700 1 -- 172.21.15.156:0/1188716459 learned_addr learned my addr 172.21.15.156:0/1188716459 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:21.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e0ebd700 1 --2- 172.21.15.156:0/1188716459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52dc140950 0x7f52dc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e1ebf700 1 -- 172.21.15.156:0/1188716459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52dc140950 msgr2=0x7f52dc0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:21.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e1ebf700 1 --2- 172.21.15.156:0/1188716459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52dc140950 0x7f52dc0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e1ebf700 1 -- 172.21.15.156:0/1188716459 >> v1:172.21.15.186:6789/0 conn(0x7f52dc0cd8c0 legacy=0x7f52dc0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e1ebf700 1 -- 172.21.15.156:0/1188716459 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc15aa80 con 0x7f52dc0b2e70 2023-11-28T11:09:21.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52e1ebf700 1 --2- 172.21.15.156:0/1188716459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52dc0b2e70 0x7f52dc06f850 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009e50 tx=0x7f52d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2bd03dee3a4d8c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52d3fff700 1 -- 172.21.15.156:0/1188716459 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400e070 con 0x7f52dc0b2e70 2023-11-28T11:09:21.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52d3fff700 1 -- 172.21.15.156:0/1188716459 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d400b910 con 0x7f52dc0b2e70 2023-11-28T11:09:21.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.397+0000 7f52d3fff700 1 -- 172.21.15.156:0/1188716459 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400bcc0 con 0x7f52dc0b2e70 2023-11-28T11:09:21.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.398+0000 7f52e3922700 1 -- 172.21.15.156:0/1188716459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52dc0b2e70 msgr2=0x7f52dc06f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.398+0000 7f52e3922700 1 --2- 172.21.15.156:0/1188716459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52dc0b2e70 0x7f52dc06f850 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009e50 tx=0x7f52d400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:21.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.398+0000 7f52e3922700 1 -- 172.21.15.156:0/1188716459 shutdown_connections 2023-11-28T11:09:21.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.398+0000 7f52e3922700 1 --2- 172.21.15.156:0/1188716459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52dc0b2e70 0x7f52dc06f850 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.398+0000 7f52e3922700 1 --2- 172.21.15.156:0/1188716459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52dc140950 0x7f52dc0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.398+0000 7f52e3922700 1 -- 172.21.15.156:0/1188716459 >> 172.21.15.156:0/1188716459 conn(0x7f52dc06aed0 msgr2=0x7f52dc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.398+0000 7f52e3922700 1 -- 172.21.15.156:0/1188716459 shutdown_connections 2023-11-28T11:09:21.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.398+0000 7f52e3922700 1 -- 172.21.15.156:0/1188716459 wait complete. 2023-11-28T11:09:21.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.399+0000 7f52e3922700 1 Processor -- start 2023-11-28T11:09:21.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.399+0000 7f52e3922700 1 -- start start 2023-11-28T11:09:21.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e3922700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52dc0b2e70 0x7f52dc0c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e3922700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52dc0cd8c0 0x7f52dc0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e3922700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52dc140950 0x7f52dc143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e3922700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc06bc90 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e3922700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc143eb0 con 0x7f52dc140950 2023-11-28T11:09:21.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e3922700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc06b380 con 0x7f52dc0b2e70 2023-11-28T11:09:21.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e0ebd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52dc0cd8c0 0x7f52dc0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e0ebd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52dc0cd8c0 0x7f52dc0b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33904/0 (socket says 172.21.15.156:33904) 2023-11-28T11:09:21.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e0ebd700 1 -- 172.21.15.156:0/2010683698 learned_addr learned my addr 172.21.15.156:0/2010683698 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:21.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e16be700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52dc0b2e70 0x7f52dc0c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e1ebf700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52dc140950 0x7f52dc143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e0ebd700 1 -- 172.21.15.156:0/2010683698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52dc0b2e70 msgr2=0x7f52dc0c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e0ebd700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52dc0b2e70 0x7f52dc0c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e0ebd700 1 -- 172.21.15.156:0/2010683698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52dc140950 msgr2=0x7f52dc143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e0ebd700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52dc140950 0x7f52dc143970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.400+0000 7f52e0ebd700 1 -- 172.21.15.156:0/2010683698 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d4009b30 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.401+0000 7f52e0ebd700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52dc0cd8c0 0x7f52dc0b2370 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00e840 tx=0x7f52cc00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.401+0000 7f52d1ffb700 1 -- 172.21.15.156:0/2010683698 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc009590 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.401+0000 7f52d1ffb700 1 -- 172.21.15.156:0/2010683698 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc013680 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.401+0000 7f52d1ffb700 1 -- 172.21.15.156:0/2010683698 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc013a10 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.401+0000 7f52e3922700 1 -- 172.21.15.156:0/2010683698 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc0b35f0 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.401+0000 7f52e3922700 1 -- 172.21.15.156:0/2010683698 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc0b3310 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.402+0000 7f52e3922700 1 -- 172.21.15.156:0/2010683698 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52dc140150 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.402+0000 7f52d1ffb700 1 -- 172.21.15.156:0/2010683698 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f52cc019030 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.403+0000 7f52d1ffb700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52c0068320 0x7f52c006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.403+0000 7f52d1ffb700 1 -- 172.21.15.156:0/2010683698 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52cc0858e0 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.403+0000 7f52e16be700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52c0068320 0x7f52c006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:21.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.403+0000 7f52e16be700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52c0068320 0x7f52c006a7d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f52d8000fb0 tx=0x7f52d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:21.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.407+0000 7f52d1ffb700 1 -- 172.21.15.156:0/2010683698 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f52cc052140 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.634+0000 7f52e3922700 1 -- 172.21.15.156:0/2010683698 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f52dc13fcf0 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.634+0000 7f52d1ffb700 1 -- 172.21.15.156:0/2010683698 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f52cc055760 con 0x7f52dc0cd8c0 2023-11-28T11:09:21.635 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:09:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.636+0000 7f52e3922700 1 -- 172.21.15.156:0/2010683698 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52c0068320 msgr2=0x7f52c006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.636+0000 7f52e3922700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52c0068320 0x7f52c006a7d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f52d8000fb0 tx=0x7f52d8009040 comp rx=0 tx=0).stop 2023-11-28T11:09:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.636+0000 7f52e3922700 1 -- 172.21.15.156:0/2010683698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52dc0cd8c0 msgr2=0x7f52dc0b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.636+0000 7f52e3922700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52dc0cd8c0 0x7f52dc0b2370 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00e840 tx=0x7f52cc00eb50 comp rx=0 tx=0).stop 2023-11-28T11:09:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.636+0000 7f52e3922700 1 -- 172.21.15.156:0/2010683698 shutdown_connections 2023-11-28T11:09:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.636+0000 7f52e3922700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52dc140950 0x7f52dc143970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.636+0000 7f52e3922700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f52c0068320 0x7f52c006a7d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.636+0000 7f52e3922700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52dc0b2e70 0x7f52dc0c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.636+0000 7f52e3922700 1 --2- 172.21.15.156:0/2010683698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52dc0cd8c0 0x7f52dc0b2370 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.636+0000 7f52e3922700 1 -- 172.21.15.156:0/2010683698 >> 172.21.15.156:0/2010683698 conn(0x7f52dc06aed0 msgr2=0x7f52dc0cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:21.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.636+0000 7f52e3922700 1 -- 172.21.15.156:0/2010683698 shutdown_connections 2023-11-28T11:09:21.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:21.636+0000 7f52e3922700 1 -- 172.21.15.156:0/2010683698 wait complete. 2023-11-28T11:09:21.649 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:09:21.649 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:09:21.649 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-28T11:09:21.649 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:09:21.649 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:21.650 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:09:21.650 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:09:21.650 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:09:21.650 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:09:21.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.650 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-28T11:09:21.668 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:21.668 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:21.668 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:21.668 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.668 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:21.668 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.668 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:21.669 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:21.669 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:21.669 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:21.669 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:21.669 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:21.669 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:09:21.669 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:21.669 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:09:21.669 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42321sec preferred_lft 42321sec 2023-11-28T11:09:21.669 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::c260:d776:a692:b0cc/64 scope link noprefixroute 2023-11-28T11:09:21.670 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.670 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:21.670 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 8e:07:21:2f:e8:e5 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:21.670 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:09:21.670 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.670 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::a871:90ff:feb9:397d/64 scope link 2023-11-28T11:09:21.670 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.670 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:09:21.670 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 8e:07:21:2f:e8:e5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:09:21.670 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::8c07:21ff:fe2f:e8e5/64 scope link 2023-11-28T11:09:21.671 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.671 INFO:teuthology.orchestra.run.smithi156.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:09:21.671 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether d2:39:d6:e9:98:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:09:21.671 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::d039:d6ff:fee9:985b/64 scope link 2023-11-28T11:09:21.671 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.671 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:09:21.734 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-28T11:09:21.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.736 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-28T11:09:21.795 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-28T11:09:21.795 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:09:21.795 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:09:21.795 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:09:21.795 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:09:21.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.796 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T11:09:21.796 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T11:09:21.796 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:09:21.796 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:09:21.796 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:09:21.796 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T11:09:21.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.901 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T11:09:21.918 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:09:21.918 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:09:21.918 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:09:21.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.919 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-28T11:09:21.985 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:21.985 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:22.042 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:09:22.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:22.042 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:22.098 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:09:22.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.099 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:09:22.185 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:09:22.185 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:09:22.185 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:09:22.194 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:09:22.194 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:09:22.194 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:09:22.194 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:09:22.194 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:09:22.269 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:09:22.269 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:09:22.269 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:09:22.269 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:09:22.269 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:09:22.269 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T11:09:22.270 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T11:09:22.270 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=351da896-8dbf-473b-9472-c3454d9368bf" will pass to kernel 2023-11-28T11:09:22.271 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:09:22.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.272 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:22.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.299 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:09:22.387 INFO:teuthology.orchestra.run.smithi156.stdout:client5047 2023-11-28T11:09:22.387 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:09:22.387 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:22.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.405 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2023-11-28T11:09:22.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.468 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2023-11-28T11:09:22.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.527 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2023-11-28T11:09:22.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.585 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2023-11-28T11:09:22.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.647 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2023-11-28T11:09:22.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.704 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-28T11:09:22.766 INFO:teuthology.orchestra.run.smithi156.stdout:./ 2023-11-28T11:09:22.766 INFO:teuthology.orchestra.run.smithi156.stdout:./rfile 2023-11-28T11:09:22.766 INFO:teuthology.orchestra.run.smithi156.stdout:./parentdir 2023-11-28T11:09:22.766 INFO:teuthology.orchestra.run.smithi156.stdout:./parentdir/pfile 2023-11-28T11:09:22.766 INFO:teuthology.orchestra.run.smithi156.stdout:./parentdir/childdir 2023-11-28T11:09:22.767 INFO:teuthology.orchestra.run.smithi156.stdout:./parentdir/childdir/cfile 2023-11-28T11:09:22.767 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:22.767 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T11:09:22.767 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T11:09:22.767 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T11:09:22.767 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-28T11:09:22.767 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T11:09:22.873 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-28T11:09:23.062 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:23.062 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T11:09:23.062 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T11:09:23.062 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T11:09:23.063 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2023-11-28T11:09:23.063 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T11:09:23.169 INFO:teuthology.orchestra.run.smithi156.stdout:1099511627778 2023-11-28T11:09:23.358 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:23.358 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T11:09:23.358 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T11:09:23.358 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T11:09:23.359 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2023-11-28T11:09:23.359 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T11:09:23.465 INFO:teuthology.orchestra.run.smithi156.stdout:1099511627776 2023-11-28T11:09:23.654 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:23.654 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T11:09:23.654 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T11:09:23.654 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T11:09:23.654 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2023-11-28T11:09:23.655 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T11:09:23.760 INFO:teuthology.orchestra.run.smithi156.stdout:1099511628279 2023-11-28T11:09:23.950 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:23.950 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T11:09:23.950 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T11:09:23.950 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T11:09:23.950 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2023-11-28T11:09:23.950 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T11:09:24.057 INFO:teuthology.orchestra.run.smithi156.stdout:1099511627777 2023-11-28T11:09:24.247 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:24.247 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T11:09:24.247 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T11:09:24.247 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T11:09:24.247 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2023-11-28T11:09:24.247 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T11:09:24.353 INFO:teuthology.orchestra.run.smithi156.stdout:1099511628280 2023-11-28T11:09:24.542 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:09:24.542 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:24.601 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:09:24.601 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:24.659 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:24.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.820+0000 7f2411b06700 1 Processor -- start 2023-11-28T11:09:24.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.820+0000 7f2411b06700 1 -- start start 2023-11-28T11:09:24.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.820+0000 7f2411b06700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f240c0cbdf0 0x7f240c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.820+0000 7f2411b06700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240c0cc5f0 con 0x7f240c15cef0 2023-11-28T11:09:24.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.820+0000 7f2411b06700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240c15d470 con 0x7f240c06ba90 2023-11-28T11:09:24.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.820+0000 7f2411b06700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240c15d750 con 0x7f240c0cbdf0 2023-11-28T11:09:24.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.821+0000 7f240b7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f240c15cef0 0x7f240c06b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48794/0 (socket says 172.21.15.156:48794) 2023-11-28T11:09:24.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.821+0000 7f240b7fe700 1 -- 172.21.15.156:0/2968770301 learned_addr learned my addr 172.21.15.156:0/2968770301 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:24.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.821+0000 7f240bfff700 1 --2- 172.21.15.156:0/2968770301 >> v2:172.21.15.186:3300/0 conn(0x7f240c0cbdf0 0x7f240c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.821+0000 7f240a7fc700 1 -- 172.21.15.156:0/2968770301 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985884271 0 0) 0x7f240c0cc5f0 con 0x7f240c15cef0 2023-11-28T11:09:24.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.821+0000 7f240a7fc700 1 -- 172.21.15.156:0/2968770301 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23f40035b0 con 0x7f240c15cef0 2023-11-28T11:09:24.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.821+0000 7f240a7fc700 1 -- 172.21.15.156:0/2968770301 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2198963842 0 0) 0x7f23f40035b0 con 0x7f240c15cef0 2023-11-28T11:09:24.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.821+0000 7f240bfff700 1 -- 172.21.15.156:0/2968770301 >> v1:172.21.15.186:6789/0 conn(0x7f240c06ba90 legacy=0x7f240c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.821+0000 7f240bfff700 1 -- 172.21.15.156:0/2968770301 >> v1:172.21.15.156:6789/0 conn(0x7f240c15cef0 legacy=0x7f240c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.821+0000 7f240bfff700 1 -- 172.21.15.156:0/2968770301 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c15da30 con 0x7f240c0cbdf0 2023-11-28T11:09:24.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.821+0000 7f240bfff700 1 --2- 172.21.15.156:0/2968770301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c0cbdf0 0x7f240c0af400 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2404009d90 tx=0x7f240400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5981046ce49070ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.821+0000 7f240a7fc700 1 -- 172.21.15.156:0/2968770301 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240400e070 con 0x7f240c0cbdf0 2023-11-28T11:09:24.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.822+0000 7f240a7fc700 1 -- 172.21.15.156:0/2968770301 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2404003a20 con 0x7f240c0cbdf0 2023-11-28T11:09:24.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.822+0000 7f240a7fc700 1 -- 172.21.15.156:0/2968770301 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc001670 con 0x7f240c0cbdf0 2023-11-28T11:09:24.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.822+0000 7f2411b06700 1 -- 172.21.15.156:0/2968770301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c0cbdf0 msgr2=0x7f240c0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.822+0000 7f2411b06700 1 --2- 172.21.15.156:0/2968770301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c0cbdf0 0x7f240c0af400 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2404009d90 tx=0x7f240400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:24.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.822+0000 7f2411b06700 1 -- 172.21.15.156:0/2968770301 shutdown_connections 2023-11-28T11:09:24.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.822+0000 7f2411b06700 1 --2- 172.21.15.156:0/2968770301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c0cbdf0 0x7f240c0af400 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.822+0000 7f2411b06700 1 -- 172.21.15.156:0/2968770301 >> 172.21.15.156:0/2968770301 conn(0x7f240c0d0000 msgr2=0x7f240c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:24.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.822+0000 7f2411b06700 1 -- 172.21.15.156:0/2968770301 shutdown_connections 2023-11-28T11:09:24.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.822+0000 7f2411b06700 1 -- 172.21.15.156:0/2968770301 wait complete. 2023-11-28T11:09:24.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.823+0000 7f2411b06700 1 Processor -- start 2023-11-28T11:09:24.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.823+0000 7f2411b06700 1 -- start start 2023-11-28T11:09:24.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.824+0000 7f2411b06700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f240c06ba90 0x7f240c0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.824+0000 7f2411b06700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c0cbdf0 0x7f240c0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.824+0000 7f2411b06700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f240c15cef0 0x7f240c1eee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.824+0000 7f2411b06700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0b47f0 con 0x7f240c15cef0 2023-11-28T11:09:24.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.824+0000 7f2411b06700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0c70d0 con 0x7f240c0cbdf0 2023-11-28T11:09:24.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.824+0000 7f2411b06700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0c7370 con 0x7f240c06ba90 2023-11-28T11:09:24.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.824+0000 7f240bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f240c15cef0 0x7f240c1eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.824+0000 7f240bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f240c15cef0 0x7f240c1eee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33924/0 (socket says 172.21.15.156:33924) 2023-11-28T11:09:24.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.824+0000 7f240bfff700 1 -- 172.21.15.156:0/1118653266 learned_addr learned my addr 172.21.15.156:0/1118653266 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:24.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.824+0000 7f240b7fe700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f240c06ba90 0x7f240c0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.824+0000 7f240affd700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c0cbdf0 0x7f240c0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f240bfff700 1 -- 172.21.15.156:0/1118653266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f240c06ba90 msgr2=0x7f240c0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f240bfff700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f240c06ba90 0x7f240c0c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f240bfff700 1 -- 172.21.15.156:0/1118653266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c0cbdf0 msgr2=0x7f240c0c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f240bfff700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c0cbdf0 0x7f240c0c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f240bfff700 1 -- 172.21.15.156:0/1118653266 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2404009a70 con 0x7f240c15cef0 2023-11-28T11:09:24.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f240b7fe700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f240c06ba90 0x7f240c0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:24.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f240affd700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c0cbdf0 0x7f240c0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:24.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f240bfff700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f240c15cef0 0x7f240c1eee70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f240c06c7b0 tx=0x7f2404004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f23f3fff700 1 -- 172.21.15.156:0/1118653266 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240401f070 con 0x7f240c15cef0 2023-11-28T11:09:24.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f23f3fff700 1 -- 172.21.15.156:0/1118653266 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24040043c0 con 0x7f240c15cef0 2023-11-28T11:09:24.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f2411b06700 1 -- 172.21.15.156:0/1118653266 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c15dff0 con 0x7f240c15cef0 2023-11-28T11:09:24.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f23f3fff700 1 -- 172.21.15.156:0/1118653266 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404004770 con 0x7f240c15cef0 2023-11-28T11:09:24.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.825+0000 7f2411b06700 1 -- 172.21.15.156:0/1118653266 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240c15dd10 con 0x7f240c15cef0 2023-11-28T11:09:24.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.826+0000 7f2411b06700 1 -- 172.21.15.156:0/1118653266 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240c0cad60 con 0x7f240c15cef0 2023-11-28T11:09:24.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.826+0000 7f23f3fff700 1 -- 172.21.15.156:0/1118653266 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2404019470 con 0x7f240c15cef0 2023-11-28T11:09:24.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.827+0000 7f23f3fff700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f23f80682f0 0x7f23f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.827+0000 7f23f3fff700 1 -- 172.21.15.156:0/1118653266 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2404088f60 con 0x7f240c15cef0 2023-11-28T11:09:24.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.827+0000 7f240b7fe700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f23f80682f0 0x7f23f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.827+0000 7f240b7fe700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f23f80682f0 0x7f23f806a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f23fc001eb0 tx=0x7f23fc001c80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:24.831+0000 7f23f3fff700 1 -- 172.21.15.156:0/1118653266 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2404055670 con 0x7f240c15cef0 2023-11-28T11:09:25.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.016+0000 7f2411b06700 1 -- 172.21.15.156:0/1118653266 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f240c0af840 con 0x7f240c15cef0 2023-11-28T11:09:25.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.017+0000 7f23f3fff700 1 -- 172.21.15.156:0/1118653266 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f2404058c90 con 0x7f240c15cef0 2023-11-28T11:09:25.018 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:25.018 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":37,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:16.393008+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:13.663041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:14.347248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:25.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.019+0000 7f2411b06700 1 -- 172.21.15.156:0/1118653266 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f23f80682f0 msgr2=0x7f23f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.019+0000 7f2411b06700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f23f80682f0 0x7f23f806a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f23fc001eb0 tx=0x7f23fc001c80 comp rx=0 tx=0).stop 2023-11-28T11:09:25.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.019+0000 7f2411b06700 1 -- 172.21.15.156:0/1118653266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f240c15cef0 msgr2=0x7f240c1eee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.019+0000 7f2411b06700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f240c15cef0 0x7f240c1eee70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f240c06c7b0 tx=0x7f2404004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.019+0000 7f2411b06700 1 -- 172.21.15.156:0/1118653266 shutdown_connections 2023-11-28T11:09:25.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.019+0000 7f2411b06700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f240c15cef0 0x7f240c1eee70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.019+0000 7f2411b06700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f23f80682f0 0x7f23f806a7a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.019+0000 7f2411b06700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f240c06ba90 0x7f240c0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.019+0000 7f2411b06700 1 --2- 172.21.15.156:0/1118653266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f240c0cbdf0 0x7f240c0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.019+0000 7f2411b06700 1 -- 172.21.15.156:0/1118653266 >> 172.21.15.156:0/1118653266 conn(0x7f240c0d0000 msgr2=0x7f240c06b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.019+0000 7f2411b06700 1 -- 172.21.15.156:0/1118653266 shutdown_connections 2023-11-28T11:09:25.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.019+0000 7f2411b06700 1 -- 172.21.15.156:0/1118653266 wait complete. 2023-11-28T11:09:25.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:09:13.663041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:09:14.347248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:09:25.033 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:25.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.154+0000 7f718d377700 1 Processor -- start 2023-11-28T11:09:25.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.154+0000 7f718d377700 1 -- start start 2023-11-28T11:09:25.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.155+0000 7f718d377700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7188158ce0 0x7f71880b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.155+0000 7f718d377700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718806bf10 con 0x7f71880cdda0 2023-11-28T11:09:25.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.155+0000 7f718d377700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71880cb0d0 con 0x7f718806fb30 2023-11-28T11:09:25.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.155+0000 7f718d377700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718806ff00 con 0x7f7188158ce0 2023-11-28T11:09:25.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.155+0000 7f71877fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f71880cdda0 0x7f718806c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48810/0 (socket says 172.21.15.156:48810) 2023-11-28T11:09:25.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.155+0000 7f71877fe700 1 -- 172.21.15.156:0/1507358875 learned_addr learned my addr 172.21.15.156:0/1507358875 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:25.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f71867fc700 1 --2- 172.21.15.156:0/1507358875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7188158ce0 0x7f71880b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769890851 0 0) 0x7f718806bf10 con 0x7f71880cdda0 2023-11-28T11:09:25.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71700035b0 con 0x7f71880cdda0 2023-11-28T11:09:25.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3877177337 0 0) 0x7f71700035b0 con 0x7f71880cdda0 2023-11-28T11:09:25.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7188158ce0 msgr2=0x7f71880b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 --2- 172.21.15.156:0/1507358875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7188158ce0 0x7f71880b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 >> v1:172.21.15.186:6789/0 conn(0x7f718806fb30 legacy=0x7f718815d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71880cb3b0 con 0x7f71880cdda0 2023-11-28T11:09:25.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f71800019f0 con 0x7f71880cdda0 2023-11-28T11:09:25.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 >> v1:172.21.15.156:6789/0 conn(0x7f71880cdda0 legacy=0x7f718806c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 --2- 172.21.15.156:0/1507358875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7188158ce0 0x7f7170003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 --2- 172.21.15.156:0/1507358875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7170003fb0 0x7f7170004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 --2- 172.21.15.156:0/1507358875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7170004d90 0x7f7170005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718806ff00 con 0x7f7170003fb0 2023-11-28T11:09:25.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717000baf0 con 0x7f7170004d90 2023-11-28T11:09:25.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717000bcb0 con 0x7f7188158ce0 2023-11-28T11:09:25.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f71867fc700 1 --2- 172.21.15.156:0/1507358875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7170003fb0 0x7f7170004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f7186ffd700 1 --2- 172.21.15.156:0/1507358875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7188158ce0 0x7f7170003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f71877fe700 1 --2- 172.21.15.156:0/1507358875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7170004d90 0x7f7170005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f71867fc700 1 -- 172.21.15.156:0/1507358875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7188158ce0 msgr2=0x7f7170003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f71867fc700 1 --2- 172.21.15.156:0/1507358875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7188158ce0 0x7f7170003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f71867fc700 1 -- 172.21.15.156:0/1507358875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7170004d90 msgr2=0x7f7170005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f71867fc700 1 --2- 172.21.15.156:0/1507358875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7170004d90 0x7f7170005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.156+0000 7f71867fc700 1 -- 172.21.15.156:0/1507358875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71700035b0 con 0x7f7170003fb0 2023-11-28T11:09:25.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.157+0000 7f71867fc700 1 --2- 172.21.15.156:0/1507358875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7170003fb0 0x7f7170004850 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7178009b30 tx=0x7f7170005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95848fef8dfb8ea4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.157+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800e6c0 con 0x7f7170003fb0 2023-11-28T11:09:25.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.157+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717800e820 con 0x7f7170003fb0 2023-11-28T11:09:25.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.157+0000 7f7185ffb700 1 -- 172.21.15.156:0/1507358875 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800eb90 con 0x7f7170003fb0 2023-11-28T11:09:25.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.157+0000 7f718d377700 1 -- 172.21.15.156:0/1507358875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7170003fb0 msgr2=0x7f7170004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.157+0000 7f718d377700 1 --2- 172.21.15.156:0/1507358875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7170003fb0 0x7f7170004850 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7178009b30 tx=0x7f7170005880 comp rx=0 tx=0).stop 2023-11-28T11:09:25.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.157+0000 7f71877fe700 1 -- 172.21.15.156:0/1507358875 reap_dead start 2023-11-28T11:09:25.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.157+0000 7f718d377700 1 -- 172.21.15.156:0/1507358875 shutdown_connections 2023-11-28T11:09:25.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.157+0000 7f718d377700 1 -- 172.21.15.156:0/1507358875 >> 172.21.15.156:0/1507358875 conn(0x7f71880cb670 msgr2=0x7f71880cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.157+0000 7f718d377700 1 -- 172.21.15.156:0/1507358875 shutdown_connections 2023-11-28T11:09:25.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.157+0000 7f718d377700 1 -- 172.21.15.156:0/1507358875 wait complete. 2023-11-28T11:09:25.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.159+0000 7f718d377700 1 Processor -- start 2023-11-28T11:09:25.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.159+0000 7f718d377700 1 -- start start 2023-11-28T11:09:25.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.159+0000 7f718d377700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7170003fb0 0x7f71880c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.159+0000 7f718d377700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718815a010 0x7f718814afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.159+0000 7f718d377700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71880c4bc0 0x7f71880c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.159+0000 7f718d377700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7188159180 con 0x7f718815a010 2023-11-28T11:09:25.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.159+0000 7f718d377700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71880c55a0 con 0x7f71880c4bc0 2023-11-28T11:09:25.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.159+0000 7f718d377700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71880c2d90 con 0x7f7170003fb0 2023-11-28T11:09:25.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f71867fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718815a010 0x7f718814afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f71867fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718815a010 0x7f718814afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33940/0 (socket says 172.21.15.156:33940) 2023-11-28T11:09:25.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f71867fc700 1 -- 172.21.15.156:0/3976848947 learned_addr learned my addr 172.21.15.156:0/3976848947 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:25.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f7186ffd700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7170003fb0 0x7f71880c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f71877fe700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71880c4bc0 0x7f71880c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f71867fc700 1 -- 172.21.15.156:0/3976848947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7170003fb0 msgr2=0x7f71880c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f71867fc700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7170003fb0 0x7f71880c2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f71867fc700 1 -- 172.21.15.156:0/3976848947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71880c4bc0 msgr2=0x7f71880c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f71867fc700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71880c4bc0 0x7f71880c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f71867fc700 1 -- 172.21.15.156:0/3976848947 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718806bba0 con 0x7f718815a010 2023-11-28T11:09:25.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f71867fc700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718815a010 0x7f718814afc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f717000bac0 tx=0x7f71780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f716f7fe700 1 -- 172.21.15.156:0/3976848947 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800e6c0 con 0x7f718815a010 2023-11-28T11:09:25.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f716f7fe700 1 -- 172.21.15.156:0/3976848947 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7178003ad0 con 0x7f718815a010 2023-11-28T11:09:25.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f716f7fe700 1 -- 172.21.15.156:0/3976848947 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178003e80 con 0x7f718815a010 2023-11-28T11:09:25.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f718d377700 1 -- 172.21.15.156:0/3976848947 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718806b8c0 con 0x7f718815a010 2023-11-28T11:09:25.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.160+0000 7f718d377700 1 -- 172.21.15.156:0/3976848947 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71880c9370 con 0x7f718815a010 2023-11-28T11:09:25.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.161+0000 7f718d377700 1 -- 172.21.15.156:0/3976848947 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71880c91e0 con 0x7f718815a010 2023-11-28T11:09:25.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.162+0000 7f716f7fe700 1 -- 172.21.15.156:0/3976848947 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7178025070 con 0x7f718815a010 2023-11-28T11:09:25.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.162+0000 7f716f7fe700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71740682f0 0x7f717406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.162+0000 7f716f7fe700 1 -- 172.21.15.156:0/3976848947 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f71780875b0 con 0x7f718815a010 2023-11-28T11:09:25.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.162+0000 7f7186ffd700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71740682f0 0x7f717406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.162+0000 7f7186ffd700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71740682f0 0x7f717406a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f717c000c00 tx=0x7f717c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.166+0000 7f716f7fe700 1 -- 172.21.15.156:0/3976848947 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7178053e10 con 0x7f718815a010 2023-11-28T11:09:25.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.348+0000 7f718d377700 1 -- 172.21.15.156:0/3976848947 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71880745c0 con 0x7f718815a010 2023-11-28T11:09:25.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.348+0000 7f716f7fe700 1 -- 172.21.15.156:0/3976848947 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7178057430 con 0x7f718815a010 2023-11-28T11:09:25.350 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:09:25.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.350+0000 7f718d377700 1 -- 172.21.15.156:0/3976848947 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71740682f0 msgr2=0x7f717406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.350+0000 7f718d377700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71740682f0 0x7f717406a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f717c000c00 tx=0x7f717c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:25.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.350+0000 7f718d377700 1 -- 172.21.15.156:0/3976848947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718815a010 msgr2=0x7f718814afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.350+0000 7f718d377700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718815a010 0x7f718814afc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f717000bac0 tx=0x7f71780047c0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.351+0000 7f718d377700 1 -- 172.21.15.156:0/3976848947 shutdown_connections 2023-11-28T11:09:25.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.351+0000 7f718d377700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71880c4bc0 0x7f71880c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.351+0000 7f718d377700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71740682f0 0x7f717406a7a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.351+0000 7f718d377700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7170003fb0 0x7f71880c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.351+0000 7f718d377700 1 --2- 172.21.15.156:0/3976848947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718815a010 0x7f718814afc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.351+0000 7f718d377700 1 -- 172.21.15.156:0/3976848947 >> 172.21.15.156:0/3976848947 conn(0x7f71880cb670 msgr2=0x7f718815dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.351+0000 7f718d377700 1 -- 172.21.15.156:0/3976848947 shutdown_connections 2023-11-28T11:09:25.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.351+0000 7f718d377700 1 -- 172.21.15.156:0/3976848947 wait complete. 2023-11-28T11:09:25.364 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:25.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.486+0000 7f8fc6046700 1 Processor -- start 2023-11-28T11:09:25.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.486+0000 7f8fc6046700 1 -- start start 2023-11-28T11:09:25.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fc6046700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8fc0160b10 0x7f8fc00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fc6046700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc006bbd0 con 0x7f8fc0160740 2023-11-28T11:09:25.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fc6046700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc00c9e10 con 0x7f8fc006b060 2023-11-28T11:09:25.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fc6046700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc00ca720 con 0x7f8fc0160b10 2023-11-28T11:09:25.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fbeffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8fc0160b10 0x7f8fc00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fbf7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f8fc0160740 0x7f8fc00cbaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48826/0 (socket says 172.21.15.156:48826) 2023-11-28T11:09:25.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fbf7fe700 1 -- 172.21.15.156:0/1414719667 learned_addr learned my addr 172.21.15.156:0/1414719667 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:25.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fbeffd700 1 -- 172.21.15.156:0/1414719667 >> v1:172.21.15.186:6789/0 conn(0x7f8fc006b060 legacy=0x7f8fc006b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fbeffd700 1 -- 172.21.15.156:0/1414719667 >> v1:172.21.15.156:6789/0 conn(0x7f8fc0160740 legacy=0x7f8fc00cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fbe7fc700 1 -- 172.21.15.156:0/1414719667 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967142300 0 0) 0x7f8fc006bbd0 con 0x7f8fc0160740 2023-11-28T11:09:25.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fbeffd700 1 -- 172.21.15.156:0/1414719667 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc006efc0 con 0x7f8fc0160b10 2023-11-28T11:09:25.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fbeffd700 1 --2- 172.21.15.156:0/1414719667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0160b10 0x7f8fc00cd390 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009840 tx=0x7f8fb4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c432cbf6b513c16 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fbe7fc700 1 -- 172.21.15.156:0/1414719667 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4011830 con 0x7f8fc0160b10 2023-11-28T11:09:25.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fbe7fc700 1 -- 172.21.15.156:0/1414719667 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb4011990 con 0x7f8fc0160b10 2023-11-28T11:09:25.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.487+0000 7f8fbe7fc700 1 -- 172.21.15.156:0/1414719667 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4011d20 con 0x7f8fc0160b10 2023-11-28T11:09:25.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.488+0000 7f8fc6046700 1 -- 172.21.15.156:0/1414719667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0160b10 msgr2=0x7f8fc00cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.488+0000 7f8fc6046700 1 --2- 172.21.15.156:0/1414719667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0160b10 0x7f8fc00cd390 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009840 tx=0x7f8fb4009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:25.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.488+0000 7f8fc6046700 1 -- 172.21.15.156:0/1414719667 shutdown_connections 2023-11-28T11:09:25.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.488+0000 7f8fc6046700 1 --2- 172.21.15.156:0/1414719667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0160b10 0x7f8fc00cd390 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.488+0000 7f8fc6046700 1 -- 172.21.15.156:0/1414719667 >> 172.21.15.156:0/1414719667 conn(0x7f8fc00b12b0 msgr2=0x7f8fc00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.488+0000 7f8fc6046700 1 -- 172.21.15.156:0/1414719667 shutdown_connections 2023-11-28T11:09:25.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.488+0000 7f8fc6046700 1 -- 172.21.15.156:0/1414719667 wait complete. 2023-11-28T11:09:25.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.489+0000 7f8fc6046700 1 Processor -- start 2023-11-28T11:09:25.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.490+0000 7f8fc6046700 1 -- start start 2023-11-28T11:09:25.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.490+0000 7f8fc6046700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fc006b060 0x7f8fc0147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.490+0000 7f8fc6046700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8fc0160740 0x7f8fc0148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.490+0000 7f8fc6046700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0160b10 0x7f8fc00c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.490+0000 7f8fc6046700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc015bfd0 con 0x7f8fc0160b10 2023-11-28T11:09:25.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.490+0000 7f8fc6046700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc014a6c0 con 0x7f8fc006b060 2023-11-28T11:09:25.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.490+0000 7f8fc6046700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00c56f0 con 0x7f8fc0160740 2023-11-28T11:09:25.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.490+0000 7f8fbffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0160b10 0x7f8fc00c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.490+0000 7f8fbffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0160b10 0x7f8fc00c6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33958/0 (socket says 172.21.15.156:33958) 2023-11-28T11:09:25.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.490+0000 7f8fbffff700 1 -- 172.21.15.156:0/570271124 learned_addr learned my addr 172.21.15.156:0/570271124 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:25.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fbeffd700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8fc0160740 0x7f8fc0148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fbf7fe700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fc006b060 0x7f8fc0147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fbffff700 1 -- 172.21.15.156:0/570271124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8fc0160740 msgr2=0x7f8fc0148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fbffff700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8fc0160740 0x7f8fc0148630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fbffff700 1 -- 172.21.15.156:0/570271124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fc006b060 msgr2=0x7f8fc0147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fbffff700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fc006b060 0x7f8fc0147ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fbffff700 1 -- 172.21.15.156:0/570271124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb40094a0 con 0x7f8fc0160b10 2023-11-28T11:09:25.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fbffff700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0160b10 0x7f8fc00c6e10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800cb50 tx=0x7f8fb800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fa7fff700 1 -- 172.21.15.156:0/570271124 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8003c00 con 0x7f8fc0160b10 2023-11-28T11:09:25.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fa7fff700 1 -- 172.21.15.156:0/570271124 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb8003d60 con 0x7f8fc0160b10 2023-11-28T11:09:25.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fa7fff700 1 -- 172.21.15.156:0/570271124 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800ab70 con 0x7f8fc0160b10 2023-11-28T11:09:25.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fc6046700 1 -- 172.21.15.156:0/570271124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc00ca3b0 con 0x7f8fc0160b10 2023-11-28T11:09:25.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.491+0000 7f8fc6046700 1 -- 172.21.15.156:0/570271124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc00ca0d0 con 0x7f8fc0160b10 2023-11-28T11:09:25.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.492+0000 7f8fc6046700 1 -- 172.21.15.156:0/570271124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc00cd8d0 con 0x7f8fc0160b10 2023-11-28T11:09:25.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.492+0000 7f8fa7fff700 1 -- 172.21.15.156:0/570271124 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8fb8010050 con 0x7f8fc0160b10 2023-11-28T11:09:25.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.493+0000 7f8fa7fff700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8fac068320 0x7f8fac06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.493+0000 7f8fa7fff700 1 -- 172.21.15.156:0/570271124 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8fb8085b20 con 0x7f8fc0160b10 2023-11-28T11:09:25.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.493+0000 7f8fbf7fe700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8fac068320 0x7f8fac06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.493+0000 7f8fbf7fe700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8fac068320 0x7f8fac06a7d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0000f60 tx=0x7f8fb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.497+0000 7f8fa7fff700 1 -- 172.21.15.156:0/570271124 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8fb8052380 con 0x7f8fc0160b10 2023-11-28T11:09:25.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.679+0000 7f8fc6046700 1 -- 172.21.15.156:0/570271124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8fc014a850 con 0x7f8fc0160b10 2023-11-28T11:09:25.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.680+0000 7f8fa7fff700 1 -- 172.21.15.156:0/570271124 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8fb80559a0 con 0x7f8fc0160b10 2023-11-28T11:09:25.681 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:09:25.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.681+0000 7f8fc6046700 1 -- 172.21.15.156:0/570271124 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8fac068320 msgr2=0x7f8fac06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.681+0000 7f8fc6046700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8fac068320 0x7f8fac06a7d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0000f60 tx=0x7f8fb0009040 comp rx=0 tx=0).stop 2023-11-28T11:09:25.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.681+0000 7f8fc6046700 1 -- 172.21.15.156:0/570271124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0160b10 msgr2=0x7f8fc00c6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.681+0000 7f8fc6046700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0160b10 0x7f8fc00c6e10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800cb50 tx=0x7f8fb800cf10 comp rx=0 tx=0).stop 2023-11-28T11:09:25.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.681+0000 7f8fc6046700 1 -- 172.21.15.156:0/570271124 shutdown_connections 2023-11-28T11:09:25.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.681+0000 7f8fc6046700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0160b10 0x7f8fc00c6e10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.681+0000 7f8fc6046700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fc006b060 0x7f8fc0147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.681+0000 7f8fc6046700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8fac068320 0x7f8fac06a7d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.682+0000 7f8fc6046700 1 --2- 172.21.15.156:0/570271124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8fc0160740 0x7f8fc0148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.682+0000 7f8fc6046700 1 -- 172.21.15.156:0/570271124 >> 172.21.15.156:0/570271124 conn(0x7f8fc00b12b0 msgr2=0x7f8fc00ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.682+0000 7f8fc6046700 1 -- 172.21.15.156:0/570271124 shutdown_connections 2023-11-28T11:09:25.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.682+0000 7f8fc6046700 1 -- 172.21.15.156:0/570271124 wait complete. 2023-11-28T11:09:25.695 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:25.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.818+0000 7feceb528700 1 Processor -- start 2023-11-28T11:09:25.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.818+0000 7feceb528700 1 -- start start 2023-11-28T11:09:25.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7feceb528700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fece4160b10 0x7fece40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7feceb528700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fece406b060 0x7fece406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7feceb528700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece40c9e10 con 0x7fece4160740 2023-11-28T11:09:25.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7feceb528700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece40ca720 con 0x7fece4160b10 2023-11-28T11:09:25.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7feceb528700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece406efc0 con 0x7fece406b060 2023-11-28T11:09:25.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7fece9ac5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fece406b060 0x7fece406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7fece9ac5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fece406b060 0x7fece406b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37830/0 (socket says 172.21.15.156:37830) 2023-11-28T11:09:25.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7fece9ac5700 1 -- 172.21.15.156:0/53262623 learned_addr learned my addr 172.21.15.156:0/53262623 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:25.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7fece8ac3700 1 --2- 172.21.15.156:0/53262623 >> v2:172.21.15.186:3300/0 conn(0x7fece4160b10 0x7fece40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7fece9ac5700 1 -- 172.21.15.156:0/53262623 >> v2:172.21.15.186:3300/0 conn(0x7fece4160b10 msgr2=0x7fece40cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7fece9ac5700 1 --2- 172.21.15.156:0/53262623 >> v2:172.21.15.186:3300/0 conn(0x7fece4160b10 0x7fece40cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7fece9ac5700 1 -- 172.21.15.156:0/53262623 >> v1:172.21.15.186:6789/0 conn(0x7fece4160740 legacy=0x7fece40cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7fece9ac5700 1 -- 172.21.15.156:0/53262623 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece40ca0d0 con 0x7fece406b060 2023-11-28T11:09:25.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.819+0000 7fece8ac3700 1 --2- 172.21.15.156:0/53262623 >> v2:172.21.15.186:3300/0 conn(0x7fece4160b10 0x7fece40cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:25.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.820+0000 7fece9ac5700 1 --2- 172.21.15.156:0/53262623 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fece406b060 0x7fece406b4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009e50 tx=0x7fecdc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6358446962872f7f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.820+0000 7fecdbfff700 1 -- 172.21.15.156:0/53262623 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc00e070 con 0x7fece406b060 2023-11-28T11:09:25.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.820+0000 7fecdbfff700 1 -- 172.21.15.156:0/53262623 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc00b910 con 0x7fece406b060 2023-11-28T11:09:25.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.820+0000 7fecdbfff700 1 -- 172.21.15.156:0/53262623 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc00bcc0 con 0x7fece406b060 2023-11-28T11:09:25.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.820+0000 7feceb528700 1 -- 172.21.15.156:0/53262623 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fece406b060 msgr2=0x7fece406b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.820+0000 7feceb528700 1 --2- 172.21.15.156:0/53262623 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fece406b060 0x7fece406b4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009e50 tx=0x7fecdc00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:25.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.820+0000 7feceb528700 1 -- 172.21.15.156:0/53262623 shutdown_connections 2023-11-28T11:09:25.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.820+0000 7feceb528700 1 --2- 172.21.15.156:0/53262623 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fece406b060 0x7fece406b4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.820+0000 7feceb528700 1 --2- 172.21.15.156:0/53262623 >> v2:172.21.15.186:3300/0 conn(0x7fece4160b10 0x7fece40cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.820+0000 7feceb528700 1 -- 172.21.15.156:0/53262623 >> 172.21.15.156:0/53262623 conn(0x7fece40b12b0 msgr2=0x7fece40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.820+0000 7feceb528700 1 -- 172.21.15.156:0/53262623 shutdown_connections 2023-11-28T11:09:25.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.820+0000 7feceb528700 1 -- 172.21.15.156:0/53262623 wait complete. 2023-11-28T11:09:25.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.821+0000 7feceb528700 1 Processor -- start 2023-11-28T11:09:25.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.822+0000 7feceb528700 1 -- start start 2023-11-28T11:09:25.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.822+0000 7feceb528700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece406b060 0x7fece41f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.822+0000 7feceb528700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fece4160740 0x7fece4159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.822+0000 7feceb528700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece4160b10 0x7fece4159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.822+0000 7feceb528700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece40c8900 con 0x7fece406b060 2023-11-28T11:09:25.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.822+0000 7feceb528700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece40c8470 con 0x7fece4160b10 2023-11-28T11:09:25.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.822+0000 7feceb528700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece41f3550 con 0x7fece4160740 2023-11-28T11:09:25.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.822+0000 7fece92c4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece406b060 0x7fece41f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.822+0000 7fece92c4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece406b060 0x7fece41f3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33968/0 (socket says 172.21.15.156:33968) 2023-11-28T11:09:25.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.822+0000 7fece92c4700 1 -- 172.21.15.156:0/1363310629 learned_addr learned my addr 172.21.15.156:0/1363310629 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:25.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7fece8ac3700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fece4160740 0x7fece4159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7fece9ac5700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece4160b10 0x7fece4159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7fece92c4700 1 -- 172.21.15.156:0/1363310629 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fece4160740 msgr2=0x7fece4159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7fece92c4700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fece4160740 0x7fece4159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7fece92c4700 1 -- 172.21.15.156:0/1363310629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece4160b10 msgr2=0x7fece4159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7fece92c4700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece4160b10 0x7fece4159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7fece92c4700 1 -- 172.21.15.156:0/1363310629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40094a0 con 0x7fece406b060 2023-11-28T11:09:25.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7fece92c4700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece406b060 0x7fece41f3010 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fece000c4e0 tx=0x7fece000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7fecd9ffb700 1 -- 172.21.15.156:0/1363310629 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece00052d0 con 0x7fece406b060 2023-11-28T11:09:25.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7fecd9ffb700 1 -- 172.21.15.156:0/1363310629 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece0005430 con 0x7fece406b060 2023-11-28T11:09:25.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7feceb528700 1 -- 172.21.15.156:0/1363310629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc009b30 con 0x7fece406b060 2023-11-28T11:09:25.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7fecd9ffb700 1 -- 172.21.15.156:0/1363310629 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece00038c0 con 0x7fece406b060 2023-11-28T11:09:25.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.823+0000 7feceb528700 1 -- 172.21.15.156:0/1363310629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece40cd8d0 con 0x7fece406b060 2023-11-28T11:09:25.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.824+0000 7feceb528700 1 -- 172.21.15.156:0/1363310629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece40ca3b0 con 0x7fece406b060 2023-11-28T11:09:25.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.824+0000 7fecd9ffb700 1 -- 172.21.15.156:0/1363310629 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fece0019030 con 0x7fece406b060 2023-11-28T11:09:25.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.825+0000 7fecd9ffb700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fecc8068320 0x7fecc806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.825+0000 7fecd9ffb700 1 -- 172.21.15.156:0/1363310629 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fece0084ff0 con 0x7fece406b060 2023-11-28T11:09:25.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.825+0000 7fece8ac3700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fecc8068320 0x7fecc806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.825+0000 7fece8ac3700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fecc8068320 0x7fecc806a7d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fece406e950 tx=0x7fecd4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:25.829+0000 7fecd9ffb700 1 -- 172.21.15.156:0/1363310629 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fece0051850 con 0x7fece406b060 2023-11-28T11:09:26.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.012+0000 7feceb528700 1 -- 172.21.15.156:0/1363310629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fece4160f60 con 0x7fece406b060 2023-11-28T11:09:26.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.012+0000 7fecd9ffb700 1 -- 172.21.15.156:0/1363310629 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7fece0054e70 con 0x7fece406b060 2023-11-28T11:09:26.014 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:09:26.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.014+0000 7feceb528700 1 -- 172.21.15.156:0/1363310629 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fecc8068320 msgr2=0x7fecc806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.014+0000 7feceb528700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fecc8068320 0x7fecc806a7d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fece406e950 tx=0x7fecd4008040 comp rx=0 tx=0).stop 2023-11-28T11:09:26.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.014+0000 7feceb528700 1 -- 172.21.15.156:0/1363310629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece406b060 msgr2=0x7fece41f3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.015+0000 7feceb528700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece406b060 0x7fece41f3010 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fece000c4e0 tx=0x7fece000c8a0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.015+0000 7feceb528700 1 -- 172.21.15.156:0/1363310629 shutdown_connections 2023-11-28T11:09:26.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.015+0000 7feceb528700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece4160b10 0x7fece4159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.015+0000 7feceb528700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece406b060 0x7fece41f3010 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.015+0000 7feceb528700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fecc8068320 0x7fecc806a7d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.015+0000 7feceb528700 1 --2- 172.21.15.156:0/1363310629 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fece4160740 0x7fece4159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.015+0000 7feceb528700 1 -- 172.21.15.156:0/1363310629 >> 172.21.15.156:0/1363310629 conn(0x7fece40b12b0 msgr2=0x7fece40b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.015+0000 7feceb528700 1 -- 172.21.15.156:0/1363310629 shutdown_connections 2023-11-28T11:09:26.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.015+0000 7feceb528700 1 -- 172.21.15.156:0/1363310629 wait complete. 2023-11-28T11:09:26.028 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:26.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.150+0000 7f43ca3e7700 1 Processor -- start 2023-11-28T11:09:26.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.150+0000 7f43ca3e7700 1 -- start start 2023-11-28T11:09:26.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.151+0000 7f43ca3e7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f43c4071040 0x7f43c4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.151+0000 7f43ca3e7700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c4071950 con 0x7f43c406dee0 2023-11-28T11:09:26.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.151+0000 7f43ca3e7700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c415d060 con 0x7f43c4160540 2023-11-28T11:09:26.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.151+0000 7f43ca3e7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c415d340 con 0x7f43c4071040 2023-11-28T11:09:26.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.151+0000 7f43c3fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f43c4071040 0x7f43c4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.151+0000 7f43c3fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f43c4071040 0x7f43c4071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33976/0 (socket says 172.21.15.156:33976) 2023-11-28T11:09:26.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.151+0000 7f43c37fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f43c406dee0 0x7f43c406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48856/0 (socket says 172.21.15.156:48856) 2023-11-28T11:09:26.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.151+0000 7f43c3fff700 1 -- 172.21.15.156:0/3143436127 learned_addr learned my addr 172.21.15.156:0/3143436127 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:26.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.151+0000 7f43c3fff700 1 -- 172.21.15.156:0/3143436127 >> v1:172.21.15.186:6789/0 conn(0x7f43c4160540 legacy=0x7f43c4160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.151+0000 7f43c3fff700 1 -- 172.21.15.156:0/3143436127 >> v1:172.21.15.156:6789/0 conn(0x7f43c406dee0 legacy=0x7f43c406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.151+0000 7f43c3fff700 1 -- 172.21.15.156:0/3143436127 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c415d620 con 0x7f43c4071040 2023-11-28T11:09:26.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.152+0000 7f43c3fff700 1 --2- 172.21.15.156:0/3143436127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c4071040 0x7f43c4071410 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f43b80098f0 tx=0x7f43b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4355e1ea6cecc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.152+0000 7f43c2ffd700 1 -- 172.21.15.156:0/3143436127 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b800b8a0 con 0x7f43c4071040 2023-11-28T11:09:26.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.152+0000 7f43c2ffd700 1 -- 172.21.15.156:0/3143436127 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b800ba00 con 0x7f43c4071040 2023-11-28T11:09:26.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.152+0000 7f43c2ffd700 1 -- 172.21.15.156:0/3143436127 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b800bd90 con 0x7f43c4071040 2023-11-28T11:09:26.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.152+0000 7f43ca3e7700 1 -- 172.21.15.156:0/3143436127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c4071040 msgr2=0x7f43c4071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.152+0000 7f43ca3e7700 1 --2- 172.21.15.156:0/3143436127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c4071040 0x7f43c4071410 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f43b80098f0 tx=0x7f43b8009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:26.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.152+0000 7f43ca3e7700 1 -- 172.21.15.156:0/3143436127 shutdown_connections 2023-11-28T11:09:26.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.152+0000 7f43ca3e7700 1 --2- 172.21.15.156:0/3143436127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c4071040 0x7f43c4071410 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.152+0000 7f43ca3e7700 1 -- 172.21.15.156:0/3143436127 >> 172.21.15.156:0/3143436127 conn(0x7f43c40d0000 msgr2=0x7f43c40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.152+0000 7f43ca3e7700 1 -- 172.21.15.156:0/3143436127 shutdown_connections 2023-11-28T11:09:26.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.152+0000 7f43ca3e7700 1 -- 172.21.15.156:0/3143436127 wait complete. 2023-11-28T11:09:26.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.154+0000 7f43ca3e7700 1 Processor -- start 2023-11-28T11:09:26.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.154+0000 7f43ca3e7700 1 -- start start 2023-11-28T11:09:26.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.154+0000 7f43ca3e7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c406dee0 0x7f43c40c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.154+0000 7f43ca3e7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c4071040 0x7f43c4153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.154+0000 7f43ca3e7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c4160540 0x7f43c415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.154+0000 7f43ca3e7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c4153a80 con 0x7f43c406dee0 2023-11-28T11:09:26.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.154+0000 7f43ca3e7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40b2e30 con 0x7f43c4160540 2023-11-28T11:09:26.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.154+0000 7f43ca3e7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c4159370 con 0x7f43c4071040 2023-11-28T11:09:26.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c406dee0 0x7f43c40c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c406dee0 0x7f43c40c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33982/0 (socket says 172.21.15.156:33982) 2023-11-28T11:09:26.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c3fff700 1 -- 172.21.15.156:0/1802002760 learned_addr learned my addr 172.21.15.156:0/1802002760 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:26.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c37fe700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c4071040 0x7f43c4153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c8984700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c4160540 0x7f43c415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c3fff700 1 -- 172.21.15.156:0/1802002760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c4071040 msgr2=0x7f43c4153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c3fff700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c4071040 0x7f43c4153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c3fff700 1 -- 172.21.15.156:0/1802002760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c4160540 msgr2=0x7f43c415ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c3fff700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c4160540 0x7f43c415ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c3fff700 1 -- 172.21.15.156:0/1802002760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8009550 con 0x7f43c406dee0 2023-11-28T11:09:26.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c37fe700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c4071040 0x7f43c4153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:26.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c8984700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c4160540 0x7f43c415ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:26.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c3fff700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c406dee0 0x7f43c40c3d80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f43b8014040 tx=0x7f43b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c0ff9700 1 -- 172.21.15.156:0/1802002760 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b80038a0 con 0x7f43c406dee0 2023-11-28T11:09:26.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c0ff9700 1 -- 172.21.15.156:0/1802002760 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b801e070 con 0x7f43c406dee0 2023-11-28T11:09:26.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43ca3e7700 1 -- 172.21.15.156:0/1802002760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c415ccd0 con 0x7f43c406dee0 2023-11-28T11:09:26.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43c0ff9700 1 -- 172.21.15.156:0/1802002760 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8003c60 con 0x7f43c406dee0 2023-11-28T11:09:26.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.155+0000 7f43ca3e7700 1 -- 172.21.15.156:0/1802002760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c415d900 con 0x7f43c406dee0 2023-11-28T11:09:26.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.156+0000 7f43ca3e7700 1 -- 172.21.15.156:0/1802002760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c406a270 con 0x7f43c406dee0 2023-11-28T11:09:26.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.157+0000 7f43c0ff9700 1 -- 172.21.15.156:0/1802002760 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f43b800f050 con 0x7f43c406dee0 2023-11-28T11:09:26.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.157+0000 7f43c0ff9700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f43b00682f0 0x7f43b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.157+0000 7f43c0ff9700 1 -- 172.21.15.156:0/1802002760 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f43b8020070 con 0x7f43c406dee0 2023-11-28T11:09:26.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.157+0000 7f43c37fe700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f43b00682f0 0x7f43b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.157+0000 7f43c37fe700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f43b00682f0 0x7f43b006a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f43c41f6270 tx=0x7f43b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.161+0000 7f43c0ff9700 1 -- 172.21.15.156:0/1802002760 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f43b8053a90 con 0x7f43c406dee0 2023-11-28T11:09:26.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.343+0000 7f43ca3e7700 1 -- 172.21.15.156:0/1802002760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43c415bb90 con 0x7f43c406dee0 2023-11-28T11:09:26.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.343+0000 7f43c0ff9700 1 -- 172.21.15.156:0/1802002760 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43b802d350 con 0x7f43c406dee0 2023-11-28T11:09:26.345 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:09:26.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.345+0000 7f43ca3e7700 1 -- 172.21.15.156:0/1802002760 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f43b00682f0 msgr2=0x7f43b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.345+0000 7f43ca3e7700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f43b00682f0 0x7f43b006a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f43c41f6270 tx=0x7f43b4009040 comp rx=0 tx=0).stop 2023-11-28T11:09:26.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.345+0000 7f43ca3e7700 1 -- 172.21.15.156:0/1802002760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c406dee0 msgr2=0x7f43c40c3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.345+0000 7f43ca3e7700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c406dee0 0x7f43c40c3d80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f43b8014040 tx=0x7f43b8004750 comp rx=0 tx=0).stop 2023-11-28T11:09:26.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.345+0000 7f43ca3e7700 1 -- 172.21.15.156:0/1802002760 shutdown_connections 2023-11-28T11:09:26.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.345+0000 7f43ca3e7700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c4160540 0x7f43c415ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.345+0000 7f43ca3e7700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c406dee0 0x7f43c40c3d80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.345+0000 7f43ca3e7700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f43b00682f0 0x7f43b006a7a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.345+0000 7f43ca3e7700 1 --2- 172.21.15.156:0/1802002760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c4071040 0x7f43c4153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.345+0000 7f43ca3e7700 1 -- 172.21.15.156:0/1802002760 >> 172.21.15.156:0/1802002760 conn(0x7f43c40d0000 msgr2=0x7f43c415f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.346+0000 7f43ca3e7700 1 -- 172.21.15.156:0/1802002760 shutdown_connections 2023-11-28T11:09:26.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.346+0000 7f43ca3e7700 1 -- 172.21.15.156:0/1802002760 wait complete. 2023-11-28T11:09:26.359 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.479+0000 7f5073a93700 1 Processor -- start 2023-11-28T11:09:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.479+0000 7f5073a93700 1 -- start start 2023-11-28T11:09:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.480+0000 7f5073a93700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f506c154e70 0x7f506c0cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.480+0000 7f5073a93700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f506c0cf360 0x7f506c071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.480+0000 7f5073a93700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f506c071580 0x7f506c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.480+0000 7f5073a93700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506c071980 con 0x7f506c0cf360 2023-11-28T11:09:26.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.480+0000 7f5073a93700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506c155240 con 0x7f506c154e70 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.480+0000 7f5073a93700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c15b100 con 0x7f506c071580 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.480+0000 7f507102e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f506c0cf360 0x7f506c071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507102e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f506c0cf360 0x7f506c071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33990/0 (socket says 172.21.15.156:33990) 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507102e700 1 -- 172.21.15.156:0/3281174232 learned_addr learned my addr 172.21.15.156:0/3281174232 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f5072030700 1 --2- 172.21.15.156:0/3281174232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f506c071580 0x7f506c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507182f700 1 --2- 172.21.15.156:0/3281174232 >> v2:172.21.15.186:3300/0 conn(0x7f506c154e70 0x7f506c0cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507102e700 1 -- 172.21.15.156:0/3281174232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f506c071580 msgr2=0x7f506c15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507102e700 1 --2- 172.21.15.156:0/3281174232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f506c071580 0x7f506c15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507102e700 1 -- 172.21.15.156:0/3281174232 >> v2:172.21.15.186:3300/0 conn(0x7f506c154e70 msgr2=0x7f506c0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507102e700 1 --2- 172.21.15.156:0/3281174232 >> v2:172.21.15.186:3300/0 conn(0x7f506c154e70 0x7f506c0cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507102e700 1 -- 172.21.15.156:0/3281174232 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c15b3e0 con 0x7f506c0cf360 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507182f700 1 --2- 172.21.15.156:0/3281174232 >> v2:172.21.15.186:3300/0 conn(0x7f506c154e70 0x7f506c0cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:26.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507102e700 1 --2- 172.21.15.156:0/3281174232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506c0cf360 0x7f506c071040 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f506c073b80 tx=0x7f5068009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a79362a2dc3d1c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507082d700 1 -- 172.21.15.156:0/3281174232 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068011830 con 0x7f506c0cf360 2023-11-28T11:09:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507082d700 1 -- 172.21.15.156:0/3281174232 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5068011990 con 0x7f506c0cf360 2023-11-28T11:09:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.481+0000 7f507082d700 1 -- 172.21.15.156:0/3281174232 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068011d20 con 0x7f506c0cf360 2023-11-28T11:09:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.482+0000 7f5073a93700 1 -- 172.21.15.156:0/3281174232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506c0cf360 msgr2=0x7f506c071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.482+0000 7f5073a93700 1 --2- 172.21.15.156:0/3281174232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506c0cf360 0x7f506c071040 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f506c073b80 tx=0x7f5068009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.482+0000 7f5073a93700 1 -- 172.21.15.156:0/3281174232 shutdown_connections 2023-11-28T11:09:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.482+0000 7f5073a93700 1 --2- 172.21.15.156:0/3281174232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f506c071580 0x7f506c15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.482+0000 7f5073a93700 1 --2- 172.21.15.156:0/3281174232 >> v2:172.21.15.186:3300/0 conn(0x7f506c154e70 0x7f506c0cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.482+0000 7f5073a93700 1 --2- 172.21.15.156:0/3281174232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506c0cf360 0x7f506c071040 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.482+0000 7f5073a93700 1 -- 172.21.15.156:0/3281174232 >> 172.21.15.156:0/3281174232 conn(0x7f506c0d0000 msgr2=0x7f506c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.482+0000 7f5073a93700 1 -- 172.21.15.156:0/3281174232 shutdown_connections 2023-11-28T11:09:26.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.482+0000 7f5073a93700 1 -- 172.21.15.156:0/3281174232 wait complete. 2023-11-28T11:09:26.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.483+0000 7f5073a93700 1 Processor -- start 2023-11-28T11:09:26.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.483+0000 7f5073a93700 1 -- start start 2023-11-28T11:09:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f5073a93700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506c071580 0x7f506c06cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f5073a93700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f506c0cf360 0x7f506c14c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f5073a93700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f506c154e70 0x7f506c14cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f5073a93700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c14d770 con 0x7f506c071580 2023-11-28T11:09:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f5073a93700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c14da50 con 0x7f506c0cf360 2023-11-28T11:09:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f5073a93700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c14dd30 con 0x7f506c154e70 2023-11-28T11:09:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f507182f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506c071580 0x7f506c06cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f507182f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506c071580 0x7f506c06cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33994/0 (socket says 172.21.15.156:33994) 2023-11-28T11:09:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f507182f700 1 -- 172.21.15.156:0/2274458882 learned_addr learned my addr 172.21.15.156:0/2274458882 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f507102e700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f506c0cf360 0x7f506c14c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f5072030700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f506c154e70 0x7f506c14cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f507182f700 1 -- 172.21.15.156:0/2274458882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f506c154e70 msgr2=0x7f506c14cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f507182f700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f506c154e70 0x7f506c14cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f507182f700 1 -- 172.21.15.156:0/2274458882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f506c0cf360 msgr2=0x7f506c14c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f507182f700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f506c0cf360 0x7f506c14c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f507182f700 1 -- 172.21.15.156:0/2274458882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5068009550 con 0x7f506c071580 2023-11-28T11:09:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f507182f700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506c071580 0x7f506c06cf50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f505c00e810 tx=0x7f505c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f50627fc700 1 -- 172.21.15.156:0/2274458882 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c00c020 con 0x7f506c071580 2023-11-28T11:09:26.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f50627fc700 1 -- 172.21.15.156:0/2274458882 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c004250 con 0x7f506c071580 2023-11-28T11:09:26.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f5073a93700 1 -- 172.21.15.156:0/2274458882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506c15b9a0 con 0x7f506c071580 2023-11-28T11:09:26.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f50627fc700 1 -- 172.21.15.156:0/2274458882 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c0036a0 con 0x7f506c071580 2023-11-28T11:09:26.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.484+0000 7f5073a93700 1 -- 172.21.15.156:0/2274458882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506c15b6c0 con 0x7f506c071580 2023-11-28T11:09:26.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.485+0000 7f5073a93700 1 -- 172.21.15.156:0/2274458882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506c0af520 con 0x7f506c071580 2023-11-28T11:09:26.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.486+0000 7f50627fc700 1 -- 172.21.15.156:0/2274458882 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f505c003800 con 0x7f506c071580 2023-11-28T11:09:26.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.486+0000 7f50627fc700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f50500682f0 0x7f505006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.486+0000 7f50627fc700 1 -- 172.21.15.156:0/2274458882 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f505c0858d0 con 0x7f506c071580 2023-11-28T11:09:26.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.486+0000 7f507102e700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f50500682f0 0x7f505006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.486+0000 7f507102e700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f50500682f0 0x7f505006a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5068009c10 tx=0x7f5068008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.490+0000 7f50627fc700 1 -- 172.21.15.156:0/2274458882 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f505c052130 con 0x7f506c071580 2023-11-28T11:09:26.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.688+0000 7f5073a93700 1 -- 172.21.15.156:0/2274458882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f506c0cca80 con 0x7f506c071580 2023-11-28T11:09:26.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.688+0000 7f50627fc700 1 -- 172.21.15.156:0/2274458882 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f505c055750 con 0x7f506c071580 2023-11-28T11:09:26.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.690+0000 7f5073a93700 1 -- 172.21.15.156:0/2274458882 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f50500682f0 msgr2=0x7f505006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.690+0000 7f5073a93700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f50500682f0 0x7f505006a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5068009c10 tx=0x7f5068008030 comp rx=0 tx=0).stop 2023-11-28T11:09:26.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.690+0000 7f5073a93700 1 -- 172.21.15.156:0/2274458882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506c071580 msgr2=0x7f506c06cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.690+0000 7f5073a93700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506c071580 0x7f506c06cf50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f505c00e810 tx=0x7f505c00eb20 comp rx=0 tx=0).stop 2023-11-28T11:09:26.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.690+0000 7f5073a93700 1 -- 172.21.15.156:0/2274458882 shutdown_connections 2023-11-28T11:09:26.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.690+0000 7f5073a93700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f50500682f0 0x7f505006a7a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.690+0000 7f5073a93700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f506c154e70 0x7f506c14cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.690+0000 7f5073a93700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506c071580 0x7f506c06cf50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.690+0000 7f5073a93700 1 --2- 172.21.15.156:0/2274458882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f506c0cf360 0x7f506c14c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.690+0000 7f5073a93700 1 -- 172.21.15.156:0/2274458882 >> 172.21.15.156:0/2274458882 conn(0x7f506c0d0000 msgr2=0x7f506c0acdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.690+0000 7f5073a93700 1 -- 172.21.15.156:0/2274458882 shutdown_connections 2023-11-28T11:09:26.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.690+0000 7f5073a93700 1 -- 172.21.15.156:0/2274458882 wait complete. 2023-11-28T11:09:26.693 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T11:09:26.704 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:09:26.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:26.704 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:26.769 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:09:26.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:26.770 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:09:26.795 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:26.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.955+0000 7f902affc700 1 Processor -- start 2023-11-28T11:09:26.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.955+0000 7f902affc700 1 -- start start 2023-11-28T11:09:26.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f902affc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9024071040 0x7f9024071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f902affc700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9024160540 0x7f90241609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f902affc700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9024071950 con 0x7f902406dee0 2023-11-28T11:09:26.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f902affc700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f902415d060 con 0x7f9024160540 2023-11-28T11:09:26.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f902affc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902415d340 con 0x7f9024071040 2023-11-28T11:09:26.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f9023fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f902406dee0 0x7f902406e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:48880/0 (socket says 172.21.15.156:48880) 2023-11-28T11:09:26.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f9023fff700 1 -- 172.21.15.156:0/961786352 learned_addr learned my addr 172.21.15.156:0/961786352 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:26.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f9028d98700 1 --2- 172.21.15.156:0/961786352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9024071040 0x7f9024071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f9029599700 1 --2- 172.21.15.156:0/961786352 >> v2:172.21.15.186:3300/0 conn(0x7f9024160540 0x7f90241609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f90237fe700 1 -- 172.21.15.156:0/961786352 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995725982 0 0) 0x7f9024071950 con 0x7f902406dee0 2023-11-28T11:09:26.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f90237fe700 1 -- 172.21.15.156:0/961786352 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90140035b0 con 0x7f902406dee0 2023-11-28T11:09:26.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f9029599700 1 -- 172.21.15.156:0/961786352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9024071040 msgr2=0x7f9024071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:26.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f9029599700 1 --2- 172.21.15.156:0/961786352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9024071040 0x7f9024071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f9029599700 1 -- 172.21.15.156:0/961786352 >> v1:172.21.15.156:6789/0 conn(0x7f902406dee0 legacy=0x7f902406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f9029599700 1 -- 172.21.15.156:0/961786352 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902415d620 con 0x7f9024160540 2023-11-28T11:09:26.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.956+0000 7f9028d98700 1 --2- 172.21.15.156:0/961786352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9024071040 0x7f9024071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:26.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.957+0000 7f9029599700 1 --2- 172.21.15.156:0/961786352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9024160540 0x7f90241609c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f901c009d90 tx=0x7f901c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb99b4dbc1fdda8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.957+0000 7f90237fe700 1 -- 172.21.15.156:0/961786352 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c00e070 con 0x7f9024160540 2023-11-28T11:09:26.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.957+0000 7f90237fe700 1 -- 172.21.15.156:0/961786352 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901c00bb20 con 0x7f9024160540 2023-11-28T11:09:26.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.957+0000 7f90237fe700 1 -- 172.21.15.156:0/961786352 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c012430 con 0x7f9024160540 2023-11-28T11:09:26.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.957+0000 7f902affc700 1 -- 172.21.15.156:0/961786352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9024160540 msgr2=0x7f90241609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.957+0000 7f902affc700 1 --2- 172.21.15.156:0/961786352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9024160540 0x7f90241609c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f901c009d90 tx=0x7f901c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.957+0000 7f902affc700 1 -- 172.21.15.156:0/961786352 shutdown_connections 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.957+0000 7f902affc700 1 --2- 172.21.15.156:0/961786352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9024160540 0x7f90241609c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.957+0000 7f902affc700 1 --2- 172.21.15.156:0/961786352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9024071040 0x7f9024071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.957+0000 7f902affc700 1 -- 172.21.15.156:0/961786352 >> 172.21.15.156:0/961786352 conn(0x7f90240d0000 msgr2=0x7f90240d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.957+0000 7f902affc700 1 -- 172.21.15.156:0/961786352 shutdown_connections 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.957+0000 7f902affc700 1 -- 172.21.15.156:0/961786352 wait complete. 2023-11-28T11:09:26.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.959+0000 7f902affc700 1 Processor -- start 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.959+0000 7f902affc700 1 -- start start 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.959+0000 7f902affc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f902406dee0 0x7f90240b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.959+0000 7f902affc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9024071040 0x7f9024153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.959+0000 7f902affc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9024160540 0x7f90240b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.959+0000 7f902affc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90241f5af0 con 0x7f9024160540 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.959+0000 7f902affc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9024153650 con 0x7f902406dee0 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.959+0000 7f902affc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90241f5d40 con 0x7f9024071040 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f9029599700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9024160540 0x7f90240b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f9029599700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9024160540 0x7f90240b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34012/0 (socket says 172.21.15.156:34012) 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f9029599700 1 -- 172.21.15.156:0/1157847447 learned_addr learned my addr 172.21.15.156:0/1157847447 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f9028d98700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f902406dee0 0x7f90240b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f9023fff700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9024071040 0x7f9024153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f9029599700 1 -- 172.21.15.156:0/1157847447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9024071040 msgr2=0x7f9024153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f9029599700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9024071040 0x7f9024153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f9029599700 1 -- 172.21.15.156:0/1157847447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f902406dee0 msgr2=0x7f90240b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f9029599700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f902406dee0 0x7f90240b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f9029599700 1 -- 172.21.15.156:0/1157847447 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90180094a0 con 0x7f9024160540 2023-11-28T11:09:26.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f9028d98700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f902406dee0 0x7f90240b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:26.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f9029599700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9024160540 0x7f90240b0a50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f901c014040 tx=0x7f901c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f90217fa700 1 -- 172.21.15.156:0/1157847447 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c012a90 con 0x7f9024160540 2023-11-28T11:09:26.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f90217fa700 1 -- 172.21.15.156:0/1157847447 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901c020070 con 0x7f9024160540 2023-11-28T11:09:26.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f902affc700 1 -- 172.21.15.156:0/1157847447 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c009a70 con 0x7f9024160540 2023-11-28T11:09:26.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f90217fa700 1 -- 172.21.15.156:0/1157847447 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c012e50 con 0x7f9024160540 2023-11-28T11:09:26.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.960+0000 7f902affc700 1 -- 172.21.15.156:0/1157847447 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902415ccd0 con 0x7f9024160540 2023-11-28T11:09:26.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.961+0000 7f902affc700 1 -- 172.21.15.156:0/1157847447 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902415d900 con 0x7f9024160540 2023-11-28T11:09:26.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.961+0000 7f90217fa700 1 -- 172.21.15.156:0/1157847447 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f901c004040 con 0x7f9024160540 2023-11-28T11:09:26.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.962+0000 7f90217fa700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90080682f0 0x7f900806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.962+0000 7f9028d98700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90080682f0 0x7f900806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.962+0000 7f90217fa700 1 -- 172.21.15.156:0/1157847447 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f901c022070 con 0x7f9024160540 2023-11-28T11:09:26.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.962+0000 7f9028d98700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90080682f0 0x7f900806a7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f90240b15f0 tx=0x7f9018008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:26.966+0000 7f90217fa700 1 -- 172.21.15.156:0/1157847447 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f901c0546d0 con 0x7f9024160540 2023-11-28T11:09:27.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.194+0000 7f902affc700 1 -- 172.21.15.156:0/1157847447 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f902406a270 con 0x7f9024160540 2023-11-28T11:09:27.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.195+0000 7f90217fa700 1 -- 172.21.15.156:0/1157847447 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f901c057cf0 con 0x7f9024160540 2023-11-28T11:09:27.196 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:27.196 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1085440106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1085440106},{"type":"v1","addr":"172.21.15.186:6834","nonce":1085440106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1548245207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1548245207},{"type":"v1","addr":"172.21.15.156:6836","nonce":1548245207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4882,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3102077957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":3102077957},{"type":"v1","addr":"172.21.15.156:6837","nonce":3102077957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:15.391255+0000","modified":"2023-11-28T11:09:18.414744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3897746301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3897746301},{"type":"v1","addr":"172.21.15.186:6835","nonce":3897746301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:09:27.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.197+0000 7f902affc700 1 -- 172.21.15.156:0/1157847447 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90080682f0 msgr2=0x7f900806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.197+0000 7f902affc700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90080682f0 0x7f900806a7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f90240b15f0 tx=0x7f9018008040 comp rx=0 tx=0).stop 2023-11-28T11:09:27.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.197+0000 7f902affc700 1 -- 172.21.15.156:0/1157847447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9024160540 msgr2=0x7f90240b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.197+0000 7f902affc700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9024160540 0x7f90240b0a50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f901c014040 tx=0x7f901c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.197+0000 7f902affc700 1 -- 172.21.15.156:0/1157847447 shutdown_connections 2023-11-28T11:09:27.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.197+0000 7f902affc700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9024160540 0x7f90240b0a50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.197+0000 7f902affc700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f902406dee0 0x7f90240b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.197+0000 7f902affc700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90080682f0 0x7f900806a7a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.197+0000 7f902affc700 1 --2- 172.21.15.156:0/1157847447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9024071040 0x7f9024153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.197+0000 7f902affc700 1 -- 172.21.15.156:0/1157847447 >> 172.21.15.156:0/1157847447 conn(0x7f90240d0000 msgr2=0x7f90240cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:27.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.197+0000 7f902affc700 1 -- 172.21.15.156:0/1157847447 shutdown_connections 2023-11-28T11:09:27.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.197+0000 7f902affc700 1 -- 172.21.15.156:0/1157847447 wait complete. 2023-11-28T11:09:27.200 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T11:09:27.210 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:09:27.374 INFO:teuthology.orchestra.run.smithi186.stdout:{"message":"","return_code":0} 2023-11-28T11:09:27.375 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:09:27.375 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.494+0000 7f204bd93700 1 Processor -- start 2023-11-28T11:09:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.495+0000 7f204bd93700 1 -- start start 2023-11-28T11:09:27.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.495+0000 7f204bd93700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f204406ec90 0x7f2044065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.495+0000 7f204bd93700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20440cec70 0x7f20440ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.495+0000 7f204bd93700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204406e240 con 0x7f20440c9f10 2023-11-28T11:09:27.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.495+0000 7f204bd93700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2044066580 con 0x7f204406ec90 2023-11-28T11:09:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.495+0000 7f204bd93700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20440cded0 con 0x7f20440cec70 2023-11-28T11:09:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f2049b2f700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f204406ec90 0x7f2044065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f204932e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20440cec70 0x7f20440ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f204a330700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f20440c9f10 0x7f204415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:59258/0 (socket says 172.21.15.156:59258) 2023-11-28T11:09:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f204a330700 1 -- 172.21.15.156:0/1250787029 learned_addr learned my addr 172.21.15.156:0/1250787029 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f2048b2d700 1 -- 172.21.15.156:0/1250787029 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851324817 0 0) 0x7f203c001490 con 0x7f20440c9f10 2023-11-28T11:09:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f2048b2d700 1 -- 172.21.15.156:0/1250787029 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20300035b0 con 0x7f20440c9f10 2023-11-28T11:09:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f2049b2f700 1 -- 172.21.15.156:0/1250787029 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20440cec70 msgr2=0x7f20440ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f2049b2f700 1 --2- 172.21.15.156:0/1250787029 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20440cec70 0x7f20440ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f2049b2f700 1 -- 172.21.15.156:0/1250787029 >> v1:172.21.15.186:6789/0 conn(0x7f20440c9f10 legacy=0x7f204415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f2049b2f700 1 -- 172.21.15.156:0/1250787029 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20440ce1b0 con 0x7f204406ec90 2023-11-28T11:09:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f204932e700 1 --2- 172.21.15.156:0/1250787029 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20440cec70 0x7f20440ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f2049b2f700 1 --2- 172.21.15.156:0/1250787029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f204406ec90 0x7f2044065ef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2034009840 tx=0x7f2034009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2c3ac6ecdd0703d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f2048b2d700 1 -- 172.21.15.156:0/1250787029 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034011830 con 0x7f204406ec90 2023-11-28T11:09:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f2048b2d700 1 -- 172.21.15.156:0/1250787029 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2034011990 con 0x7f204406ec90 2023-11-28T11:09:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.496+0000 7f2048b2d700 1 -- 172.21.15.156:0/1250787029 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034011d20 con 0x7f204406ec90 2023-11-28T11:09:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.497+0000 7f204bd93700 1 -- 172.21.15.156:0/1250787029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f204406ec90 msgr2=0x7f2044065ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.497+0000 7f204bd93700 1 --2- 172.21.15.156:0/1250787029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f204406ec90 0x7f2044065ef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2034009840 tx=0x7f2034009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.497+0000 7f204bd93700 1 -- 172.21.15.156:0/1250787029 shutdown_connections 2023-11-28T11:09:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.497+0000 7f204bd93700 1 --2- 172.21.15.156:0/1250787029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f204406ec90 0x7f2044065ef0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.497+0000 7f204bd93700 1 --2- 172.21.15.156:0/1250787029 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20440cec70 0x7f20440ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.497+0000 7f204bd93700 1 -- 172.21.15.156:0/1250787029 >> 172.21.15.156:0/1250787029 conn(0x7f20440b12b0 msgr2=0x7f20440b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.497+0000 7f204bd93700 1 -- 172.21.15.156:0/1250787029 shutdown_connections 2023-11-28T11:09:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.497+0000 7f204bd93700 1 -- 172.21.15.156:0/1250787029 wait complete. 2023-11-28T11:09:27.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.498+0000 7f204bd93700 1 Processor -- start 2023-11-28T11:09:27.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.498+0000 7f204bd93700 1 -- start start 2023-11-28T11:09:27.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f204bd93700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f204406ec90 0x7f20440c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f204bd93700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20440c9f10 0x7f2044151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f204bd93700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20440cec70 0x7f20440c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f204bd93700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20440b37b0 con 0x7f204406ec90 2023-11-28T11:09:27.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f204bd93700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2044152130 con 0x7f20440c9f10 2023-11-28T11:09:27.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f204bd93700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20440b2b30 con 0x7f20440cec70 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f2049b2f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f204406ec90 0x7f20440c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f2049b2f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f204406ec90 0x7f20440c3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34022/0 (socket says 172.21.15.156:34022) 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f2049b2f700 1 -- 172.21.15.156:0/1146117882 learned_addr learned my addr 172.21.15.156:0/1146117882 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f204932e700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20440c9f10 0x7f2044151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f204a330700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20440cec70 0x7f20440c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f2049b2f700 1 -- 172.21.15.156:0/1146117882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20440cec70 msgr2=0x7f20440c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f2049b2f700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20440cec70 0x7f20440c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f2049b2f700 1 -- 172.21.15.156:0/1146117882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20440c9f10 msgr2=0x7f2044151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f2049b2f700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20440c9f10 0x7f2044151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f2049b2f700 1 -- 172.21.15.156:0/1146117882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20400094a0 con 0x7f204406ec90 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.499+0000 7f204932e700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20440c9f10 0x7f2044151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.500+0000 7f2049b2f700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f204406ec90 0x7f20440c3040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2034009fd0 tx=0x7f2034004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.500+0000 7f203a7fc700 1 -- 172.21.15.156:0/1146117882 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034004590 con 0x7f204406ec90 2023-11-28T11:09:27.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.500+0000 7f203a7fc700 1 -- 172.21.15.156:0/1146117882 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203401e070 con 0x7f204406ec90 2023-11-28T11:09:27.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.500+0000 7f203a7fc700 1 -- 172.21.15.156:0/1146117882 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20340038e0 con 0x7f204406ec90 2023-11-28T11:09:27.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.500+0000 7f204bd93700 1 -- 172.21.15.156:0/1146117882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20340094a0 con 0x7f204406ec90 2023-11-28T11:09:27.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.500+0000 7f204bd93700 1 -- 172.21.15.156:0/1146117882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20440ce770 con 0x7f204406ec90 2023-11-28T11:09:27.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.501+0000 7f204bd93700 1 -- 172.21.15.156:0/1146117882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20440ce490 con 0x7f204406ec90 2023-11-28T11:09:27.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.501+0000 7f203a7fc700 1 -- 172.21.15.156:0/1146117882 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f203400f050 con 0x7f204406ec90 2023-11-28T11:09:27.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.501+0000 7f203a7fc700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20280682f0 0x7f202806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.502+0000 7f203a7fc700 1 -- 172.21.15.156:0/1146117882 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2034086a40 con 0x7f204406ec90 2023-11-28T11:09:27.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.502+0000 7f204932e700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20280682f0 0x7f202806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.502+0000 7f204932e700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20280682f0 0x7f202806a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f204415b240 tx=0x7f2040009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.506+0000 7f203a7fc700 1 -- 172.21.15.156:0/1146117882 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2034053220 con 0x7f204406ec90 2023-11-28T11:09:27.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.731+0000 7f204bd93700 1 -- 172.21.15.156:0/1146117882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20440cb6c0 con 0x7f204406ec90 2023-11-28T11:09:27.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.732+0000 7f203a7fc700 1 -- 172.21.15.156:0/1146117882 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f2034015090 con 0x7f204406ec90 2023-11-28T11:09:27.733 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:27.733 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1085440106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1085440106},{"type":"v1","addr":"172.21.15.186:6834","nonce":1085440106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1548245207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1548245207},{"type":"v1","addr":"172.21.15.156:6836","nonce":1548245207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4882,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3102077957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":3102077957},{"type":"v1","addr":"172.21.15.156:6837","nonce":3102077957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:15.391255+0000","modified":"2023-11-28T11:09:18.414744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3897746301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3897746301},{"type":"v1","addr":"172.21.15.186:6835","nonce":3897746301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:09:27.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.734+0000 7f204bd93700 1 -- 172.21.15.156:0/1146117882 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20280682f0 msgr2=0x7f202806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.734+0000 7f204bd93700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20280682f0 0x7f202806a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f204415b240 tx=0x7f2040009210 comp rx=0 tx=0).stop 2023-11-28T11:09:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.734+0000 7f204bd93700 1 -- 172.21.15.156:0/1146117882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f204406ec90 msgr2=0x7f20440c3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.734+0000 7f204bd93700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f204406ec90 0x7f20440c3040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2034009fd0 tx=0x7f2034004750 comp rx=0 tx=0).stop 2023-11-28T11:09:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.734+0000 7f204bd93700 1 -- 172.21.15.156:0/1146117882 shutdown_connections 2023-11-28T11:09:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.734+0000 7f204bd93700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f20280682f0 0x7f202806a7a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.734+0000 7f204bd93700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20440cec70 0x7f20440c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.734+0000 7f204bd93700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f204406ec90 0x7f20440c3040 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.734+0000 7f204bd93700 1 --2- 172.21.15.156:0/1146117882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20440c9f10 0x7f2044151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.734+0000 7f204bd93700 1 -- 172.21.15.156:0/1146117882 >> 172.21.15.156:0/1146117882 conn(0x7f20440b12b0 msgr2=0x7f20440b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.734+0000 7f204bd93700 1 -- 172.21.15.156:0/1146117882 shutdown_connections 2023-11-28T11:09:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:27.734+0000 7f204bd93700 1 -- 172.21.15.156:0/1146117882 wait complete. 2023-11-28T11:09:27.737 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T11:09:27.748 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json tag path /parentdir mytag 2023-11-28T11:09:27.896 INFO:teuthology.orchestra.run.smithi186.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2023-11-28T11:09:27.896 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2023-11-28T11:09:37.898 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2023-11-28T11:09:37.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.950+0000 7f4a1ee7a080 1 Processor -- start 2023-11-28T11:09:37.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.950+0000 7f4a1ee7a080 1 -- start start 2023-11-28T11:09:37.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a1ee7a080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590d8c065f0 0x5590d8c05150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a1ee7a080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c44100 0x5590d8c4bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a1ee7a080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590d8cda300 0x5590d8cd79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a1ee7a080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d8c48030 con 0x5590d8c44100 2023-11-28T11:09:37.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a1ee7a080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d8c05690 con 0x5590d8cda300 2023-11-28T11:09:37.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a1ee7a080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590d8c458d0 con 0x5590d8c065f0 2023-11-28T11:09:37.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a0ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c44100 0x5590d8c4bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a0ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c44100 0x5590d8c4bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34038/0 (socket says 172.21.15.156:34038) 2023-11-28T11:09:37.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a0ffff700 1 -- 172.21.15.156:0/205648990 learned_addr learned my addr 172.21.15.156:0/205648990 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:37.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a1577e700 1 --2- 172.21.15.156:0/205648990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590d8cda300 0x5590d8cd79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a14f7d700 1 --2- 172.21.15.156:0/205648990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590d8c065f0 0x5590d8c05150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a0ffff700 1 -- 172.21.15.156:0/205648990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590d8c065f0 msgr2=0x5590d8c05150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:37.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a0ffff700 1 --2- 172.21.15.156:0/205648990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590d8c065f0 0x5590d8c05150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a0ffff700 1 -- 172.21.15.156:0/205648990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590d8cda300 msgr2=0x5590d8cd79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:37.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a0ffff700 1 --2- 172.21.15.156:0/205648990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590d8cda300 0x5590d8cd79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a0ffff700 1 -- 172.21.15.156:0/205648990 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a00005de0 con 0x5590d8c44100 2023-11-28T11:09:37.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a14f7d700 1 --2- 172.21.15.156:0/205648990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590d8c065f0 0x5590d8c05150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:37.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a1577e700 1 --2- 172.21.15.156:0/205648990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590d8cda300 0x5590d8cd79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:37.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a0ffff700 1 --2- 172.21.15.156:0/205648990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c44100 0x5590d8c4bf20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4a00015040 tx=0x7f4a0000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9b1be7021ef016 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.951+0000 7f4a0effd700 1 -- 172.21.15.156:0/205648990 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00008670 con 0x5590d8c44100 2023-11-28T11:09:37.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.952+0000 7f4a0effd700 1 -- 172.21.15.156:0/205648990 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a00008b40 con 0x5590d8c44100 2023-11-28T11:09:37.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.952+0000 7f4a0effd700 1 -- 172.21.15.156:0/205648990 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0001e450 con 0x5590d8c44100 2023-11-28T11:09:37.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.952+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/205648990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c44100 msgr2=0x5590d8c4bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.952+0000 7f4a1ee7a080 1 --2- 172.21.15.156:0/205648990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c44100 0x5590d8c4bf20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4a00015040 tx=0x7f4a0000d000 comp rx=0 tx=0).stop 2023-11-28T11:09:37.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.952+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/205648990 shutdown_connections 2023-11-28T11:09:37.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.952+0000 7f4a1ee7a080 1 --2- 172.21.15.156:0/205648990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590d8cda300 0x5590d8cd79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.952+0000 7f4a1ee7a080 1 --2- 172.21.15.156:0/205648990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590d8c065f0 0x5590d8c05150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.952+0000 7f4a1ee7a080 1 --2- 172.21.15.156:0/205648990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c44100 0x5590d8c4bf20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.952+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/205648990 >> 172.21.15.156:0/205648990 conn(0x5590d8c31b60 msgr2=0x5590d8c35190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:37.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.952+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/205648990 shutdown_connections 2023-11-28T11:09:37.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.952+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/205648990 wait complete. 2023-11-28T11:09:37.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.953+0000 7f4a1ee7a080 1 Processor -- start 2023-11-28T11:09:37.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.953+0000 7f4a1ee7a080 1 -- start start 2023-11-28T11:09:37.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.953+0000 7f4a1ee7a080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c065f0 0x5590d8c03e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.953+0000 7f4a1ee7a080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590d8c44100 0x5590d8ccb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.953+0000 7f4a1ee7a080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590d8cda300 0x5590d8ccb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a1ee7a080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d8c03400 con 0x5590d8c065f0 2023-11-28T11:09:37.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a1ee7a080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d8c35ca0 con 0x5590d8cda300 2023-11-28T11:09:37.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a1ee7a080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590d8c4ea20 con 0x5590d8c44100 2023-11-28T11:09:37.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a14f7d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c065f0 0x5590d8c03e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a14f7d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c065f0 0x5590d8c03e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34042/0 (socket says 172.21.15.156:34042) 2023-11-28T11:09:37.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a14f7d700 1 -- 172.21.15.156:0/1797062688 learned_addr learned my addr 172.21.15.156:0/1797062688 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:37.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a0ffff700 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590d8c44100 0x5590d8ccb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a1577e700 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590d8cda300 0x5590d8ccb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a14f7d700 1 -- 172.21.15.156:0/1797062688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590d8c44100 msgr2=0x5590d8ccb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a14f7d700 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590d8c44100 0x5590d8ccb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a14f7d700 1 -- 172.21.15.156:0/1797062688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590d8cda300 msgr2=0x5590d8ccb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a14f7d700 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590d8cda300 0x5590d8ccb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a14f7d700 1 -- 172.21.15.156:0/1797062688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590d8b4b4a0 con 0x5590d8c065f0 2023-11-28T11:09:37.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a14f7d700 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c065f0 0x5590d8c03e50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800bee0 tx=0x7f4a080076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a0dffb700 1 -- 172.21.15.156:0/1797062688 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08010070 con 0x5590d8c065f0 2023-11-28T11:09:37.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a0dffb700 1 -- 172.21.15.156:0/1797062688 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a08007e00 con 0x5590d8c065f0 2023-11-28T11:09:37.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/1797062688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5590d8b724f0 con 0x5590d8c065f0 2023-11-28T11:09:37.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a0dffb700 1 -- 172.21.15.156:0/1797062688 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08015660 con 0x5590d8c065f0 2023-11-28T11:09:37.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.954+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/1797062688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590d8b58120 con 0x5590d8c065f0 2023-11-28T11:09:37.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.956+0000 7f4a0dffb700 1 -- 172.21.15.156:0/1797062688 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4a0801b020 con 0x5590d8c065f0 2023-11-28T11:09:37.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.956+0000 7f4a0dffb700 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f49f8068e20 0x7f49f806b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.956+0000 7f4a0dffb700 1 -- 172.21.15.156:0/1797062688 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4a08083f80 con 0x5590d8c065f0 2023-11-28T11:09:37.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.956+0000 7f4a0ffff700 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f49f8068e20 0x7f49f806b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.956+0000 7f4a1ee7a080 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x5590d8c04390 0x5590d8cd4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.956+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/1797062688 --> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) v8 -- 0x5590d8c4c580 con 0x5590d8c04390 2023-11-28T11:09:37.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.957+0000 7f4a0ffff700 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f49f8068e20 0x7f49f806b2d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4a00015040 tx=0x7f4a00014040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.957+0000 7f4a1577e700 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x5590d8c04390 0x5590d8cd4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.957+0000 7f4a1577e700 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x5590d8c04390 0x5590d8cd4e60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.958+0000 7f4a1577e700 1 -- 172.21.15.156:0/1797062688 <== osd.7 v2:172.21.15.186:6816/103780 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f4a1000d660 con 0x5590d8c04390 2023-11-28T11:09:37.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.958+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/1797062688 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x5590d8c04390 msgr2=0x5590d8cd4e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.958+0000 7f4a1ee7a080 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x5590d8c04390 0x5590d8cd4e60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.958+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/1797062688 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f49f8068e20 msgr2=0x7f49f806b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.958+0000 7f4a1ee7a080 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f49f8068e20 0x7f49f806b2d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4a00015040 tx=0x7f4a00014040 comp rx=0 tx=0).stop 2023-11-28T11:09:37.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.958+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/1797062688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c065f0 msgr2=0x5590d8c03e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.958+0000 7f4a1ee7a080 1 --2- 172.21.15.156:0/1797062688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590d8c065f0 0x5590d8c03e50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800bee0 tx=0x7f4a080076d0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.958+0000 7f4a1577e700 1 -- 172.21.15.156:0/1797062688 reap_dead start 2023-11-28T11:09:37.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.958+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/1797062688 shutdown_connections 2023-11-28T11:09:37.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.958+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/1797062688 >> 172.21.15.156:0/1797062688 conn(0x5590d8c31b60 msgr2=0x5590d8c32310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:37.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.958+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/1797062688 shutdown_connections 2023-11-28T11:09:37.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:37.958+0000 7f4a1ee7a080 1 -- 172.21.15.156:0/1797062688 wait complete. 2023-11-28T11:09:37.965 INFO:teuthology.orchestra.run.smithi156.stdout:mytag 2023-11-28T11:09:37.965 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2023-11-28T11:09:38.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.010+0000 7f86bb21b080 1 Processor -- start 2023-11-28T11:09:38.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.010+0000 7f86bb21b080 1 -- start start 2023-11-28T11:09:38.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.011+0000 7f86bb21b080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33a73b0 0x55b3b33e5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.011+0000 7f86bb21b080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b3b33d67e0 0x55b3b33d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.011+0000 7f86bb21b080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3b347d310 0x55b3b33d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.011+0000 7f86bb21b080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3b32e1fe0 con 0x55b3b33a73b0 2023-11-28T11:09:38.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.011+0000 7f86bb21b080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3b347bf40 con 0x55b3b347d310 2023-11-28T11:09:38.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.011+0000 7f86bb21b080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3b347c190 con 0x55b3b33d67e0 2023-11-28T11:09:38.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.011+0000 7f86b131e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33a73b0 0x55b3b33e5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.011+0000 7f86b131e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33a73b0 0x55b3b33e5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34056/0 (socket says 172.21.15.156:34056) 2023-11-28T11:09:38.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.011+0000 7f86b131e700 1 -- 172.21.15.156:0/846213951 learned_addr learned my addr 172.21.15.156:0/846213951 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:38.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.011+0000 7f86b0b1d700 1 --2- 172.21.15.156:0/846213951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b3b33d67e0 0x55b3b33d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.011+0000 7f86b1b1f700 1 --2- 172.21.15.156:0/846213951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3b347d310 0x55b3b33d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.011+0000 7f86b131e700 1 -- 172.21.15.156:0/846213951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b3b33d67e0 msgr2=0x55b3b33d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86b131e700 1 --2- 172.21.15.156:0/846213951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b3b33d67e0 0x55b3b33d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86b131e700 1 -- 172.21.15.156:0/846213951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3b347d310 msgr2=0x55b3b33d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86b131e700 1 --2- 172.21.15.156:0/846213951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3b347d310 0x55b3b33d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86b131e700 1 -- 172.21.15.156:0/846213951 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ac00a1c0 con 0x55b3b33a73b0 2023-11-28T11:09:38.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86b0b1d700 1 --2- 172.21.15.156:0/846213951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b3b33d67e0 0x55b3b33d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:38.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86b1b1f700 1 --2- 172.21.15.156:0/846213951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3b347d310 0x55b3b33d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:38.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86b131e700 1 --2- 172.21.15.156:0/846213951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33a73b0 0x55b3b33e5cb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f86a8001ce0 tx=0x7f86a800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7d2491c5e5b444 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86a37fe700 1 -- 172.21.15.156:0/846213951 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8014070 con 0x55b3b33a73b0 2023-11-28T11:09:38.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86a37fe700 1 -- 172.21.15.156:0/846213951 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a8008030 con 0x55b3b33a73b0 2023-11-28T11:09:38.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86a37fe700 1 -- 172.21.15.156:0/846213951 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a800bd90 con 0x55b3b33a73b0 2023-11-28T11:09:38.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86bb21b080 1 -- 172.21.15.156:0/846213951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33a73b0 msgr2=0x55b3b33e5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86bb21b080 1 --2- 172.21.15.156:0/846213951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33a73b0 0x55b3b33e5cb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f86a8001ce0 tx=0x7f86a800b4c0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86bb21b080 1 -- 172.21.15.156:0/846213951 shutdown_connections 2023-11-28T11:09:38.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86bb21b080 1 --2- 172.21.15.156:0/846213951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3b347d310 0x55b3b33d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86bb21b080 1 --2- 172.21.15.156:0/846213951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33a73b0 0x55b3b33e5cb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86bb21b080 1 --2- 172.21.15.156:0/846213951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b3b33d67e0 0x55b3b33d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86bb21b080 1 -- 172.21.15.156:0/846213951 >> 172.21.15.156:0/846213951 conn(0x55b3b33d3b60 msgr2=0x55b3b347c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.012+0000 7f86bb21b080 1 -- 172.21.15.156:0/846213951 shutdown_connections 2023-11-28T11:09:38.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.013+0000 7f86bb21b080 1 -- 172.21.15.156:0/846213951 wait complete. 2023-11-28T11:09:38.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.013+0000 7f86bb21b080 1 Processor -- start 2023-11-28T11:09:38.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.013+0000 7f86bb21b080 1 -- start start 2023-11-28T11:09:38.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86bb21b080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3b33a73b0 0x55b3b34741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86bb21b080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33d67e0 0x55b3b347fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86bb21b080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b3b347d310 0x55b3b33cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86bb21b080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3b3474d60 con 0x55b3b33d67e0 2023-11-28T11:09:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86bb21b080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3b3475210 con 0x55b3b33a73b0 2023-11-28T11:09:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86bb21b080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3b33cfde0 con 0x55b3b347d310 2023-11-28T11:09:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86b0b1d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33d67e0 0x55b3b347fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86b0b1d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33d67e0 0x55b3b347fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34064/0 (socket says 172.21.15.156:34064) 2023-11-28T11:09:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86b0b1d700 1 -- 172.21.15.156:0/3393188271 learned_addr learned my addr 172.21.15.156:0/3393188271 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86b1b1f700 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b3b347d310 0x55b3b33cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86b131e700 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3b33a73b0 0x55b3b34741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86b0b1d700 1 -- 172.21.15.156:0/3393188271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b3b347d310 msgr2=0x55b3b33cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86b0b1d700 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b3b347d310 0x55b3b33cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86b0b1d700 1 -- 172.21.15.156:0/3393188271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3b33a73b0 msgr2=0x55b3b34741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86b0b1d700 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3b33a73b0 0x55b3b34741d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86b0b1d700 1 -- 172.21.15.156:0/3393188271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a800a040 con 0x55b3b33d67e0 2023-11-28T11:09:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86b131e700 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3b33a73b0 0x55b3b34741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86b0b1d700 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33d67e0 0x55b3b347fbc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009fd0 tx=0x7f86a4008730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.014+0000 7f86a27fc700 1 -- 172.21.15.156:0/3393188271 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4018070 con 0x55b3b33d67e0 2023-11-28T11:09:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.015+0000 7f86a27fc700 1 -- 172.21.15.156:0/3393188271 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a4010040 con 0x55b3b33d67e0 2023-11-28T11:09:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.015+0000 7f86bb21b080 1 -- 172.21.15.156:0/3393188271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b3b32fa120 con 0x55b3b33d67e0 2023-11-28T11:09:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.015+0000 7f86a27fc700 1 -- 172.21.15.156:0/3393188271 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a40067d0 con 0x55b3b33d67e0 2023-11-28T11:09:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.015+0000 7f86bb21b080 1 -- 172.21.15.156:0/3393188271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3b32ed830 con 0x55b3b33d67e0 2023-11-28T11:09:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.016+0000 7f86a27fc700 1 -- 172.21.15.156:0/3393188271 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f86a4006930 con 0x55b3b33d67e0 2023-11-28T11:09:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.016+0000 7f86a27fc700 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8694066c40 0x7f86940690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.016+0000 7f86a27fc700 1 -- 172.21.15.156:0/3393188271 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f86a40847f0 con 0x55b3b33d67e0 2023-11-28T11:09:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.016+0000 7f86b131e700 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8694066c40 0x7f86940690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.016+0000 7f86bb21b080 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x55b3b3480100 0x55b3b3521c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.016+0000 7f86bb21b080 1 -- 172.21.15.156:0/3393188271 --> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) v8 -- 0x55b3b35155f0 con 0x55b3b3480100 2023-11-28T11:09:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.017+0000 7f86b131e700 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8694066c40 0x7f86940690f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f86a800b500 tx=0x7f86a8009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.017+0000 7f86b1b1f700 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x55b3b3480100 0x55b3b3521c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.017+0000 7f86b1b1f700 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x55b3b3480100 0x55b3b3521c10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.018+0000 7f86b1b1f700 1 -- 172.21.15.156:0/3393188271 <== osd.5 v2:172.21.15.186:6808/103778 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f86ac00e9e0 con 0x55b3b3480100 2023-11-28T11:09:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.018+0000 7f86bb21b080 1 -- 172.21.15.156:0/3393188271 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x55b3b3480100 msgr2=0x55b3b3521c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.018+0000 7f86bb21b080 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x55b3b3480100 0x55b3b3521c10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.018+0000 7f86bb21b080 1 -- 172.21.15.156:0/3393188271 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8694066c40 msgr2=0x7f86940690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.018+0000 7f86bb21b080 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8694066c40 0x7f86940690f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f86a800b500 tx=0x7f86a8009000 comp rx=0 tx=0).stop 2023-11-28T11:09:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.018+0000 7f86bb21b080 1 -- 172.21.15.156:0/3393188271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33d67e0 msgr2=0x55b3b347fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.018+0000 7f86bb21b080 1 --2- 172.21.15.156:0/3393188271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3b33d67e0 0x55b3b347fbc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009fd0 tx=0x7f86a4008730 comp rx=0 tx=0).stop 2023-11-28T11:09:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.018+0000 7f86b1b1f700 1 -- 172.21.15.156:0/3393188271 reap_dead start 2023-11-28T11:09:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.018+0000 7f86bb21b080 1 -- 172.21.15.156:0/3393188271 shutdown_connections 2023-11-28T11:09:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.018+0000 7f86bb21b080 1 -- 172.21.15.156:0/3393188271 >> 172.21.15.156:0/3393188271 conn(0x55b3b33d3b60 msgr2=0x55b3b33e5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.018+0000 7f86bb21b080 1 -- 172.21.15.156:0/3393188271 shutdown_connections 2023-11-28T11:09:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.018+0000 7f86bb21b080 1 -- 172.21.15.156:0/3393188271 wait complete. 2023-11-28T11:09:38.030 INFO:teuthology.orchestra.run.smithi156.stdout:mytag 2023-11-28T11:09:38.030 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-28T11:09:38.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.074+0000 7f71841dc080 1 Processor -- start 2023-11-28T11:09:38.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.074+0000 7f71841dc080 1 -- start start 2023-11-28T11:09:38.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f71841dc080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55660e3289d0 0x55660e3b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f71841dc080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 0x55660e30c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f71841dc080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55660e3bac10 0x55660e30de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f71841dc080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55660e218fe0 con 0x55660e3b6450 2023-11-28T11:09:38.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f71841dc080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55660e3272f0 con 0x55660e3289d0 2023-11-28T11:09:38.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f71841dc080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55660e30d5e0 con 0x55660e3bac10 2023-11-28T11:09:38.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f7179ade700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 0x55660e30c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f7179ade700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 0x55660e30c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34080/0 (socket says 172.21.15.156:34080) 2023-11-28T11:09:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f7179ade700 1 -- 172.21.15.156:0/3670848628 learned_addr learned my addr 172.21.15.156:0/3670848628 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f717a2df700 1 --2- 172.21.15.156:0/3670848628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55660e3289d0 0x55660e3b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f717aae0700 1 --2- 172.21.15.156:0/3670848628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55660e3bac10 0x55660e30de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f7179ade700 1 -- 172.21.15.156:0/3670848628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55660e3bac10 msgr2=0x55660e30de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f7179ade700 1 --2- 172.21.15.156:0/3670848628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55660e3bac10 0x55660e30de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f7179ade700 1 -- 172.21.15.156:0/3670848628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55660e3289d0 msgr2=0x55660e3b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f7179ade700 1 --2- 172.21.15.156:0/3670848628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55660e3289d0 0x55660e3b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f7179ade700 1 -- 172.21.15.156:0/3670848628 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c005d20 con 0x55660e3b6450 2023-11-28T11:09:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f7179ade700 1 --2- 172.21.15.156:0/3670848628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 0x55660e30c130 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f717000a7a0 tx=0x7f717000aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19fd1b3808531f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f7178adc700 1 -- 172.21.15.156:0/3670848628 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717000f070 con 0x55660e3b6450 2023-11-28T11:09:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f7178adc700 1 -- 172.21.15.156:0/3670848628 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7170008580 con 0x55660e3b6450 2023-11-28T11:09:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.075+0000 7f7178adc700 1 -- 172.21.15.156:0/3670848628 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71700088f0 con 0x55660e3b6450 2023-11-28T11:09:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.076+0000 7f71841dc080 1 -- 172.21.15.156:0/3670848628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 msgr2=0x55660e30c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.076+0000 7f71841dc080 1 --2- 172.21.15.156:0/3670848628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 0x55660e30c130 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f717000a7a0 tx=0x7f717000aab0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.076+0000 7f71841dc080 1 -- 172.21.15.156:0/3670848628 shutdown_connections 2023-11-28T11:09:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.076+0000 7f71841dc080 1 --2- 172.21.15.156:0/3670848628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55660e3bac10 0x55660e30de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.076+0000 7f71841dc080 1 --2- 172.21.15.156:0/3670848628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55660e3289d0 0x55660e3b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.076+0000 7f71841dc080 1 --2- 172.21.15.156:0/3670848628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 0x55660e30c130 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.076+0000 7f71841dc080 1 -- 172.21.15.156:0/3670848628 >> 172.21.15.156:0/3670848628 conn(0x55660e30ab60 msgr2=0x55660e3b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.076+0000 7f71841dc080 1 -- 172.21.15.156:0/3670848628 shutdown_connections 2023-11-28T11:09:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.076+0000 7f71841dc080 1 -- 172.21.15.156:0/3670848628 wait complete. 2023-11-28T11:09:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f71841dc080 1 Processor -- start 2023-11-28T11:09:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f71841dc080 1 -- start start 2023-11-28T11:09:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f71841dc080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55660e3289d0 0x55660e31cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f71841dc080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 0x55660e3a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f71841dc080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55660e3bac10 0x55660e3a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f71841dc080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55660e31f610 con 0x55660e3b6450 2023-11-28T11:09:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f71841dc080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55660e3a5c80 con 0x55660e3289d0 2023-11-28T11:09:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f71841dc080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55660e3a3ac0 con 0x55660e3bac10 2023-11-28T11:09:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f7179ade700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 0x55660e3a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f7179ade700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 0x55660e3a6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34086/0 (socket says 172.21.15.156:34086) 2023-11-28T11:09:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f7179ade700 1 -- 172.21.15.156:0/2099116158 learned_addr learned my addr 172.21.15.156:0/2099116158 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f717a2df700 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55660e3289d0 0x55660e31cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.077+0000 7f717aae0700 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55660e3bac10 0x55660e3a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f7179ade700 1 -- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55660e3bac10 msgr2=0x55660e3a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f7179ade700 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55660e3bac10 0x55660e3a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f7179ade700 1 -- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55660e3289d0 msgr2=0x55660e31cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f7179ade700 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55660e3289d0 0x55660e31cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f7179ade700 1 -- 172.21.15.156:0/2099116158 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717000a420 con 0x55660e3b6450 2023-11-28T11:09:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f717a2df700 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55660e3289d0 0x55660e31cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f717aae0700 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55660e3bac10 0x55660e3a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f7179ade700 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 0x55660e3a6580 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7170014000 tx=0x7f7170003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f716b7fe700 1 -- 172.21.15.156:0/2099116158 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170020860 con 0x55660e3b6450 2023-11-28T11:09:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f716b7fe700 1 -- 172.21.15.156:0/2099116158 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71700209e0 con 0x55660e3b6450 2023-11-28T11:09:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f71841dc080 1 -- 172.21.15.156:0/2099116158 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55660e231120 con 0x55660e3b6450 2023-11-28T11:09:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f716b7fe700 1 -- 172.21.15.156:0/2099116158 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170020d90 con 0x55660e3b6450 2023-11-28T11:09:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.078+0000 7f71841dc080 1 -- 172.21.15.156:0/2099116158 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55660e224830 con 0x55660e3b6450 2023-11-28T11:09:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.079+0000 7f716b7fe700 1 -- 172.21.15.156:0/2099116158 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7170026020 con 0x55660e3b6450 2023-11-28T11:09:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.080+0000 7f716b7fe700 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f715c066c10 0x7f715c0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.080+0000 7f716b7fe700 1 -- 172.21.15.156:0/2099116158 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7170085b70 con 0x55660e3b6450 2023-11-28T11:09:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.080+0000 7f717a2df700 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f715c066c10 0x7f715c0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.080+0000 7f71841dc080 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x55660e3a8fb0 0x55660e450680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.080+0000 7f71841dc080 1 -- 172.21.15.156:0/2099116158 --> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) v8 -- 0x55660e3a54d0 con 0x55660e3a8fb0 2023-11-28T11:09:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.080+0000 7f717a2df700 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f715c066c10 0x7f715c0690c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f716c015040 tx=0x7f716c00f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.080+0000 7f717aae0700 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x55660e3a8fb0 0x55660e450680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.080+0000 7f717aae0700 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x55660e3a8fb0 0x55660e450680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.081+0000 7f717aae0700 1 -- 172.21.15.156:0/2099116158 <== osd.5 v2:172.21.15.186:6808/103778 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f7174003450 con 0x55660e3a8fb0 2023-11-28T11:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.081+0000 7f71841dc080 1 -- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x55660e3a8fb0 msgr2=0x55660e450680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.081+0000 7f71841dc080 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x55660e3a8fb0 0x55660e450680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.081+0000 7f71841dc080 1 -- 172.21.15.156:0/2099116158 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f715c066c10 msgr2=0x7f715c0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.081+0000 7f71841dc080 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f715c066c10 0x7f715c0690c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f716c015040 tx=0x7f716c00f000 comp rx=0 tx=0).stop 2023-11-28T11:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.081+0000 7f71841dc080 1 -- 172.21.15.156:0/2099116158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 msgr2=0x55660e3a6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.081+0000 7f71841dc080 1 --2- 172.21.15.156:0/2099116158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55660e3b6450 0x55660e3a6580 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7170014000 tx=0x7f7170003020 comp rx=0 tx=0).stop 2023-11-28T11:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.081+0000 7f717aae0700 1 -- 172.21.15.156:0/2099116158 reap_dead start 2023-11-28T11:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.081+0000 7f71841dc080 1 -- 172.21.15.156:0/2099116158 shutdown_connections 2023-11-28T11:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.081+0000 7f71841dc080 1 -- 172.21.15.156:0/2099116158 >> 172.21.15.156:0/2099116158 conn(0x55660e30ab60 msgr2=0x55660e2e0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.081+0000 7f71841dc080 1 -- 172.21.15.156:0/2099116158 shutdown_connections 2023-11-28T11:09:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.081+0000 7f71841dc080 1 -- 172.21.15.156:0/2099116158 wait complete. 2023-11-28T11:09:38.088 INFO:teuthology.orchestra.run.smithi156.stdout:mytag 2023-11-28T11:09:38.089 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-28T11:09:38.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.171+0000 7f35c7eaa080 1 Processor -- start 2023-11-28T11:09:38.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.171+0000 7f35c7eaa080 1 -- start start 2023-11-28T11:09:38.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35c7eaa080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755221c0 0x5563755f5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35c7eaa080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563755692f0 0x5563755f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35c7eaa080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5563755f80d0 0x55637554c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35c7eaa080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637545afe0 con 0x5563755221c0 2023-11-28T11:09:38.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35c7eaa080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563755ee410 con 0x5563755692f0 2023-11-28T11:09:38.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35c7eaa080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55637554d690 con 0x5563755f80d0 2023-11-28T11:09:38.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35bdfad700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755221c0 0x5563755f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35bdfad700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755221c0 0x5563755f5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34100/0 (socket says 172.21.15.156:34100) 2023-11-28T11:09:38.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35bdfad700 1 -- 172.21.15.156:0/105050053 learned_addr learned my addr 172.21.15.156:0/105050053 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:38.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35be7ae700 1 --2- 172.21.15.156:0/105050053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5563755f80d0 0x55637554c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35bd7ac700 1 --2- 172.21.15.156:0/105050053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563755692f0 0x5563755f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35bdfad700 1 -- 172.21.15.156:0/105050053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5563755f80d0 msgr2=0x55637554c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35bdfad700 1 --2- 172.21.15.156:0/105050053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5563755f80d0 0x55637554c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35bdfad700 1 -- 172.21.15.156:0/105050053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563755692f0 msgr2=0x5563755f74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35bdfad700 1 --2- 172.21.15.156:0/105050053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563755692f0 0x5563755f74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35bdfad700 1 -- 172.21.15.156:0/105050053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b800a100 con 0x5563755221c0 2023-11-28T11:09:38.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35be7ae700 1 --2- 172.21.15.156:0/105050053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5563755f80d0 0x55637554c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:38.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.172+0000 7f35bdfad700 1 --2- 172.21.15.156:0/105050053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755221c0 0x5563755f5680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f35b400b270 tx=0x7f35b400b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e139e309b9ed3668 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.173+0000 7f35affff700 1 -- 172.21.15.156:0/105050053 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4014070 con 0x5563755221c0 2023-11-28T11:09:38.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.173+0000 7f35affff700 1 -- 172.21.15.156:0/105050053 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b400f3f0 con 0x5563755221c0 2023-11-28T11:09:38.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.173+0000 7f35affff700 1 -- 172.21.15.156:0/105050053 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b400f760 con 0x5563755221c0 2023-11-28T11:09:38.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.173+0000 7f35c7eaa080 1 -- 172.21.15.156:0/105050053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755221c0 msgr2=0x5563755f5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.173+0000 7f35c7eaa080 1 --2- 172.21.15.156:0/105050053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755221c0 0x5563755f5680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f35b400b270 tx=0x7f35b400b580 comp rx=0 tx=0).stop 2023-11-28T11:09:38.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.173+0000 7f35c7eaa080 1 -- 172.21.15.156:0/105050053 shutdown_connections 2023-11-28T11:09:38.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.173+0000 7f35c7eaa080 1 --2- 172.21.15.156:0/105050053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5563755f80d0 0x55637554c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.173+0000 7f35c7eaa080 1 --2- 172.21.15.156:0/105050053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755221c0 0x5563755f5680 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.173+0000 7f35c7eaa080 1 --2- 172.21.15.156:0/105050053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563755692f0 0x5563755f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.173+0000 7f35c7eaa080 1 -- 172.21.15.156:0/105050053 >> 172.21.15.156:0/105050053 conn(0x55637551ec90 msgr2=0x5563755f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.173+0000 7f35c7eaa080 1 -- 172.21.15.156:0/105050053 shutdown_connections 2023-11-28T11:09:38.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.173+0000 7f35c7eaa080 1 -- 172.21.15.156:0/105050053 wait complete. 2023-11-28T11:09:38.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.174+0000 7f35c7eaa080 1 Processor -- start 2023-11-28T11:09:38.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.174+0000 7f35c7eaa080 1 -- start start 2023-11-28T11:09:38.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.174+0000 7f35c7eaa080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563755221c0 0x5563755ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.174+0000 7f35c7eaa080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5563755692f0 0x5563755e9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35c7eaa080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755f80d0 0x5563755eac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35c7eaa080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563755e96c0 con 0x5563755f80d0 2023-11-28T11:09:38.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35c7eaa080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563755e8ee0 con 0x5563755221c0 2023-11-28T11:09:38.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35c7eaa080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55637554ded0 con 0x5563755692f0 2023-11-28T11:09:38.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35be7ae700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755f80d0 0x5563755eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35be7ae700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755f80d0 0x5563755eac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34110/0 (socket says 172.21.15.156:34110) 2023-11-28T11:09:38.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35be7ae700 1 -- 172.21.15.156:0/2932697533 learned_addr learned my addr 172.21.15.156:0/2932697533 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:38.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35bd7ac700 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5563755692f0 0x5563755e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35bdfad700 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563755221c0 0x5563755ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35be7ae700 1 -- 172.21.15.156:0/2932697533 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5563755692f0 msgr2=0x5563755e9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35be7ae700 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5563755692f0 0x5563755e9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35be7ae700 1 -- 172.21.15.156:0/2932697533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563755221c0 msgr2=0x5563755ecaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35be7ae700 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563755221c0 0x5563755ecaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35be7ae700 1 -- 172.21.15.156:0/2932697533 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b4001b40 con 0x5563755f80d0 2023-11-28T11:09:38.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35be7ae700 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755f80d0 0x5563755eac90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f35b8003000 tx=0x7f35b8003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35aeffd700 1 -- 172.21.15.156:0/2932697533 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b801b650 con 0x5563755f80d0 2023-11-28T11:09:38.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35aeffd700 1 -- 172.21.15.156:0/2932697533 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b801b7b0 con 0x5563755f80d0 2023-11-28T11:09:38.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35aeffd700 1 -- 172.21.15.156:0/2932697533 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b801bb60 con 0x5563755f80d0 2023-11-28T11:09:38.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35c7eaa080 1 -- 172.21.15.156:0/2932697533 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556375473120 con 0x5563755f80d0 2023-11-28T11:09:38.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.175+0000 7f35c7eaa080 1 -- 172.21.15.156:0/2932697533 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556375466830 con 0x5563755f80d0 2023-11-28T11:09:38.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.177+0000 7f35aeffd700 1 -- 172.21.15.156:0/2932697533 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f35b802c070 con 0x5563755f80d0 2023-11-28T11:09:38.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.177+0000 7f35aeffd700 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35a0066c10 0x7f35a00690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.177+0000 7f35aeffd700 1 -- 172.21.15.156:0/2932697533 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f35b8085e40 con 0x5563755f80d0 2023-11-28T11:09:38.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.177+0000 7f35bdfad700 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35a0066c10 0x7f35a00690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.177+0000 7f35c7eaa080 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x5563755fb670 0x556375552230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.177+0000 7f35c7eaa080 1 -- 172.21.15.156:0/2932697533 --> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) v8 -- 0x5563755eb1d0 con 0x5563755fb670 2023-11-28T11:09:38.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.177+0000 7f35bd7ac700 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x5563755fb670 0x556375552230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.177+0000 7f35bdfad700 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35a0066c10 0x7f35a00690c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f35b400b270 tx=0x7f35b4011040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35bd7ac700 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x5563755fb670 0x556375552230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35bd7ac700 1 -- 172.21.15.156:0/2932697533 <== osd.5 v2:172.21.15.186:6808/103778 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f35b0009960 con 0x5563755fb670 2023-11-28T11:09:38.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35c7eaa080 1 -- 172.21.15.156:0/2932697533 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x5563755fb670 msgr2=0x556375552230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35c7eaa080 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x5563755fb670 0x556375552230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35c7eaa080 1 -- 172.21.15.156:0/2932697533 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35a0066c10 msgr2=0x7f35a00690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35c7eaa080 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35a0066c10 0x7f35a00690c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f35b400b270 tx=0x7f35b4011040 comp rx=0 tx=0).stop 2023-11-28T11:09:38.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35c7eaa080 1 -- 172.21.15.156:0/2932697533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755f80d0 msgr2=0x5563755eac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35c7eaa080 1 --2- 172.21.15.156:0/2932697533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5563755f80d0 0x5563755eac90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f35b8003000 tx=0x7f35b8003330 comp rx=0 tx=0).stop 2023-11-28T11:09:38.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35be7ae700 1 -- 172.21.15.156:0/2932697533 reap_dead start 2023-11-28T11:09:38.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35c7eaa080 1 -- 172.21.15.156:0/2932697533 shutdown_connections 2023-11-28T11:09:38.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35c7eaa080 1 -- 172.21.15.156:0/2932697533 >> 172.21.15.156:0/2932697533 conn(0x55637551ec90 msgr2=0x55637551f060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35c7eaa080 1 -- 172.21.15.156:0/2932697533 shutdown_connections 2023-11-28T11:09:38.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.178+0000 7f35c7eaa080 1 -- 172.21.15.156:0/2932697533 wait complete. 2023-11-28T11:09:38.189 INFO:teuthology.orchestra.run.smithi156.stdout:mytag 2023-11-28T11:09:38.189 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-28T11:09:38.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.272+0000 7f0208068080 1 Processor -- start 2023-11-28T11:09:38.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.272+0000 7f0208068080 1 -- start start 2023-11-28T11:09:38.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:09:38.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f0208068080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3215019d0 0x55e3214b9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f0208068080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32158f450 0x55e321592510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f0208068080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 0x55e32158f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f0208068080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3213f1fe0 con 0x55e321593de0 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f0208068080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e321592200 con 0x55e3215019d0 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f0208068080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3214e76a0 con 0x55e32158f450 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fe96c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 0x55e32158f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fe96c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 0x55e32158f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34126/0 (socket says 172.21.15.156:34126) 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fe96c700 1 -- 172.21.15.156:0/2965015675 learned_addr learned my addr 172.21.15.156:0/2965015675 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fe16b700 1 --2- 172.21.15.156:0/2965015675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3215019d0 0x55e3214b9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fd96a700 1 --2- 172.21.15.156:0/2965015675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32158f450 0x55e321592510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fe96c700 1 -- 172.21.15.156:0/2965015675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32158f450 msgr2=0x55e321592510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fe96c700 1 --2- 172.21.15.156:0/2965015675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32158f450 0x55e321592510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fe96c700 1 -- 172.21.15.156:0/2965015675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3215019d0 msgr2=0x55e3214b9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fe96c700 1 --2- 172.21.15.156:0/2965015675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3215019d0 0x55e3214b9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fe96c700 1 -- 172.21.15.156:0/2965015675 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f800a100 con 0x55e321593de0 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fe96c700 1 --2- 172.21.15.156:0/2965015675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 0x55e32158f820 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f01f800a880 tx=0x7f01f8005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6865506b0571b342 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fc968700 1 -- 172.21.15.156:0/2965015675 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f800a400 con 0x55e321593de0 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fc968700 1 -- 172.21.15.156:0/2965015675 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f8012660 con 0x55e321593de0 2023-11-28T11:09:38.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.273+0000 7f01fc968700 1 -- 172.21.15.156:0/2965015675 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f8012a10 con 0x55e321593de0 2023-11-28T11:09:38.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.274+0000 7f0208068080 1 -- 172.21.15.156:0/2965015675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 msgr2=0x55e32158f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.274+0000 7f0208068080 1 --2- 172.21.15.156:0/2965015675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 0x55e32158f820 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f01f800a880 tx=0x7f01f8005d80 comp rx=0 tx=0).stop 2023-11-28T11:09:38.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.274+0000 7f0208068080 1 -- 172.21.15.156:0/2965015675 shutdown_connections 2023-11-28T11:09:38.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.274+0000 7f0208068080 1 --2- 172.21.15.156:0/2965015675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 0x55e32158f820 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.274+0000 7f0208068080 1 --2- 172.21.15.156:0/2965015675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3215019d0 0x55e3214b9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.274+0000 7f0208068080 1 --2- 172.21.15.156:0/2965015675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32158f450 0x55e321592510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.274+0000 7f0208068080 1 -- 172.21.15.156:0/2965015675 >> 172.21.15.156:0/2965015675 conn(0x55e3214e3b60 msgr2=0x55e3214f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.274+0000 7f0208068080 1 -- 172.21.15.156:0/2965015675 shutdown_connections 2023-11-28T11:09:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.274+0000 7f0208068080 1 -- 172.21.15.156:0/2965015675 wait complete. 2023-11-28T11:09:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.275+0000 7f0208068080 1 Processor -- start 2023-11-28T11:09:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.275+0000 7f0208068080 1 -- start start 2023-11-28T11:09:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.275+0000 7f0208068080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e3215019d0 0x55e3215938a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.275+0000 7f0208068080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32158f450 0x55e321585630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.275+0000 7f0208068080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 0x55e3214f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.275+0000 7f0208068080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3215928e0 con 0x55e321593de0 2023-11-28T11:09:38.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.275+0000 7f0208068080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3214fd3c0 con 0x55e32158f450 2023-11-28T11:09:38.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.275+0000 7f0208068080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3214fd0e0 con 0x55e3215019d0 2023-11-28T11:09:38.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01fe96c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 0x55e3214f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01fe96c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 0x55e3214f8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34132/0 (socket says 172.21.15.156:34132) 2023-11-28T11:09:38.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01fe96c700 1 -- 172.21.15.156:0/3987715547 learned_addr learned my addr 172.21.15.156:0/3987715547 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:38.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01fd96a700 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32158f450 0x55e321585630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01fe16b700 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e3215019d0 0x55e3215938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01fe96c700 1 -- 172.21.15.156:0/3987715547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e3215019d0 msgr2=0x55e3215938a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01fe96c700 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e3215019d0 0x55e3215938a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01fe96c700 1 -- 172.21.15.156:0/3987715547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32158f450 msgr2=0x55e321585630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01fe96c700 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32158f450 0x55e321585630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01fe96c700 1 -- 172.21.15.156:0/3987715547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f800a560 con 0x55e321593de0 2023-11-28T11:09:38.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01fe96c700 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 0x55e3214f8320 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55e3214e3760 tx=0x7f01f80033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01ef7fe700 1 -- 172.21.15.156:0/3987715547 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f8013bc0 con 0x55e321593de0 2023-11-28T11:09:38.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01ef7fe700 1 -- 172.21.15.156:0/3987715547 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f8013d40 con 0x55e321593de0 2023-11-28T11:09:38.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f0208068080 1 -- 172.21.15.156:0/3987715547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e3213fd830 con 0x55e321593de0 2023-11-28T11:09:38.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f01ef7fe700 1 -- 172.21.15.156:0/3987715547 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f801d680 con 0x55e321593de0 2023-11-28T11:09:38.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.276+0000 7f0208068080 1 -- 172.21.15.156:0/3987715547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f0005de0 con 0x55e321593de0 2023-11-28T11:09:38.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.277+0000 7f01ef7fe700 1 -- 172.21.15.156:0/3987715547 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f01f8019020 con 0x55e321593de0 2023-11-28T11:09:38.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.278+0000 7f01ef7fe700 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f01e0066c10 0x7f01e00690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.278+0000 7f01ef7fe700 1 -- 172.21.15.156:0/3987715547 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f01f8086ef0 con 0x55e321593de0 2023-11-28T11:09:38.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.278+0000 7f01fe16b700 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f01e0066c10 0x7f01e00690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.278+0000 7f0208068080 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x55e32158e1f0 0x55e3214f9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.278+0000 7f0208068080 1 -- 172.21.15.156:0/3987715547 --> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) v8 -- 0x55e3214fa040 con 0x55e32158e1f0 2023-11-28T11:09:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.278+0000 7f01fe16b700 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f01e0066c10 0x7f01e00690c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f01f4000c00 tx=0x7f01f4001740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.278+0000 7f01fd96a700 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x55e32158e1f0 0x55e3214f9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f01fd96a700 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x55e32158e1f0 0x55e3214f9b00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f01fd96a700 1 -- 172.21.15.156:0/3987715547 <== osd.7 v2:172.21.15.186:6816/103780 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f01f0019040 con 0x55e32158e1f0 2023-11-28T11:09:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2023-11-28T11:09:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f0208068080 1 -- 172.21.15.156:0/3987715547 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x55e32158e1f0 msgr2=0x55e3214f9b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f0208068080 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x55e32158e1f0 0x55e3214f9b00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f0208068080 1 -- 172.21.15.156:0/3987715547 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f01e0066c10 msgr2=0x7f01e00690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f0208068080 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f01e0066c10 0x7f01e00690c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f01f4000c00 tx=0x7f01f4001740 comp rx=0 tx=0).stop 2023-11-28T11:09:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f0208068080 1 -- 172.21.15.156:0/3987715547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 msgr2=0x55e3214f8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f0208068080 1 --2- 172.21.15.156:0/3987715547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e321593de0 0x55e3214f8320 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55e3214e3760 tx=0x7f01f80033a0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f01fe96c700 1 -- 172.21.15.156:0/3987715547 reap_dead start 2023-11-28T11:09:38.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f0208068080 1 -- 172.21.15.156:0/3987715547 shutdown_connections 2023-11-28T11:09:38.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f0208068080 1 -- 172.21.15.156:0/3987715547 >> 172.21.15.156:0/3987715547 conn(0x55e3214e3b60 msgr2=0x55e3215909c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f0208068080 1 -- 172.21.15.156:0/3987715547 shutdown_connections 2023-11-28T11:09:38.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.279+0000 7f0208068080 1 -- 172.21.15.156:0/3987715547 wait complete. 2023-11-28T11:09:38.297 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:09:38.298 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:09:38.345 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:09:38.389 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:09:38.389 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:09:38.390 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:38.408 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2550 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:09:38.409 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:09:38.409 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:38.448 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:09:38.449 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:09:38.449 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:38.466 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2550 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:09:38.466 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:09:38.466 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:38.506 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:09:38.506 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:38.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.625+0000 7fdd58b8e700 1 Processor -- start 2023-11-28T11:09:38.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.625+0000 7fdd58b8e700 1 -- start start 2023-11-28T11:09:38.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.626+0000 7fdd58b8e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd54070c50 0x7fdd540b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.626+0000 7fdd58b8e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdd540b0bb0 0x7fdd5415add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.626+0000 7fdd58b8e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd540b1010 con 0x7fdd54158200 2023-11-28T11:09:38.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.626+0000 7fdd58b8e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5415f070 con 0x7fdd540b0bb0 2023-11-28T11:09:38.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.626+0000 7fdd58b8e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5415e1a0 con 0x7fdd54070c50 2023-11-28T11:09:38.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.626+0000 7fdd5259c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fdd54158200 0x7fdd540caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49016/0 (socket says 172.21.15.156:49016) 2023-11-28T11:09:38.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.626+0000 7fdd5259c700 1 -- 172.21.15.156:0/758888487 learned_addr learned my addr 172.21.15.156:0/758888487 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:38.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.626+0000 7fdd52d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdd540b0bb0 0x7fdd5415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.627+0000 7fdd51d9b700 1 --2- 172.21.15.156:0/758888487 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd54070c50 0x7fdd540b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.627+0000 7fdd52d9d700 1 -- 172.21.15.156:0/758888487 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd54070c50 msgr2=0x7fdd540b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.627+0000 7fdd52d9d700 1 --2- 172.21.15.156:0/758888487 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd54070c50 0x7fdd540b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.627+0000 7fdd52d9d700 1 -- 172.21.15.156:0/758888487 >> v1:172.21.15.156:6789/0 conn(0x7fdd54158200 legacy=0x7fdd540caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.627+0000 7fdd5159a700 1 -- 172.21.15.156:0/758888487 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383938211 0 0) 0x7fdd540b1010 con 0x7fdd54158200 2023-11-28T11:09:38.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.627+0000 7fdd52d9d700 1 -- 172.21.15.156:0/758888487 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd540cb180 con 0x7fdd540b0bb0 2023-11-28T11:09:38.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.627+0000 7fdd52d9d700 1 --2- 172.21.15.156:0/758888487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd540b0bb0 0x7fdd5415add0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009d90 tx=0x7fdd4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0859b37050f58de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.627+0000 7fdd5159a700 1 -- 172.21.15.156:0/758888487 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00e070 con 0x7fdd540b0bb0 2023-11-28T11:09:38.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.627+0000 7fdd5159a700 1 -- 172.21.15.156:0/758888487 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c00bbd0 con 0x7fdd540b0bb0 2023-11-28T11:09:38.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.627+0000 7fdd5159a700 1 -- 172.21.15.156:0/758888487 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c012430 con 0x7fdd540b0bb0 2023-11-28T11:09:38.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.628+0000 7fdd58b8e700 1 -- 172.21.15.156:0/758888487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd540b0bb0 msgr2=0x7fdd5415add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.628+0000 7fdd58b8e700 1 --2- 172.21.15.156:0/758888487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd540b0bb0 0x7fdd5415add0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009d90 tx=0x7fdd4c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:38.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.628+0000 7fdd58b8e700 1 -- 172.21.15.156:0/758888487 shutdown_connections 2023-11-28T11:09:38.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.628+0000 7fdd58b8e700 1 --2- 172.21.15.156:0/758888487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd540b0bb0 0x7fdd5415add0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.628+0000 7fdd58b8e700 1 --2- 172.21.15.156:0/758888487 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd54070c50 0x7fdd540b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.628+0000 7fdd58b8e700 1 -- 172.21.15.156:0/758888487 >> 172.21.15.156:0/758888487 conn(0x7fdd540cb670 msgr2=0x7fdd540cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.628+0000 7fdd58b8e700 1 -- 172.21.15.156:0/758888487 shutdown_connections 2023-11-28T11:09:38.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.628+0000 7fdd58b8e700 1 -- 172.21.15.156:0/758888487 wait complete. 2023-11-28T11:09:38.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.629+0000 7fdd58b8e700 1 Processor -- start 2023-11-28T11:09:38.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.629+0000 7fdd58b8e700 1 -- start start 2023-11-28T11:09:38.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd58b8e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd54070c50 0x7fdd540c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd58b8e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd540b0bb0 0x7fdd54156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd58b8e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd54158200 0x7fdd54159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd58b8e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540c6210 con 0x7fdd54070c50 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd58b8e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540c64f0 con 0x7fdd540b0bb0 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd58b8e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540c67d0 con 0x7fdd54158200 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd5259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd54070c50 0x7fdd540c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd5259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd54070c50 0x7fdd540c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34144/0 (socket says 172.21.15.156:34144) 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd5259c700 1 -- 172.21.15.156:0/849665774 learned_addr learned my addr 172.21.15.156:0/849665774 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd52d9d700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd54158200 0x7fdd54159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd51d9b700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd540b0bb0 0x7fdd54156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd5259c700 1 -- 172.21.15.156:0/849665774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd54158200 msgr2=0x7fdd54159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd5259c700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd54158200 0x7fdd54159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd5259c700 1 -- 172.21.15.156:0/849665774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd540b0bb0 msgr2=0x7fdd54156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd5259c700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd540b0bb0 0x7fdd54156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd5259c700 1 -- 172.21.15.156:0/849665774 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c009a70 con 0x7fdd54070c50 2023-11-28T11:09:38.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd51d9b700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd540b0bb0 0x7fdd54156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:38.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.630+0000 7fdd5259c700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd54070c50 0x7fdd540c5c30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdd440011e0 tx=0x7fdd4400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.631+0000 7fdd42ffd700 1 -- 172.21.15.156:0/849665774 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44004c80 con 0x7fdd54070c50 2023-11-28T11:09:38.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.631+0000 7fdd42ffd700 1 -- 172.21.15.156:0/849665774 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd44004e00 con 0x7fdd54070c50 2023-11-28T11:09:38.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.631+0000 7fdd42ffd700 1 -- 172.21.15.156:0/849665774 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44007310 con 0x7fdd54070c50 2023-11-28T11:09:38.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.631+0000 7fdd58b8e700 1 -- 172.21.15.156:0/849665774 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd541585d0 con 0x7fdd54070c50 2023-11-28T11:09:38.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.631+0000 7fdd58b8e700 1 -- 172.21.15.156:0/849665774 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd540710f0 con 0x7fdd54070c50 2023-11-28T11:09:38.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.631+0000 7fdd58b8e700 1 -- 172.21.15.156:0/849665774 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5415e3f0 con 0x7fdd54070c50 2023-11-28T11:09:38.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.632+0000 7fdd42ffd700 1 -- 172.21.15.156:0/849665774 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdd4401a020 con 0x7fdd54070c50 2023-11-28T11:09:38.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.632+0000 7fdd42ffd700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd34068320 0x7fdd3406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.632+0000 7fdd42ffd700 1 -- 172.21.15.156:0/849665774 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdd44085890 con 0x7fdd54070c50 2023-11-28T11:09:38.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.632+0000 7fdd51d9b700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd34068320 0x7fdd3406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.633+0000 7fdd51d9b700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd34068320 0x7fdd3406a7d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009820 tx=0x7fdd48008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.637+0000 7fdd42ffd700 1 -- 172.21.15.156:0/849665774 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdd440521a0 con 0x7fdd54070c50 2023-11-28T11:09:38.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.862+0000 7fdd58b8e700 1 -- 172.21.15.156:0/849665774 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd5406fd60 con 0x7fdd54070c50 2023-11-28T11:09:38.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.862+0000 7fdd42ffd700 1 -- 172.21.15.156:0/849665774 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fdd440557c0 con 0x7fdd54070c50 2023-11-28T11:09:38.864 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:38.865 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1085440106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1085440106},{"type":"v1","addr":"172.21.15.186:6834","nonce":1085440106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1548245207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1548245207},{"type":"v1","addr":"172.21.15.156:6836","nonce":1548245207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4882,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3102077957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":3102077957},{"type":"v1","addr":"172.21.15.156:6837","nonce":3102077957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:15.391255+0000","modified":"2023-11-28T11:09:18.414744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3897746301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3897746301},{"type":"v1","addr":"172.21.15.186:6835","nonce":3897746301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:09:38.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.864+0000 7fdd58b8e700 1 -- 172.21.15.156:0/849665774 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd34068320 msgr2=0x7fdd3406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.864+0000 7fdd58b8e700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd34068320 0x7fdd3406a7d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009820 tx=0x7fdd48008040 comp rx=0 tx=0).stop 2023-11-28T11:09:38.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.864+0000 7fdd58b8e700 1 -- 172.21.15.156:0/849665774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd54070c50 msgr2=0x7fdd540c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.864+0000 7fdd58b8e700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd54070c50 0x7fdd540c5c30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdd440011e0 tx=0x7fdd4400cae0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.864+0000 7fdd58b8e700 1 -- 172.21.15.156:0/849665774 shutdown_connections 2023-11-28T11:09:38.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.864+0000 7fdd58b8e700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd34068320 0x7fdd3406a7d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.864+0000 7fdd58b8e700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd54158200 0x7fdd54159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.864+0000 7fdd58b8e700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd54070c50 0x7fdd540c5c30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.864+0000 7fdd58b8e700 1 --2- 172.21.15.156:0/849665774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd540b0bb0 0x7fdd54156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.864+0000 7fdd58b8e700 1 -- 172.21.15.156:0/849665774 >> 172.21.15.156:0/849665774 conn(0x7fdd540cb670 msgr2=0x7fdd5406d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.864+0000 7fdd58b8e700 1 -- 172.21.15.156:0/849665774 shutdown_connections 2023-11-28T11:09:38.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.864+0000 7fdd58b8e700 1 -- 172.21.15.156:0/849665774 wait complete. 2023-11-28T11:09:38.868 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T11:09:38.878 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:38.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.997+0000 7f7da3dbb700 1 Processor -- start 2023-11-28T11:09:38.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.997+0000 7f7da3dbb700 1 -- start start 2023-11-28T11:09:39.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da3dbb700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7d9c140950 0x7f7d9c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da3dbb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7d9c0b2e70 0x7f7d9c06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da3dbb700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c13ff00 con 0x7f7d9c0cd8c0 2023-11-28T11:09:39.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da3dbb700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9c1404c0 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da3dbb700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9c140dc0 con 0x7f7d9c140950 2023-11-28T11:09:39.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da1b57700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7d9c0cd8c0 0x7f7d9c0cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49030/0 (socket says 172.21.15.156:49030) 2023-11-28T11:09:39.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da1b57700 1 -- 172.21.15.156:0/323935812 learned_addr learned my addr 172.21.15.156:0/323935812 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:39.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da2358700 1 --2- 172.21.15.156:0/323935812 >> v2:172.21.15.156:3300/0 conn(0x7f7d9c0b2e70 0x7f7d9c06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da1356700 1 --2- 172.21.15.156:0/323935812 >> v2:172.21.15.186:3300/0 conn(0x7f7d9c140950 0x7f7d9c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da2358700 1 -- 172.21.15.156:0/323935812 >> v2:172.21.15.186:3300/0 conn(0x7f7d9c140950 msgr2=0x7f7d9c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:39.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da2358700 1 --2- 172.21.15.156:0/323935812 >> v2:172.21.15.186:3300/0 conn(0x7f7d9c140950 0x7f7d9c0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da2358700 1 -- 172.21.15.156:0/323935812 >> v1:172.21.15.156:6789/0 conn(0x7f7d9c0cd8c0 legacy=0x7f7d9c0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da0b55700 1 -- 172.21.15.156:0/323935812 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829850203 0 0) 0x7f7d9c13ff00 con 0x7f7d9c0cd8c0 2023-11-28T11:09:39.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.998+0000 7f7da2358700 1 -- 172.21.15.156:0/323935812 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c15aa80 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.999+0000 7f7da2358700 1 --2- 172.21.15.156:0/323935812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d9c0b2e70 0x7f7d9c06f850 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c071c40 tx=0x7f7d9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5037f669d346d186 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.999+0000 7f7da0b55700 1 -- 172.21.15.156:0/323935812 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9400e070 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.999+0000 7f7da0b55700 1 -- 172.21.15.156:0/323935812 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9400bbd0 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.999+0000 7f7da0b55700 1 -- 172.21.15.156:0/323935812 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d94012430 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.999+0000 7f7da3dbb700 1 -- 172.21.15.156:0/323935812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d9c0b2e70 msgr2=0x7f7d9c06f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.999+0000 7f7da3dbb700 1 --2- 172.21.15.156:0/323935812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d9c0b2e70 0x7f7d9c06f850 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c071c40 tx=0x7f7d9400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:39.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.999+0000 7f7da3dbb700 1 -- 172.21.15.156:0/323935812 shutdown_connections 2023-11-28T11:09:39.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.999+0000 7f7da3dbb700 1 --2- 172.21.15.156:0/323935812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d9c0b2e70 0x7f7d9c06f850 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.999+0000 7f7da3dbb700 1 --2- 172.21.15.156:0/323935812 >> v2:172.21.15.186:3300/0 conn(0x7f7d9c140950 0x7f7d9c0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.999+0000 7f7da3dbb700 1 -- 172.21.15.156:0/323935812 >> 172.21.15.156:0/323935812 conn(0x7f7d9c06aed0 msgr2=0x7f7d9c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.999+0000 7f7da3dbb700 1 -- 172.21.15.156:0/323935812 shutdown_connections 2023-11-28T11:09:39.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:38.999+0000 7f7da3dbb700 1 -- 172.21.15.156:0/323935812 wait complete. 2023-11-28T11:09:39.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.001+0000 7f7da3dbb700 1 Processor -- start 2023-11-28T11:09:39.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.001+0000 7f7da3dbb700 1 -- start start 2023-11-28T11:09:39.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da3dbb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d9c0b2e70 0x7f7d9c0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da3dbb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d9c0cd8c0 0x7f7d9c156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da3dbb700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d9c140950 0x7f7d9c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da3dbb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0c8610 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da3dbb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0c88f0 con 0x7f7d9c0cd8c0 2023-11-28T11:09:39.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da3dbb700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0c8bd0 con 0x7f7d9c140950 2023-11-28T11:09:39.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da1b57700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d9c0b2e70 0x7f7d9c0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da1b57700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d9c0b2e70 0x7f7d9c0c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34158/0 (socket says 172.21.15.156:34158) 2023-11-28T11:09:39.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da1b57700 1 -- 172.21.15.156:0/2857088364 learned_addr learned my addr 172.21.15.156:0/2857088364 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:39.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da1356700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d9c0cd8c0 0x7f7d9c156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da1b57700 1 -- 172.21.15.156:0/2857088364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d9c140950 msgr2=0x7f7d9c1586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da1b57700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d9c140950 0x7f7d9c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da1b57700 1 -- 172.21.15.156:0/2857088364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d9c0cd8c0 msgr2=0x7f7d9c156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da1b57700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d9c0cd8c0 0x7f7d9c156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da1b57700 1 -- 172.21.15.156:0/2857088364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d94009a70 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da1356700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d9c0cd8c0 0x7f7d9c156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.002+0000 7f7da1b57700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d9c0b2e70 0x7f7d9c0c7f80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0010e0 tx=0x7f7d8c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.003+0000 7f7d927fc700 1 -- 172.21.15.156:0/2857088364 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0074e0 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.003+0000 7f7d927fc700 1 -- 172.21.15.156:0/2857088364 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8c007640 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.003+0000 7f7da3dbb700 1 -- 172.21.15.156:0/2857088364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c0b35f0 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.003+0000 7f7d927fc700 1 -- 172.21.15.156:0/2857088364 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c005a70 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.003+0000 7f7da3dbb700 1 -- 172.21.15.156:0/2857088364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c0b3310 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.003+0000 7f7da3dbb700 1 -- 172.21.15.156:0/2857088364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9c140150 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.004+0000 7f7d927fc700 1 -- 172.21.15.156:0/2857088364 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d8c019030 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.004+0000 7f7d927fc700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7d800682f0 0x7f7d8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.004+0000 7f7d927fc700 1 -- 172.21.15.156:0/2857088364 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d8c085150 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.004+0000 7f7da1356700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7d800682f0 0x7f7d8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.005+0000 7f7da1356700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7d800682f0 0x7f7d8006a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0c9650 tx=0x7f7d98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.009+0000 7f7d927fc700 1 -- 172.21.15.156:0/2857088364 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7d8c051920 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.195+0000 7f7da3dbb700 1 -- 172.21.15.156:0/2857088364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d9c15bf90 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.197+0000 7f7d927fc700 1 -- 172.21.15.156:0/2857088364 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f7d8c017070 con 0x7f7d9c0b2e70 2023-11-28T11:09:39.198 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:39.199 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":37,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:16.393008+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:13.663041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:14.347248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:39.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.199+0000 7f7da3dbb700 1 -- 172.21.15.156:0/2857088364 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7d800682f0 msgr2=0x7f7d8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.199+0000 7f7da3dbb700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7d800682f0 0x7f7d8006a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0c9650 tx=0x7f7d98008040 comp rx=0 tx=0).stop 2023-11-28T11:09:39.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.199+0000 7f7da3dbb700 1 -- 172.21.15.156:0/2857088364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d9c0b2e70 msgr2=0x7f7d9c0c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.199+0000 7f7da3dbb700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d9c0b2e70 0x7f7d9c0c7f80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0010e0 tx=0x7f7d8c00cae0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.199+0000 7f7da3dbb700 1 -- 172.21.15.156:0/2857088364 shutdown_connections 2023-11-28T11:09:39.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.199+0000 7f7da3dbb700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7d800682f0 0x7f7d8006a7a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.199+0000 7f7da3dbb700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d9c140950 0x7f7d9c1586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.199+0000 7f7da3dbb700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d9c0b2e70 0x7f7d9c0c7f80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.199+0000 7f7da3dbb700 1 --2- 172.21.15.156:0/2857088364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d9c0cd8c0 0x7f7d9c156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.199+0000 7f7da3dbb700 1 -- 172.21.15.156:0/2857088364 >> 172.21.15.156:0/2857088364 conn(0x7f7d9c06aed0 msgr2=0x7f7d9c1e50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.199+0000 7f7da3dbb700 1 -- 172.21.15.156:0/2857088364 shutdown_connections 2023-11-28T11:09:39.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.199+0000 7f7da3dbb700 1 -- 172.21.15.156:0/2857088364 wait complete. 2023-11-28T11:09:39.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:09:13.663041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:09:14.347248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:09:39.213 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:39.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.336+0000 7f0dfd98a700 1 Processor -- start 2023-11-28T11:09:39.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.336+0000 7f0dfd98a700 1 -- start start 2023-11-28T11:09:39.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0dfd98a700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0df8161340 0x7f0df80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0dfd98a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0df80d2320 0x7f0df80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0dfd98a700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df80d1950 con 0x7f0df815c9b0 2023-11-28T11:09:39.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0dfd98a700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df80b5800 con 0x7f0df8161340 2023-11-28T11:09:39.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0dfd98a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df80b5300 con 0x7f0df80d2320 2023-11-28T11:09:39.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0df67fc700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0df8161340 0x7f0df80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0df77fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0df80d2320 0x7f0df80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0df67fc700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0df8161340 0x7f0df80d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:45182/0 (socket says 172.21.15.156:45182) 2023-11-28T11:09:39.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0df67fc700 1 -- 172.21.15.156:0/2061818742 learned_addr learned my addr 172.21.15.156:0/2061818742 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:39.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0df6ffd700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f0df815c9b0 0x7f0df80b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:59404/0 (socket says 172.21.15.156:59404) 2023-11-28T11:09:39.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0df67fc700 1 -- 172.21.15.156:0/2061818742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0df80d2320 msgr2=0x7f0df80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:39.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0df67fc700 1 --2- 172.21.15.156:0/2061818742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0df80d2320 0x7f0df80cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0df67fc700 1 -- 172.21.15.156:0/2061818742 >> v1:172.21.15.186:6789/0 conn(0x7f0df815c9b0 legacy=0x7f0df80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0df67fc700 1 -- 172.21.15.156:0/2061818742 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df80b5520 con 0x7f0df8161340 2023-11-28T11:09:39.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0df5ffb700 1 -- 172.21.15.156:0/2061818742 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703249294 0 0) 0x7f0df80d1950 con 0x7f0df815c9b0 2023-11-28T11:09:39.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0df77fe700 1 --2- 172.21.15.156:0/2061818742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0df80d2320 0x7f0df80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:39.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.337+0000 7f0df67fc700 1 --2- 172.21.15.156:0/2061818742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df8161340 0x7f0df80d1de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0098f0 tx=0x7f0dec009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86802382f338a260 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.338+0000 7f0df5ffb700 1 -- 172.21.15.156:0/2061818742 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec011830 con 0x7f0df8161340 2023-11-28T11:09:39.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.338+0000 7f0df5ffb700 1 -- 172.21.15.156:0/2061818742 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec011990 con 0x7f0df8161340 2023-11-28T11:09:39.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.338+0000 7f0df5ffb700 1 -- 172.21.15.156:0/2061818742 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec011d20 con 0x7f0df8161340 2023-11-28T11:09:39.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.338+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2061818742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df8161340 msgr2=0x7f0df80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.338+0000 7f0dfd98a700 1 --2- 172.21.15.156:0/2061818742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df8161340 0x7f0df80d1de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0098f0 tx=0x7f0dec009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:39.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.338+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2061818742 shutdown_connections 2023-11-28T11:09:39.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.338+0000 7f0dfd98a700 1 --2- 172.21.15.156:0/2061818742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0df80d2320 0x7f0df80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.338+0000 7f0dfd98a700 1 --2- 172.21.15.156:0/2061818742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df8161340 0x7f0df80d1de0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.338+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2061818742 >> 172.21.15.156:0/2061818742 conn(0x7f0df8072600 msgr2=0x7f0df80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.338+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2061818742 shutdown_connections 2023-11-28T11:09:39.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.338+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2061818742 wait complete. 2023-11-28T11:09:39.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.339+0000 7f0dfd98a700 1 Processor -- start 2023-11-28T11:09:39.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.339+0000 7f0dfd98a700 1 -- start start 2023-11-28T11:09:39.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.340+0000 7f0dfd98a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0df80d2320 0x7f0df8157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.340+0000 7f0dfd98a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0df815c9b0 0x7f0df80c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.340+0000 7f0dfd98a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df8161340 0x7f0df814d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.340+0000 7f0dfd98a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df80c90c0 con 0x7f0df80d2320 2023-11-28T11:09:39.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.340+0000 7f0dfd98a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df80c8230 con 0x7f0df8161340 2023-11-28T11:09:39.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.340+0000 7f0dfd98a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df80c84b0 con 0x7f0df815c9b0 2023-11-28T11:09:39.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.340+0000 7f0df6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0df80d2320 0x7f0df8157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.340+0000 7f0df6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0df80d2320 0x7f0df8157530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34172/0 (socket says 172.21.15.156:34172) 2023-11-28T11:09:39.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.340+0000 7f0df6ffd700 1 -- 172.21.15.156:0/2744899810 learned_addr learned my addr 172.21.15.156:0/2744899810 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:39.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0df77fe700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df8161340 0x7f0df814d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0df67fc700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0df815c9b0 0x7f0df80c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0df6ffd700 1 -- 172.21.15.156:0/2744899810 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0df815c9b0 msgr2=0x7f0df80c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0df6ffd700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0df815c9b0 0x7f0df80c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0df6ffd700 1 -- 172.21.15.156:0/2744899810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df8161340 msgr2=0x7f0df814d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0df6ffd700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df8161340 0x7f0df814d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0df6ffd700 1 -- 172.21.15.156:0/2744899810 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df0009a70 con 0x7f0df80d2320 2023-11-28T11:09:39.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0df67fc700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0df815c9b0 0x7f0df80c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:39.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0df6ffd700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0df80d2320 0x7f0df8157530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0de800c580 tx=0x7f0de800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0ddffff700 1 -- 172.21.15.156:0/2744899810 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8004c80 con 0x7f0df80d2320 2023-11-28T11:09:39.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0ddffff700 1 -- 172.21.15.156:0/2744899810 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de8004e00 con 0x7f0df80d2320 2023-11-28T11:09:39.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2744899810 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec009550 con 0x7f0df80d2320 2023-11-28T11:09:39.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0ddffff700 1 -- 172.21.15.156:0/2744899810 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8007310 con 0x7f0df80d2320 2023-11-28T11:09:39.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.341+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2744899810 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df80d15e0 con 0x7f0df80d2320 2023-11-28T11:09:39.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.342+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2744899810 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df80d1300 con 0x7f0df80d2320 2023-11-28T11:09:39.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.342+0000 7f0ddffff700 1 -- 172.21.15.156:0/2744899810 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0de801a020 con 0x7f0df80d2320 2023-11-28T11:09:39.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.343+0000 7f0ddffff700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0de40682f0 0x7f0de406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.343+0000 7f0ddffff700 1 -- 172.21.15.156:0/2744899810 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0de8085780 con 0x7f0df80d2320 2023-11-28T11:09:39.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.343+0000 7f0df67fc700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0de40682f0 0x7f0de406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.343+0000 7f0df67fc700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0de40682f0 0x7f0de406a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0dec000c00 tx=0x7f0dec008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.347+0000 7f0ddffff700 1 -- 172.21.15.156:0/2744899810 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0de8051f80 con 0x7f0df80d2320 2023-11-28T11:09:39.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.531+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2744899810 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0df815cd80 con 0x7f0df80d2320 2023-11-28T11:09:39.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.531+0000 7f0ddffff700 1 -- 172.21.15.156:0/2744899810 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0de80555a0 con 0x7f0df80d2320 2023-11-28T11:09:39.533 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:09:39.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.534+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2744899810 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0de40682f0 msgr2=0x7f0de406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.534+0000 7f0dfd98a700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0de40682f0 0x7f0de406a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0dec000c00 tx=0x7f0dec008030 comp rx=0 tx=0).stop 2023-11-28T11:09:39.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.534+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2744899810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0df80d2320 msgr2=0x7f0df8157530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.534+0000 7f0dfd98a700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0df80d2320 0x7f0df8157530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0de800c580 tx=0x7f0de800c940 comp rx=0 tx=0).stop 2023-11-28T11:09:39.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.534+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2744899810 shutdown_connections 2023-11-28T11:09:39.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.534+0000 7f0dfd98a700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0df8161340 0x7f0df814d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.534+0000 7f0dfd98a700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0df80d2320 0x7f0df8157530 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.534+0000 7f0dfd98a700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0de40682f0 0x7f0de406a7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.534+0000 7f0dfd98a700 1 --2- 172.21.15.156:0/2744899810 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0df815c9b0 0x7f0df80c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.534+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2744899810 >> 172.21.15.156:0/2744899810 conn(0x7f0df8072600 msgr2=0x7f0df8159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.534+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2744899810 shutdown_connections 2023-11-28T11:09:39.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.534+0000 7f0dfd98a700 1 -- 172.21.15.156:0/2744899810 wait complete. 2023-11-28T11:09:39.547 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:39.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.670+0000 7ff10acd1700 1 Processor -- start 2023-11-28T11:09:39.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.670+0000 7ff10acd1700 1 -- start start 2023-11-28T11:09:39.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.670+0000 7ff10acd1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff10406dee0 0x7ff10406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.670+0000 7ff10acd1700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff104160540 0x7ff1041609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.670+0000 7ff10acd1700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10415d060 con 0x7ff104071040 2023-11-28T11:09:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.670+0000 7ff10acd1700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10415d340 con 0x7ff10406dee0 2023-11-28T11:09:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.670+0000 7ff10acd1700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10415d620 con 0x7ff104160540 2023-11-28T11:09:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff103fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff10406dee0 0x7ff10406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff108a6d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff104071040 0x7ff104071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49058/0 (socket says 172.21.15.156:49058) 2023-11-28T11:09:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff103fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff10406dee0 0x7ff10406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34182/0 (socket says 172.21.15.156:34182) 2023-11-28T11:09:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff108a6d700 1 -- 172.21.15.156:0/3714180992 learned_addr learned my addr 172.21.15.156:0/3714180992 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff10926e700 1 --2- 172.21.15.156:0/3714180992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff104160540 0x7ff1041609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff103fff700 1 -- 172.21.15.156:0/3714180992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff104160540 msgr2=0x7ff1041609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff103fff700 1 --2- 172.21.15.156:0/3714180992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff104160540 0x7ff1041609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff103fff700 1 -- 172.21.15.156:0/3714180992 >> v1:172.21.15.156:6789/0 conn(0x7ff104071040 legacy=0x7ff104071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff103fff700 1 -- 172.21.15.156:0/3714180992 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10415d900 con 0x7ff10406dee0 2023-11-28T11:09:39.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff1037fe700 1 -- 172.21.15.156:0/3714180992 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173516545 0 0) 0x7ff10415d060 con 0x7ff104071040 2023-11-28T11:09:39.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff103fff700 1 --2- 172.21.15.156:0/3714180992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff10406dee0 0x7ff10406e330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009840 tx=0x7ff0f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b89230682976bc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff1037fe700 1 -- 172.21.15.156:0/3714180992 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0011830 con 0x7ff10406dee0 2023-11-28T11:09:39.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff1037fe700 1 -- 172.21.15.156:0/3714180992 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f0011990 con 0x7ff10406dee0 2023-11-28T11:09:39.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.671+0000 7ff1037fe700 1 -- 172.21.15.156:0/3714180992 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0011d20 con 0x7ff10406dee0 2023-11-28T11:09:39.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.672+0000 7ff10acd1700 1 -- 172.21.15.156:0/3714180992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff10406dee0 msgr2=0x7ff10406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.672+0000 7ff10acd1700 1 --2- 172.21.15.156:0/3714180992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff10406dee0 0x7ff10406e330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009840 tx=0x7ff0f0009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:39.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.672+0000 7ff10acd1700 1 -- 172.21.15.156:0/3714180992 shutdown_connections 2023-11-28T11:09:39.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.672+0000 7ff10acd1700 1 --2- 172.21.15.156:0/3714180992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff104160540 0x7ff1041609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.672+0000 7ff10acd1700 1 --2- 172.21.15.156:0/3714180992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff10406dee0 0x7ff10406e330 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.672+0000 7ff10acd1700 1 -- 172.21.15.156:0/3714180992 >> 172.21.15.156:0/3714180992 conn(0x7ff1040d0000 msgr2=0x7ff1040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.672+0000 7ff10acd1700 1 -- 172.21.15.156:0/3714180992 shutdown_connections 2023-11-28T11:09:39.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.672+0000 7ff10acd1700 1 -- 172.21.15.156:0/3714180992 wait complete. 2023-11-28T11:09:39.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.673+0000 7ff10acd1700 1 Processor -- start 2023-11-28T11:09:39.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.673+0000 7ff10acd1700 1 -- start start 2023-11-28T11:09:39.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff10acd1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff10406dee0 0x7ff10415e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff10acd1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff104071040 0x7ff1040b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff10acd1700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff104160540 0x7ff1040b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff10acd1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040b5ee0 con 0x7ff10406dee0 2023-11-28T11:09:39.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff10acd1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1041f6b50 con 0x7ff104071040 2023-11-28T11:09:39.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff10acd1700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040c4b90 con 0x7ff104160540 2023-11-28T11:09:39.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff108a6d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff10406dee0 0x7ff10415e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff108a6d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff10406dee0 0x7ff10415e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34186/0 (socket says 172.21.15.156:34186) 2023-11-28T11:09:39.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff108a6d700 1 -- 172.21.15.156:0/2326936542 learned_addr learned my addr 172.21.15.156:0/2326936542 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:39.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff103fff700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff104071040 0x7ff1040b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff10926e700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff104160540 0x7ff1040b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff108a6d700 1 -- 172.21.15.156:0/2326936542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff104160540 msgr2=0x7ff1040b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff108a6d700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff104160540 0x7ff1040b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff108a6d700 1 -- 172.21.15.156:0/2326936542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff104071040 msgr2=0x7ff1040b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff108a6d700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff104071040 0x7ff1040b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff108a6d700 1 -- 172.21.15.156:0/2326936542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f00094a0 con 0x7ff10406dee0 2023-11-28T11:09:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.674+0000 7ff10926e700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff104160540 0x7ff1040b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.675+0000 7ff108a6d700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff10406dee0 0x7ff10415e360 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800c550 tx=0x7ff0f800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.675+0000 7ff1017fa700 1 -- 172.21.15.156:0/2326936542 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8004c80 con 0x7ff10406dee0 2023-11-28T11:09:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.675+0000 7ff1017fa700 1 -- 172.21.15.156:0/2326936542 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f8004e00 con 0x7ff10406dee0 2023-11-28T11:09:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.675+0000 7ff10acd1700 1 -- 172.21.15.156:0/2326936542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10415ccd0 con 0x7ff10406dee0 2023-11-28T11:09:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.675+0000 7ff1017fa700 1 -- 172.21.15.156:0/2326936542 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8007310 con 0x7ff10406dee0 2023-11-28T11:09:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.675+0000 7ff10acd1700 1 -- 172.21.15.156:0/2326936542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10406e870 con 0x7ff10406dee0 2023-11-28T11:09:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.676+0000 7ff10acd1700 1 -- 172.21.15.156:0/2326936542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10406a270 con 0x7ff10406dee0 2023-11-28T11:09:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.676+0000 7ff1017fa700 1 -- 172.21.15.156:0/2326936542 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff0f801a020 con 0x7ff10406dee0 2023-11-28T11:09:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.676+0000 7ff1017fa700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0e80682f0 0x7ff0e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.677+0000 7ff1017fa700 1 -- 172.21.15.156:0/2326936542 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff0f80857a0 con 0x7ff10406dee0 2023-11-28T11:09:39.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.677+0000 7ff103fff700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0e80682f0 0x7ff0e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.677+0000 7ff103fff700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0e80682f0 0x7ff0e806a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00090b0 tx=0x7ff0f001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.681+0000 7ff1017fa700 1 -- 172.21.15.156:0/2326936542 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff0f8052000 con 0x7ff10406dee0 2023-11-28T11:09:39.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.862+0000 7ff10acd1700 1 -- 172.21.15.156:0/2326936542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff10406f440 con 0x7ff10406dee0 2023-11-28T11:09:39.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.863+0000 7ff1017fa700 1 -- 172.21.15.156:0/2326936542 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff0f8055620 con 0x7ff10406dee0 2023-11-28T11:09:39.864 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:09:39.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.865+0000 7ff10acd1700 1 -- 172.21.15.156:0/2326936542 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0e80682f0 msgr2=0x7ff0e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.865+0000 7ff10acd1700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0e80682f0 0x7ff0e806a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00090b0 tx=0x7ff0f001c040 comp rx=0 tx=0).stop 2023-11-28T11:09:39.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.865+0000 7ff10acd1700 1 -- 172.21.15.156:0/2326936542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff10406dee0 msgr2=0x7ff10415e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.865+0000 7ff10acd1700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff10406dee0 0x7ff10415e360 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800c550 tx=0x7ff0f800c860 comp rx=0 tx=0).stop 2023-11-28T11:09:39.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.865+0000 7ff10acd1700 1 -- 172.21.15.156:0/2326936542 shutdown_connections 2023-11-28T11:09:39.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.865+0000 7ff10acd1700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0e80682f0 0x7ff0e806a7a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.865+0000 7ff10acd1700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff104160540 0x7ff1040b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.865+0000 7ff10acd1700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff10406dee0 0x7ff10415e360 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.865+0000 7ff10acd1700 1 --2- 172.21.15.156:0/2326936542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff104071040 0x7ff1040b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.865+0000 7ff10acd1700 1 -- 172.21.15.156:0/2326936542 >> 172.21.15.156:0/2326936542 conn(0x7ff1040d0000 msgr2=0x7ff1040ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.865+0000 7ff10acd1700 1 -- 172.21.15.156:0/2326936542 shutdown_connections 2023-11-28T11:09:39.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:39.865+0000 7ff10acd1700 1 -- 172.21.15.156:0/2326936542 wait complete. 2023-11-28T11:09:39.878 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:40.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.001+0000 7f659553c700 1 Processor -- start 2023-11-28T11:09:40.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.001+0000 7f659553c700 1 -- start start 2023-11-28T11:09:40.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f659553c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f659015ca50 0x7f659006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f659553c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f65900cce10 0x7f65900cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f659553c700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65900cf5e0 con 0x7f659006b8c0 2023-11-28T11:09:40.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f659553c700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659006bd60 con 0x7f65900cce10 2023-11-28T11:09:40.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f659553c700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65900cd470 con 0x7f659015ca50 2023-11-28T11:09:40.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f658f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f65900cce10 0x7f65900cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f658f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f65900cce10 0x7f65900cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34198/0 (socket says 172.21.15.156:34198) 2023-11-28T11:09:40.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f658f7fe700 1 -- 172.21.15.156:0/2967747037 learned_addr learned my addr 172.21.15.156:0/2967747037 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:40.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f658effd700 1 --2- 172.21.15.156:0/2967747037 >> v2:172.21.15.186:3300/0 conn(0x7f659015ca50 0x7f659006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f658f7fe700 1 -- 172.21.15.156:0/2967747037 >> v2:172.21.15.186:3300/0 conn(0x7f659015ca50 msgr2=0x7f659006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f658f7fe700 1 --2- 172.21.15.156:0/2967747037 >> v2:172.21.15.186:3300/0 conn(0x7f659015ca50 0x7f659006b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f658f7fe700 1 -- 172.21.15.156:0/2967747037 >> v1:172.21.15.186:6789/0 conn(0x7f659006b8c0 legacy=0x7f65900cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f658f7fe700 1 -- 172.21.15.156:0/2967747037 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65900cecb0 con 0x7f65900cce10 2023-11-28T11:09:40.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f658effd700 1 --2- 172.21.15.156:0/2967747037 >> v2:172.21.15.186:3300/0 conn(0x7f659015ca50 0x7f659006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:40.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f658f7fe700 1 --2- 172.21.15.156:0/2967747037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65900cce10 0x7f65900cfa70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6590071240 tx=0x7f658800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3bc0efdb015989 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.002+0000 7f658dffb700 1 -- 172.21.15.156:0/2967747037 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658800e070 con 0x7f65900cce10 2023-11-28T11:09:40.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.003+0000 7f658dffb700 1 -- 172.21.15.156:0/2967747037 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6588003a20 con 0x7f65900cce10 2023-11-28T11:09:40.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.003+0000 7f658dffb700 1 -- 172.21.15.156:0/2967747037 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588003dd0 con 0x7f65900cce10 2023-11-28T11:09:40.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.003+0000 7f659553c700 1 -- 172.21.15.156:0/2967747037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65900cce10 msgr2=0x7f65900cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.003+0000 7f659553c700 1 --2- 172.21.15.156:0/2967747037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65900cce10 0x7f65900cfa70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6590071240 tx=0x7f658800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:40.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.003+0000 7f659553c700 1 -- 172.21.15.156:0/2967747037 shutdown_connections 2023-11-28T11:09:40.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.003+0000 7f659553c700 1 --2- 172.21.15.156:0/2967747037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65900cce10 0x7f65900cfa70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.003+0000 7f659553c700 1 --2- 172.21.15.156:0/2967747037 >> v2:172.21.15.186:3300/0 conn(0x7f659015ca50 0x7f659006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.003+0000 7f659553c700 1 -- 172.21.15.156:0/2967747037 >> 172.21.15.156:0/2967747037 conn(0x7f659009a230 msgr2=0x7f659008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.003+0000 7f659553c700 1 -- 172.21.15.156:0/2967747037 shutdown_connections 2023-11-28T11:09:40.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.003+0000 7f659553c700 1 -- 172.21.15.156:0/2967747037 wait complete. 2023-11-28T11:09:40.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.004+0000 7f659553c700 1 Processor -- start 2023-11-28T11:09:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.004+0000 7f659553c700 1 -- start start 2023-11-28T11:09:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.005+0000 7f659553c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f659006b8c0 0x7f659015bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.005+0000 7f659553c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65900cce10 0x7f6590156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.005+0000 7f659553c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f659015ca50 0x7f65900b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.005+0000 7f659553c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659015b8c0 con 0x7f65900cce10 2023-11-28T11:09:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.005+0000 7f659553c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659015ac40 con 0x7f659015ca50 2023-11-28T11:09:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.005+0000 7f659553c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659015af20 con 0x7f659006b8c0 2023-11-28T11:09:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.005+0000 7f658e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65900cce10 0x7f6590156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.005+0000 7f658e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65900cce10 0x7f6590156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34202/0 (socket says 172.21.15.156:34202) 2023-11-28T11:09:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.005+0000 7f658e7fc700 1 -- 172.21.15.156:0/1233792823 learned_addr learned my addr 172.21.15.156:0/1233792823 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.005+0000 7f658f7fe700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f659015ca50 0x7f65900b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.005+0000 7f658effd700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f659006b8c0 0x7f659015bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.006+0000 7f658e7fc700 1 -- 172.21.15.156:0/1233792823 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f659006b8c0 msgr2=0x7f659015bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.006+0000 7f658e7fc700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f659006b8c0 0x7f659015bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.006+0000 7f658e7fc700 1 -- 172.21.15.156:0/1233792823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f659015ca50 msgr2=0x7f65900b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.006+0000 7f658e7fc700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f659015ca50 0x7f65900b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.006+0000 7f658e7fc700 1 -- 172.21.15.156:0/1233792823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6588009b30 con 0x7f65900cce10 2023-11-28T11:09:40.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.006+0000 7f658e7fc700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65900cce10 0x7f6590156670 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f658000c4d0 tx=0x7f658000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.006+0000 7f65777fe700 1 -- 172.21.15.156:0/1233792823 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580011790 con 0x7f65900cce10 2023-11-28T11:09:40.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.006+0000 7f65777fe700 1 -- 172.21.15.156:0/1233792823 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65800118f0 con 0x7f65900cce10 2023-11-28T11:09:40.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.006+0000 7f659553c700 1 -- 172.21.15.156:0/1233792823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65900cf270 con 0x7f65900cce10 2023-11-28T11:09:40.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.006+0000 7f65777fe700 1 -- 172.21.15.156:0/1233792823 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580011ca0 con 0x7f65900cce10 2023-11-28T11:09:40.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.006+0000 7f659553c700 1 -- 172.21.15.156:0/1233792823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65900cef90 con 0x7f65900cce10 2023-11-28T11:09:40.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.007+0000 7f659553c700 1 -- 172.21.15.156:0/1233792823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659015ce20 con 0x7f65900cce10 2023-11-28T11:09:40.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.007+0000 7f65777fe700 1 -- 172.21.15.156:0/1233792823 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6580010050 con 0x7f65900cce10 2023-11-28T11:09:40.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.008+0000 7f65777fe700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f657c0682f0 0x7f657c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.008+0000 7f658effd700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f657c0682f0 0x7f657c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.008+0000 7f65777fe700 1 -- 172.21.15.156:0/1233792823 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6580085870 con 0x7f65900cce10 2023-11-28T11:09:40.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.008+0000 7f658effd700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f657c0682f0 0x7f657c06a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6584006fd0 tx=0x7f6584008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.012+0000 7f65777fe700 1 -- 172.21.15.156:0/1233792823 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f65800520d0 con 0x7f65900cce10 2023-11-28T11:09:40.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.196+0000 7f659553c700 1 -- 172.21.15.156:0/1233792823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65900c57b0 con 0x7f65900cce10 2023-11-28T11:09:40.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.196+0000 7f65777fe700 1 -- 172.21.15.156:0/1233792823 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65800556f0 con 0x7f65900cce10 2023-11-28T11:09:40.197 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:09:40.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.198+0000 7f659553c700 1 -- 172.21.15.156:0/1233792823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f657c0682f0 msgr2=0x7f657c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.198+0000 7f659553c700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f657c0682f0 0x7f657c06a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6584006fd0 tx=0x7f6584008040 comp rx=0 tx=0).stop 2023-11-28T11:09:40.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.198+0000 7f659553c700 1 -- 172.21.15.156:0/1233792823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65900cce10 msgr2=0x7f6590156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.198+0000 7f659553c700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65900cce10 0x7f6590156670 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f658000c4d0 tx=0x7f658000c7e0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.198+0000 7f659553c700 1 -- 172.21.15.156:0/1233792823 shutdown_connections 2023-11-28T11:09:40.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.198+0000 7f659553c700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f659015ca50 0x7f65900b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.198+0000 7f659553c700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f657c0682f0 0x7f657c06a7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.198+0000 7f659553c700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f659006b8c0 0x7f659015bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.198+0000 7f659553c700 1 --2- 172.21.15.156:0/1233792823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65900cce10 0x7f6590156670 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.198+0000 7f659553c700 1 -- 172.21.15.156:0/1233792823 >> 172.21.15.156:0/1233792823 conn(0x7f659009a230 msgr2=0x7f65900d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.198+0000 7f659553c700 1 -- 172.21.15.156:0/1233792823 shutdown_connections 2023-11-28T11:09:40.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.198+0000 7f659553c700 1 -- 172.21.15.156:0/1233792823 wait complete. 2023-11-28T11:09:40.212 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:40.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.334+0000 7f857f9f7700 1 Processor -- start 2023-11-28T11:09:40.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.334+0000 7f857f9f7700 1 -- start start 2023-11-28T11:09:40.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857f9f7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f857815cef0 0x7f85780af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857f9f7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f857806ba90 0x7f85780ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857f9f7700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85780cc5f0 con 0x7f85780cbdf0 2023-11-28T11:09:40.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857f9f7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857815d470 con 0x7f857815cef0 2023-11-28T11:09:40.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857f9f7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857815d750 con 0x7f857806ba90 2023-11-28T11:09:40.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857d793700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f857815cef0 0x7f85780af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857d793700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f857815cef0 0x7f85780af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34208/0 (socket says 172.21.15.156:34208) 2023-11-28T11:09:40.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857d793700 1 -- 172.21.15.156:0/3318521765 learned_addr learned my addr 172.21.15.156:0/3318521765 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:40.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857cf92700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f85780cbdf0 0x7f857806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49086/0 (socket says 172.21.15.156:49086) 2023-11-28T11:09:40.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857df94700 1 --2- 172.21.15.156:0/3318521765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f857806ba90 0x7f85780ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857d793700 1 -- 172.21.15.156:0/3318521765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f857806ba90 msgr2=0x7f85780ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857d793700 1 --2- 172.21.15.156:0/3318521765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f857806ba90 0x7f85780ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857d793700 1 -- 172.21.15.156:0/3318521765 >> v1:172.21.15.156:6789/0 conn(0x7f85780cbdf0 legacy=0x7f857806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f856ffff700 1 -- 172.21.15.156:0/3318521765 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037230758 0 0) 0x7f85780cc5f0 con 0x7f85780cbdf0 2023-11-28T11:09:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857d793700 1 -- 172.21.15.156:0/3318521765 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857815da30 con 0x7f857815cef0 2023-11-28T11:09:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f857d793700 1 --2- 172.21.15.156:0/3318521765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f857815cef0 0x7f85780af400 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8574009920 tx=0x7f8574009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c34f1f01f1e1abc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f856ffff700 1 -- 172.21.15.156:0/3318521765 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857400ba20 con 0x7f857815cef0 2023-11-28T11:09:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f856ffff700 1 -- 172.21.15.156:0/3318521765 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857400bb80 con 0x7f857815cef0 2023-11-28T11:09:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.335+0000 7f856ffff700 1 -- 172.21.15.156:0/3318521765 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574009e70 con 0x7f857815cef0 2023-11-28T11:09:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.336+0000 7f857f9f7700 1 -- 172.21.15.156:0/3318521765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f857815cef0 msgr2=0x7f85780af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.336+0000 7f857f9f7700 1 --2- 172.21.15.156:0/3318521765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f857815cef0 0x7f85780af400 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8574009920 tx=0x7f8574009c30 comp rx=0 tx=0).stop 2023-11-28T11:09:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.336+0000 7f857f9f7700 1 -- 172.21.15.156:0/3318521765 shutdown_connections 2023-11-28T11:09:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.336+0000 7f857f9f7700 1 --2- 172.21.15.156:0/3318521765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f857806ba90 0x7f85780ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.336+0000 7f857f9f7700 1 --2- 172.21.15.156:0/3318521765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f857815cef0 0x7f85780af400 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.336+0000 7f857f9f7700 1 -- 172.21.15.156:0/3318521765 >> 172.21.15.156:0/3318521765 conn(0x7f85780d0000 msgr2=0x7f85780d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.336+0000 7f857f9f7700 1 -- 172.21.15.156:0/3318521765 shutdown_connections 2023-11-28T11:09:40.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.336+0000 7f857f9f7700 1 -- 172.21.15.156:0/3318521765 wait complete. 2023-11-28T11:09:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.337+0000 7f857f9f7700 1 Processor -- start 2023-11-28T11:09:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.337+0000 7f857f9f7700 1 -- start start 2023-11-28T11:09:40.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857f9f7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f857806ba90 0x7f85781f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857f9f7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85780cbdf0 0x7f85780b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857f9f7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f857815cef0 0x7f85780ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857f9f7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8578099bb0 con 0x7f857806ba90 2023-11-28T11:09:40.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857f9f7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85780b4fb0 con 0x7f85780cbdf0 2023-11-28T11:09:40.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857f9f7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85780c7170 con 0x7f857815cef0 2023-11-28T11:09:40.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857d793700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f857806ba90 0x7f85781f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857d793700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f857806ba90 0x7f85781f2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34214/0 (socket says 172.21.15.156:34214) 2023-11-28T11:09:40.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857d793700 1 -- 172.21.15.156:0/2453278439 learned_addr learned my addr 172.21.15.156:0/2453278439 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:40.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857cf92700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85780cbdf0 0x7f85780b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857df94700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f857815cef0 0x7f85780ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857d793700 1 -- 172.21.15.156:0/2453278439 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f857815cef0 msgr2=0x7f85780ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857d793700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f857815cef0 0x7f85780ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857d793700 1 -- 172.21.15.156:0/2453278439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85780cbdf0 msgr2=0x7f85780b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857d793700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85780cbdf0 0x7f85780b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857d793700 1 -- 172.21.15.156:0/2453278439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8574009550 con 0x7f857806ba90 2023-11-28T11:09:40.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.338+0000 7f857df94700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f857815cef0 0x7f85780ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:40.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.339+0000 7f857d793700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f857806ba90 0x7f85781f2310 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8574000c00 tx=0x7f8574004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.339+0000 7f856dffb700 1 -- 172.21.15.156:0/2453278439 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574004590 con 0x7f857806ba90 2023-11-28T11:09:40.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.339+0000 7f856dffb700 1 -- 172.21.15.156:0/2453278439 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857401e070 con 0x7f857806ba90 2023-11-28T11:09:40.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.339+0000 7f857f9f7700 1 -- 172.21.15.156:0/2453278439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857815dff0 con 0x7f857806ba90 2023-11-28T11:09:40.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.339+0000 7f856dffb700 1 -- 172.21.15.156:0/2453278439 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85740038a0 con 0x7f857806ba90 2023-11-28T11:09:40.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.339+0000 7f857f9f7700 1 -- 172.21.15.156:0/2453278439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857815dd10 con 0x7f857806ba90 2023-11-28T11:09:40.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.340+0000 7f857f9f7700 1 -- 172.21.15.156:0/2453278439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85780cab90 con 0x7f857806ba90 2023-11-28T11:09:40.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.340+0000 7f856dffb700 1 -- 172.21.15.156:0/2453278439 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f857400f050 con 0x7f857806ba90 2023-11-28T11:09:40.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.341+0000 7f856dffb700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f855c0682f0 0x7f855c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.341+0000 7f856dffb700 1 -- 172.21.15.156:0/2453278439 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8574020070 con 0x7f857806ba90 2023-11-28T11:09:40.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.341+0000 7f857cf92700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f855c0682f0 0x7f855c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.341+0000 7f857cf92700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f855c0682f0 0x7f855c06a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f85781f2850 tx=0x7f8568007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.345+0000 7f856dffb700 1 -- 172.21.15.156:0/2453278439 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f85740b85e0 con 0x7f857806ba90 2023-11-28T11:09:40.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.526+0000 7f857f9f7700 1 -- 172.21.15.156:0/2453278439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f85780d0e60 con 0x7f857806ba90 2023-11-28T11:09:40.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.527+0000 7f856dffb700 1 -- 172.21.15.156:0/2453278439 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8574053970 con 0x7f857806ba90 2023-11-28T11:09:40.528 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:09:40.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.529+0000 7f857f9f7700 1 -- 172.21.15.156:0/2453278439 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f855c0682f0 msgr2=0x7f855c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.529+0000 7f857f9f7700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f855c0682f0 0x7f855c06a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f85781f2850 tx=0x7f8568007de0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.529+0000 7f857f9f7700 1 -- 172.21.15.156:0/2453278439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f857806ba90 msgr2=0x7f85781f2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.529+0000 7f857f9f7700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f857806ba90 0x7f85781f2310 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8574000c00 tx=0x7f8574004750 comp rx=0 tx=0).stop 2023-11-28T11:09:40.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.529+0000 7f857f9f7700 1 -- 172.21.15.156:0/2453278439 shutdown_connections 2023-11-28T11:09:40.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.529+0000 7f857f9f7700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f855c0682f0 0x7f855c06a7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.529+0000 7f857f9f7700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f857815cef0 0x7f85780ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.529+0000 7f857f9f7700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f857806ba90 0x7f85781f2310 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.529+0000 7f857f9f7700 1 --2- 172.21.15.156:0/2453278439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85780cbdf0 0x7f85780b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.529+0000 7f857f9f7700 1 -- 172.21.15.156:0/2453278439 >> 172.21.15.156:0/2453278439 conn(0x7f85780d0000 msgr2=0x7f85780caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.529+0000 7f857f9f7700 1 -- 172.21.15.156:0/2453278439 shutdown_connections 2023-11-28T11:09:40.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.529+0000 7f857f9f7700 1 -- 172.21.15.156:0/2453278439 wait complete. 2023-11-28T11:09:40.543 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:40.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.661+0000 7f887fb8a700 1 Processor -- start 2023-11-28T11:09:40.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.662+0000 7f887fb8a700 1 -- start start 2023-11-28T11:09:40.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.662+0000 7f887fb8a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f887815cef0 0x7f88780af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.662+0000 7f887fb8a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88780cc5f0 con 0x7f887806ba90 2023-11-28T11:09:40.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.662+0000 7f887fb8a700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887815d470 con 0x7f88780cbdf0 2023-11-28T11:09:40.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.662+0000 7f887fb8a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887815d750 con 0x7f887815cef0 2023-11-28T11:09:40.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887e127700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f887806ba90 0x7f88780ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49104/0 (socket says 172.21.15.156:49104) 2023-11-28T11:09:40.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887e127700 1 -- 172.21.15.156:0/1821137934 learned_addr learned my addr 172.21.15.156:0/1821137934 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:40.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426244953 0 0) 0x7f88780cc5f0 con 0x7f887806ba90 2023-11-28T11:09:40.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88640035b0 con 0x7f887806ba90 2023-11-28T11:09:40.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887d926700 1 --2- 172.21.15.156:0/1821137934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f887815cef0 0x7f88780af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2301993243 0 0) 0x7f88640035b0 con 0x7f887806ba90 2023-11-28T11:09:40.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f887815cef0 msgr2=0x7f88780af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 --2- 172.21.15.156:0/1821137934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f887815cef0 0x7f88780af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 >> v1:172.21.15.186:6789/0 conn(0x7f88780cbdf0 legacy=0x7f887806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887815da30 con 0x7f887806ba90 2023-11-28T11:09:40.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f88700019f0 con 0x7f887806ba90 2023-11-28T11:09:40.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 >> v1:172.21.15.156:6789/0 conn(0x7f887806ba90 legacy=0x7f88780ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 --2- 172.21.15.156:0/1821137934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f887815cef0 0x7f8864003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 --2- 172.21.15.156:0/1821137934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8864003f70 0x7f8864004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 --2- 172.21.15.156:0/1821137934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8864004d90 0x7f8864005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887815d750 con 0x7f8864003f70 2023-11-28T11:09:40.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886400ba40 con 0x7f8864004d90 2023-11-28T11:09:40.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886400bc00 con 0x7f887815cef0 2023-11-28T11:09:40.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887d125700 1 --2- 172.21.15.156:0/1821137934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8864003f70 0x7f8864004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887d926700 1 --2- 172.21.15.156:0/1821137934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f887815cef0 0x7f8864003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.663+0000 7f887e127700 1 --2- 172.21.15.156:0/1821137934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8864004d90 0x7f8864005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887d125700 1 -- 172.21.15.156:0/1821137934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f887815cef0 msgr2=0x7f8864003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887d125700 1 --2- 172.21.15.156:0/1821137934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f887815cef0 0x7f8864003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887d125700 1 -- 172.21.15.156:0/1821137934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8864004d90 msgr2=0x7f8864005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887d125700 1 --2- 172.21.15.156:0/1821137934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8864004d90 0x7f8864005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887d125700 1 -- 172.21.15.156:0/1821137934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88640035b0 con 0x7f8864003f70 2023-11-28T11:09:40.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887d125700 1 --2- 172.21.15.156:0/1821137934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8864003f70 0x7f8864004850 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f88740018d0 tx=0x7f8864005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9382f89ca952daa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400c690 con 0x7f8864003f70 2023-11-28T11:09:40.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887400c7f0 con 0x7f8864003f70 2023-11-28T11:09:40.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887c924700 1 -- 172.21.15.156:0/1821137934 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400cba0 con 0x7f8864003f70 2023-11-28T11:09:40.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887fb8a700 1 -- 172.21.15.156:0/1821137934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8864003f70 msgr2=0x7f8864004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887fb8a700 1 --2- 172.21.15.156:0/1821137934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8864003f70 0x7f8864004850 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f88740018d0 tx=0x7f8864005880 comp rx=0 tx=0).stop 2023-11-28T11:09:40.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887e127700 1 -- 172.21.15.156:0/1821137934 reap_dead start 2023-11-28T11:09:40.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887fb8a700 1 -- 172.21.15.156:0/1821137934 shutdown_connections 2023-11-28T11:09:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.664+0000 7f887fb8a700 1 -- 172.21.15.156:0/1821137934 >> 172.21.15.156:0/1821137934 conn(0x7f88780d0000 msgr2=0x7f88780d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.665+0000 7f887fb8a700 1 -- 172.21.15.156:0/1821137934 shutdown_connections 2023-11-28T11:09:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.665+0000 7f887fb8a700 1 -- 172.21.15.156:0/1821137934 wait complete. 2023-11-28T11:09:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.666+0000 7f887fb8a700 1 Processor -- start 2023-11-28T11:09:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.666+0000 7f887fb8a700 1 -- start start 2023-11-28T11:09:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.666+0000 7f887fb8a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8864003f70 0x7f887814fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.666+0000 7f887fb8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f887813eaa0 0x7f887813cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.666+0000 7f887fb8a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f887813d490 0x7f88781544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.666+0000 7f887fb8a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88780cb0f0 con 0x7f887813eaa0 2023-11-28T11:09:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.666+0000 7f887fb8a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887813da00 con 0x7f887813d490 2023-11-28T11:09:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.666+0000 7f887fb8a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887813dce0 con 0x7f8864003f70 2023-11-28T11:09:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887d125700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f887813eaa0 0x7f887813cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887d125700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f887813eaa0 0x7f887813cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34236/0 (socket says 172.21.15.156:34236) 2023-11-28T11:09:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887d125700 1 -- 172.21.15.156:0/2297826004 learned_addr learned my addr 172.21.15.156:0/2297826004 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887e127700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f887813d490 0x7f88781544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887d926700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8864003f70 0x7f887814fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887d125700 1 -- 172.21.15.156:0/2297826004 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8864003f70 msgr2=0x7f887814fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887d125700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8864003f70 0x7f887814fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887d125700 1 -- 172.21.15.156:0/2297826004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f887813d490 msgr2=0x7f88781544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887d125700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f887813d490 0x7f88781544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887d125700 1 -- 172.21.15.156:0/2297826004 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887815dff0 con 0x7f887813eaa0 2023-11-28T11:09:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887d926700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8864003f70 0x7f887814fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887d125700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f887813eaa0 0x7f887813cf50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f886400ba10 tx=0x7f8874001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f886e7fc700 1 -- 172.21.15.156:0/2297826004 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400cb60 con 0x7f887813eaa0 2023-11-28T11:09:40.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f886e7fc700 1 -- 172.21.15.156:0/2297826004 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874021070 con 0x7f887813eaa0 2023-11-28T11:09:40.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887fb8a700 1 -- 172.21.15.156:0/2297826004 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887815dd10 con 0x7f887813eaa0 2023-11-28T11:09:40.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f886e7fc700 1 -- 172.21.15.156:0/2297826004 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874001330 con 0x7f887813eaa0 2023-11-28T11:09:40.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.667+0000 7f887fb8a700 1 -- 172.21.15.156:0/2297826004 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88780af400 con 0x7f887813eaa0 2023-11-28T11:09:40.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.668+0000 7f887fb8a700 1 -- 172.21.15.156:0/2297826004 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88780d0e60 con 0x7f887813eaa0 2023-11-28T11:09:40.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.669+0000 7f886e7fc700 1 -- 172.21.15.156:0/2297826004 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8874010050 con 0x7f887813eaa0 2023-11-28T11:09:40.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.669+0000 7f886e7fc700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f885c0682f0 0x7f885c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.669+0000 7f886e7fc700 1 -- 172.21.15.156:0/2297826004 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8874086e60 con 0x7f887813eaa0 2023-11-28T11:09:40.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.669+0000 7f887d926700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f885c0682f0 0x7f885c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.669+0000 7f887d926700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f885c0682f0 0x7f885c06a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f887813bd00 tx=0x7f8868008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.673+0000 7f886e7fc700 1 -- 172.21.15.156:0/2297826004 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f88740536c0 con 0x7f887813eaa0 2023-11-28T11:09:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.898+0000 7f887fb8a700 1 -- 172.21.15.156:0/2297826004 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88780cd620 con 0x7f887813eaa0 2023-11-28T11:09:40.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.899+0000 7f886e7fc700 1 -- 172.21.15.156:0/2297826004 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f8874056ce0 con 0x7f887813eaa0 2023-11-28T11:09:40.900 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:40.900 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1085440106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1085440106},{"type":"v1","addr":"172.21.15.186:6834","nonce":1085440106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1548245207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1548245207},{"type":"v1","addr":"172.21.15.156:6836","nonce":1548245207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4882,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3102077957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":3102077957},{"type":"v1","addr":"172.21.15.156:6837","nonce":3102077957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:15.391255+0000","modified":"2023-11-28T11:09:18.414744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3897746301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3897746301},{"type":"v1","addr":"172.21.15.186:6835","nonce":3897746301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:09:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.901+0000 7f887fb8a700 1 -- 172.21.15.156:0/2297826004 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f885c0682f0 msgr2=0x7f885c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.901+0000 7f887fb8a700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f885c0682f0 0x7f885c06a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f887813bd00 tx=0x7f8868008040 comp rx=0 tx=0).stop 2023-11-28T11:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.901+0000 7f887fb8a700 1 -- 172.21.15.156:0/2297826004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f887813eaa0 msgr2=0x7f887813cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.901+0000 7f887fb8a700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f887813eaa0 0x7f887813cf50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f886400ba10 tx=0x7f8874001230 comp rx=0 tx=0).stop 2023-11-28T11:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.901+0000 7f887fb8a700 1 -- 172.21.15.156:0/2297826004 shutdown_connections 2023-11-28T11:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.901+0000 7f887fb8a700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f887813d490 0x7f88781544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.901+0000 7f887fb8a700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f885c0682f0 0x7f885c06a7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.901+0000 7f887fb8a700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8864003f70 0x7f887814fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.901+0000 7f887fb8a700 1 --2- 172.21.15.156:0/2297826004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f887813eaa0 0x7f887813cf50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.901+0000 7f887fb8a700 1 -- 172.21.15.156:0/2297826004 >> 172.21.15.156:0/2297826004 conn(0x7f88780d0000 msgr2=0x7f88780afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.901+0000 7f887fb8a700 1 -- 172.21.15.156:0/2297826004 shutdown_connections 2023-11-28T11:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:40.901+0000 7f887fb8a700 1 -- 172.21.15.156:0/2297826004 wait complete. 2023-11-28T11:09:40.904 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T11:09:40.914 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:41.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.033+0000 7f36948f4700 1 Processor -- start 2023-11-28T11:09:41.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.033+0000 7f36948f4700 1 -- start start 2023-11-28T11:09:41.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.033+0000 7f36948f4700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369006ba90 0x7f36900af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f36948f4700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f36900cbdf0 0x7f36900ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f36948f4700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36900cc5f0 con 0x7f369015cef0 2023-11-28T11:09:41.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f36948f4700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369015d470 con 0x7f36900cbdf0 2023-11-28T11:09:41.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f36948f4700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369015d750 con 0x7f369006ba90 2023-11-28T11:09:41.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f368ed9d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f36900cbdf0 0x7f36900ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f368dd9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369006ba90 0x7f36900af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f368ed9d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f36900cbdf0 0x7f36900ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:45260/0 (socket says 172.21.15.156:45260) 2023-11-28T11:09:41.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f368ed9d700 1 -- 172.21.15.156:0/1479511338 learned_addr learned my addr 172.21.15.156:0/1479511338 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:41.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f368dd9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369006ba90 0x7f36900af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:38108/0 (socket says 172.21.15.156:38108) 2023-11-28T11:09:41.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f368e59c700 1 --1- 172.21.15.156:0/1479511338 >> v1:172.21.15.186:6789/0 conn(0x7f369015cef0 0x7f369006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:59480/0 (socket says 172.21.15.156:59480) 2023-11-28T11:09:41.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f368ed9d700 1 -- 172.21.15.156:0/1479511338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369006ba90 msgr2=0x7f36900af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:41.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f368ed9d700 1 --2- 172.21.15.156:0/1479511338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369006ba90 0x7f36900af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f368ed9d700 1 -- 172.21.15.156:0/1479511338 >> v1:172.21.15.186:6789/0 conn(0x7f369015cef0 legacy=0x7f369006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f368ed9d700 1 -- 172.21.15.156:0/1479511338 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369015da30 con 0x7f36900cbdf0 2023-11-28T11:09:41.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.034+0000 7f368dd9b700 1 --2- 172.21.15.156:0/1479511338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369006ba90 0x7f36900af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:41.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.035+0000 7f368ed9d700 1 --2- 172.21.15.156:0/1479511338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36900cbdf0 0x7f36900ca650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3688009e50 tx=0x7f368800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96af365cb779b3fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.035+0000 7f368d59a700 1 -- 172.21.15.156:0/1479511338 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368800e070 con 0x7f36900cbdf0 2023-11-28T11:09:41.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.035+0000 7f368d59a700 1 -- 172.21.15.156:0/1479511338 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368800b910 con 0x7f36900cbdf0 2023-11-28T11:09:41.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.035+0000 7f368d59a700 1 -- 172.21.15.156:0/1479511338 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368800bcc0 con 0x7f36900cbdf0 2023-11-28T11:09:41.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.035+0000 7f36948f4700 1 -- 172.21.15.156:0/1479511338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36900cbdf0 msgr2=0x7f36900ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.035+0000 7f36948f4700 1 --2- 172.21.15.156:0/1479511338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36900cbdf0 0x7f36900ca650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3688009e50 tx=0x7f368800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:41.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.035+0000 7f36948f4700 1 -- 172.21.15.156:0/1479511338 shutdown_connections 2023-11-28T11:09:41.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.035+0000 7f36948f4700 1 --2- 172.21.15.156:0/1479511338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36900cbdf0 0x7f36900ca650 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.035+0000 7f36948f4700 1 --2- 172.21.15.156:0/1479511338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369006ba90 0x7f36900af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.035+0000 7f36948f4700 1 -- 172.21.15.156:0/1479511338 >> 172.21.15.156:0/1479511338 conn(0x7f36900d0000 msgr2=0x7f36900d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.035+0000 7f36948f4700 1 -- 172.21.15.156:0/1479511338 shutdown_connections 2023-11-28T11:09:41.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.035+0000 7f36948f4700 1 -- 172.21.15.156:0/1479511338 wait complete. 2023-11-28T11:09:41.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.037+0000 7f36948f4700 1 Processor -- start 2023-11-28T11:09:41.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.037+0000 7f36948f4700 1 -- start start 2023-11-28T11:09:41.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.037+0000 7f36948f4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f369006ba90 0x7f3690155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.037+0000 7f36948f4700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36900cbdf0 0x7f369015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.037+0000 7f36948f4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f369015cef0 0x7f369015a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.037+0000 7f36948f4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36900b4f20 con 0x7f369015cef0 2023-11-28T11:09:41.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f36948f4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36900b5200 con 0x7f369006ba90 2023-11-28T11:09:41.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f36948f4700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36900b54e0 con 0x7f36900cbdf0 2023-11-28T11:09:41.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f368ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f369015cef0 0x7f369015a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f368ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f369015cef0 0x7f369015a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34252/0 (socket says 172.21.15.156:34252) 2023-11-28T11:09:41.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f368ed9d700 1 -- 172.21.15.156:0/3881578462 learned_addr learned my addr 172.21.15.156:0/3881578462 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:41.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f368e59c700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f369006ba90 0x7f3690155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f368dd9b700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36900cbdf0 0x7f369015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f368ed9d700 1 -- 172.21.15.156:0/3881578462 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36900cbdf0 msgr2=0x7f369015a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f368ed9d700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36900cbdf0 0x7f369015a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f368ed9d700 1 -- 172.21.15.156:0/3881578462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f369006ba90 msgr2=0x7f3690155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f368ed9d700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f369006ba90 0x7f3690155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f368ed9d700 1 -- 172.21.15.156:0/3881578462 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800094a0 con 0x7f369015cef0 2023-11-28T11:09:41.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f368dd9b700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36900cbdf0 0x7f369015a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:41.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f368ed9d700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f369015cef0 0x7f369015a820 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f369006c7b0 tx=0x7f3688004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f367effd700 1 -- 172.21.15.156:0/3881578462 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688004870 con 0x7f369015cef0 2023-11-28T11:09:41.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f367effd700 1 -- 172.21.15.156:0/3881578462 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36880099b0 con 0x7f369015cef0 2023-11-28T11:09:41.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f36948f4700 1 -- 172.21.15.156:0/3881578462 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3688009b30 con 0x7f369015cef0 2023-11-28T11:09:41.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f367effd700 1 -- 172.21.15.156:0/3881578462 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368801c680 con 0x7f369015cef0 2023-11-28T11:09:41.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.038+0000 7f36948f4700 1 -- 172.21.15.156:0/3881578462 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369015dff0 con 0x7f369015cef0 2023-11-28T11:09:41.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.039+0000 7f36948f4700 1 -- 172.21.15.156:0/3881578462 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369015dd10 con 0x7f369015cef0 2023-11-28T11:09:41.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.040+0000 7f367effd700 1 -- 172.21.15.156:0/3881578462 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f368802c070 con 0x7f369015cef0 2023-11-28T11:09:41.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.040+0000 7f367effd700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f36700682f0 0x7f367006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.040+0000 7f367effd700 1 -- 172.21.15.156:0/3881578462 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3688087870 con 0x7f369015cef0 2023-11-28T11:09:41.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.040+0000 7f368e59c700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f36700682f0 0x7f367006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.040+0000 7f368e59c700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f36700682f0 0x7f367006a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3684000f60 tx=0x7f3684007ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.044+0000 7f367effd700 1 -- 172.21.15.156:0/3881578462 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f36880540d0 con 0x7f369015cef0 2023-11-28T11:09:41.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.233+0000 7f36948f4700 1 -- 172.21.15.156:0/3881578462 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36900cab90 con 0x7f369015cef0 2023-11-28T11:09:41.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.235+0000 7f367effd700 1 -- 172.21.15.156:0/3881578462 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f36880576f0 con 0x7f369015cef0 2023-11-28T11:09:41.236 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:41.236 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":37,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:16.393008+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:13.663041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:14.347248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:41.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.237+0000 7f36948f4700 1 -- 172.21.15.156:0/3881578462 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f36700682f0 msgr2=0x7f367006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.237+0000 7f36948f4700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f36700682f0 0x7f367006a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3684000f60 tx=0x7f3684007ed0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.237+0000 7f36948f4700 1 -- 172.21.15.156:0/3881578462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f369015cef0 msgr2=0x7f369015a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.237+0000 7f36948f4700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f369015cef0 0x7f369015a820 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f369006c7b0 tx=0x7f3688004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.237+0000 7f36948f4700 1 -- 172.21.15.156:0/3881578462 shutdown_connections 2023-11-28T11:09:41.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.237+0000 7f36948f4700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f369015cef0 0x7f369015a820 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.237+0000 7f36948f4700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f369006ba90 0x7f3690155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.237+0000 7f36948f4700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f36700682f0 0x7f367006a7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.237+0000 7f36948f4700 1 --2- 172.21.15.156:0/3881578462 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36900cbdf0 0x7f369015a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.237+0000 7f36948f4700 1 -- 172.21.15.156:0/3881578462 >> 172.21.15.156:0/3881578462 conn(0x7f36900d0000 msgr2=0x7f36900caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.237+0000 7f36948f4700 1 -- 172.21.15.156:0/3881578462 shutdown_connections 2023-11-28T11:09:41.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.237+0000 7f36948f4700 1 -- 172.21.15.156:0/3881578462 wait complete. 2023-11-28T11:09:41.250 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:09:41.251 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:41.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.371+0000 7fd9ae628700 1 Processor -- start 2023-11-28T11:09:41.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.371+0000 7fd9ae628700 1 -- start start 2023-11-28T11:09:41.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9ae628700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd9a80cd8c0 0x7fd9a8140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9ae628700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9a80cdc90 0x7fd9a80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9ae628700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a813ff00 con 0x7fd9a80b2e70 2023-11-28T11:09:41.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9ae628700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a81404c0 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9ae628700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a815aa80 con 0x7fd9a80cdc90 2023-11-28T11:09:41.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9a7fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd9a80cd8c0 0x7fd9a8140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9a77fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9a80cdc90 0x7fd9a80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9acbc5700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fd9a80b2e70 0x7fd9a806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:59498/0 (socket says 172.21.15.156:59498) 2023-11-28T11:09:41.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9a77fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9a80cdc90 0x7fd9a80b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:38122/0 (socket says 172.21.15.156:38122) 2023-11-28T11:09:41.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9a7fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd9a80cd8c0 0x7fd9a8140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:45270/0 (socket says 172.21.15.156:45270) 2023-11-28T11:09:41.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9acbc5700 1 -- 172.21.15.156:0/3330695765 learned_addr learned my addr 172.21.15.156:0/3330695765 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:41.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9a6ffd700 1 -- 172.21.15.156:0/3330695765 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681874903 0 0) 0x7fd9a813ff00 con 0x7fd9a80b2e70 2023-11-28T11:09:41.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9a77fe700 1 -- 172.21.15.156:0/3330695765 >> v2:172.21.15.186:3300/0 conn(0x7fd9a80cd8c0 msgr2=0x7fd9a8140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:41.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9a77fe700 1 --2- 172.21.15.156:0/3330695765 >> v2:172.21.15.186:3300/0 conn(0x7fd9a80cd8c0 0x7fd9a8140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9a77fe700 1 -- 172.21.15.156:0/3330695765 >> v1:172.21.15.186:6789/0 conn(0x7fd9a80b2e70 legacy=0x7fd9a806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9a77fe700 1 -- 172.21.15.156:0/3330695765 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a80b3310 con 0x7fd9a80cdc90 2023-11-28T11:09:41.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.372+0000 7fd9a7fff700 1 --2- 172.21.15.156:0/3330695765 >> v2:172.21.15.186:3300/0 conn(0x7fd9a80cd8c0 0x7fd9a8140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:41.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.373+0000 7fd9a77fe700 1 --2- 172.21.15.156:0/3330695765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9a80cdc90 0x7fd9a80b2930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd998009840 tx=0x7fd998009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7f99fbbaae8ce5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.373+0000 7fd9a6ffd700 1 -- 172.21.15.156:0/3330695765 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998011830 con 0x7fd9a80cdc90 2023-11-28T11:09:41.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.373+0000 7fd9a6ffd700 1 -- 172.21.15.156:0/3330695765 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd998011990 con 0x7fd9a80cdc90 2023-11-28T11:09:41.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.373+0000 7fd9a6ffd700 1 -- 172.21.15.156:0/3330695765 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998011d20 con 0x7fd9a80cdc90 2023-11-28T11:09:41.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.373+0000 7fd9ae628700 1 -- 172.21.15.156:0/3330695765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9a80cdc90 msgr2=0x7fd9a80b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.373+0000 7fd9ae628700 1 --2- 172.21.15.156:0/3330695765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9a80cdc90 0x7fd9a80b2930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd998009840 tx=0x7fd998009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:41.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.373+0000 7fd9ae628700 1 -- 172.21.15.156:0/3330695765 shutdown_connections 2023-11-28T11:09:41.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.373+0000 7fd9ae628700 1 --2- 172.21.15.156:0/3330695765 >> v2:172.21.15.186:3300/0 conn(0x7fd9a80cd8c0 0x7fd9a8140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.373+0000 7fd9ae628700 1 --2- 172.21.15.156:0/3330695765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9a80cdc90 0x7fd9a80b2930 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.373+0000 7fd9ae628700 1 -- 172.21.15.156:0/3330695765 >> 172.21.15.156:0/3330695765 conn(0x7fd9a806aed0 msgr2=0x7fd9a806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.373+0000 7fd9ae628700 1 -- 172.21.15.156:0/3330695765 shutdown_connections 2023-11-28T11:09:41.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.373+0000 7fd9ae628700 1 -- 172.21.15.156:0/3330695765 wait complete. 2023-11-28T11:09:41.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.375+0000 7fd9ae628700 1 Processor -- start 2023-11-28T11:09:41.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.375+0000 7fd9ae628700 1 -- start start 2023-11-28T11:09:41.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9ae628700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9a80b2e70 0x7fd9a8154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9ae628700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a80cd8c0 0x7fd9a813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9ae628700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9a80cdc90 0x7fd9a813d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9ae628700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a806b790 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9ae628700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81547c0 con 0x7fd9a80cdc90 2023-11-28T11:09:41.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9ae628700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80b67f0 con 0x7fd9a80b2e70 2023-11-28T11:09:41.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9a77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a80cd8c0 0x7fd9a813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9a77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a80cd8c0 0x7fd9a813ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34264/0 (socket says 172.21.15.156:34264) 2023-11-28T11:09:41.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9a77fe700 1 -- 172.21.15.156:0/2437924442 learned_addr learned my addr 172.21.15.156:0/2437924442 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:41.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9a7fff700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9a80b2e70 0x7fd9a8154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9acbc5700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9a80cdc90 0x7fd9a813d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9a77fe700 1 -- 172.21.15.156:0/2437924442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9a80b2e70 msgr2=0x7fd9a8154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9a77fe700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9a80b2e70 0x7fd9a8154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9a77fe700 1 -- 172.21.15.156:0/2437924442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9a80cdc90 msgr2=0x7fd9a813d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9a77fe700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9a80cdc90 0x7fd9a813d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.376+0000 7fd9a77fe700 1 -- 172.21.15.156:0/2437924442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c009430 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.377+0000 7fd9a77fe700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a80cd8c0 0x7fd9a813ccf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd998009fd0 tx=0x7fd998004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.377+0000 7fd9a4ff9700 1 -- 172.21.15.156:0/2437924442 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9980036a0 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.377+0000 7fd9a4ff9700 1 -- 172.21.15.156:0/2437924442 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99801e070 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.377+0000 7fd9ae628700 1 -- 172.21.15.156:0/2437924442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9980094a0 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.377+0000 7fd9a4ff9700 1 -- 172.21.15.156:0/2437924442 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998003a60 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.377+0000 7fd9ae628700 1 -- 172.21.15.156:0/2437924442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a8140150 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.378+0000 7fd9ae628700 1 -- 172.21.15.156:0/2437924442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a80b35f0 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.378+0000 7fd9a4ff9700 1 -- 172.21.15.156:0/2437924442 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd99800f050 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.378+0000 7fd9a4ff9700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd9940682f0 0x7fd99406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.379+0000 7fd9a4ff9700 1 -- 172.21.15.156:0/2437924442 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd998086bf0 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.379+0000 7fd9a7fff700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd9940682f0 0x7fd99406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.379+0000 7fd9a7fff700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd9940682f0 0x7fd99406a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009d50 tx=0x7fd99c0091c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.383+0000 7fd9a4ff9700 1 -- 172.21.15.156:0/2437924442 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd998053360 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.610+0000 7fd9ae628700 1 -- 172.21.15.156:0/2437924442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9a80ce100 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.610+0000 7fd9a4ff9700 1 -- 172.21.15.156:0/2437924442 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fd998020090 con 0x7fd9a80cd8c0 2023-11-28T11:09:41.612 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:41.612 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1085440106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1085440106},{"type":"v1","addr":"172.21.15.186:6834","nonce":1085440106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1548245207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1548245207},{"type":"v1","addr":"172.21.15.156:6836","nonce":1548245207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4882,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3102077957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":3102077957},{"type":"v1","addr":"172.21.15.156:6837","nonce":3102077957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:15.391255+0000","modified":"2023-11-28T11:09:18.414744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3897746301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3897746301},{"type":"v1","addr":"172.21.15.186:6835","nonce":3897746301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:09:41.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.612+0000 7fd9ae628700 1 -- 172.21.15.156:0/2437924442 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd9940682f0 msgr2=0x7fd99406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.612+0000 7fd9ae628700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd9940682f0 0x7fd99406a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009d50 tx=0x7fd99c0091c0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.612+0000 7fd9ae628700 1 -- 172.21.15.156:0/2437924442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a80cd8c0 msgr2=0x7fd9a813ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.612+0000 7fd9ae628700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a80cd8c0 0x7fd9a813ccf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd998009fd0 tx=0x7fd998004750 comp rx=0 tx=0).stop 2023-11-28T11:09:41.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.612+0000 7fd9ae628700 1 -- 172.21.15.156:0/2437924442 shutdown_connections 2023-11-28T11:09:41.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.612+0000 7fd9ae628700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9a80cdc90 0x7fd9a813d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.612+0000 7fd9ae628700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd9940682f0 0x7fd99406a7a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.612+0000 7fd9ae628700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9a80b2e70 0x7fd9a8154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.612+0000 7fd9ae628700 1 --2- 172.21.15.156:0/2437924442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a80cd8c0 0x7fd9a813ccf0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.612+0000 7fd9ae628700 1 -- 172.21.15.156:0/2437924442 >> 172.21.15.156:0/2437924442 conn(0x7fd9a806aed0 msgr2=0x7fd9a80cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.613+0000 7fd9ae628700 1 -- 172.21.15.156:0/2437924442 shutdown_connections 2023-11-28T11:09:41.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.613+0000 7fd9ae628700 1 -- 172.21.15.156:0/2437924442 wait complete. 2023-11-28T11:09:41.615 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T11:09:41.626 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:41.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.747+0000 7fa2b3335700 1 Processor -- start 2023-11-28T11:09:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.747+0000 7fa2b3335700 1 -- start start 2023-11-28T11:09:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b3335700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac06dee0 0x7fa2ac06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b3335700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa2ac160540 0x7fa2ac1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b3335700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac15d060 con 0x7fa2ac071040 2023-11-28T11:09:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b3335700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ac15d340 con 0x7fa2ac160540 2023-11-28T11:09:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b3335700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac15d620 con 0x7fa2ac06dee0 2023-11-28T11:09:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b18d2700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa2ac160540 0x7fa2ac1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b08d0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac06dee0 0x7fa2ac06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b18d2700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa2ac160540 0x7fa2ac1609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:45288/0 (socket says 172.21.15.156:45288) 2023-11-28T11:09:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b18d2700 1 -- 172.21.15.156:0/3499578650 learned_addr learned my addr 172.21.15.156:0/3499578650 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b08d0700 1 -- 172.21.15.156:0/3499578650 >> v2:172.21.15.186:3300/0 conn(0x7fa2ac160540 msgr2=0x7fa2ac1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b08d0700 1 --2- 172.21.15.156:0/3499578650 >> v2:172.21.15.186:3300/0 conn(0x7fa2ac160540 0x7fa2ac1609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b08d0700 1 -- 172.21.15.156:0/3499578650 >> v1:172.21.15.186:6789/0 conn(0x7fa2ac071040 legacy=0x7fa2ac071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b08d0700 1 -- 172.21.15.156:0/3499578650 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac15d900 con 0x7fa2ac06dee0 2023-11-28T11:09:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.748+0000 7fa2b18d2700 1 --2- 172.21.15.156:0/3499578650 >> v2:172.21.15.186:3300/0 conn(0x7fa2ac160540 0x7fa2ac1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.749+0000 7fa2b08d0700 1 --2- 172.21.15.156:0/3499578650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac06dee0 0x7fa2ac06e330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009840 tx=0x7fa29c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6597394f400bae49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.749+0000 7fa2a3fff700 1 -- 172.21.15.156:0/3499578650 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c011830 con 0x7fa2ac06dee0 2023-11-28T11:09:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.749+0000 7fa2a3fff700 1 -- 172.21.15.156:0/3499578650 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29c011990 con 0x7fa2ac06dee0 2023-11-28T11:09:41.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.749+0000 7fa2a3fff700 1 -- 172.21.15.156:0/3499578650 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c011d20 con 0x7fa2ac06dee0 2023-11-28T11:09:41.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.749+0000 7fa2b3335700 1 -- 172.21.15.156:0/3499578650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac06dee0 msgr2=0x7fa2ac06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.749+0000 7fa2b3335700 1 --2- 172.21.15.156:0/3499578650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac06dee0 0x7fa2ac06e330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009840 tx=0x7fa29c009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:41.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.749+0000 7fa2b3335700 1 -- 172.21.15.156:0/3499578650 shutdown_connections 2023-11-28T11:09:41.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.749+0000 7fa2b3335700 1 --2- 172.21.15.156:0/3499578650 >> v2:172.21.15.186:3300/0 conn(0x7fa2ac160540 0x7fa2ac1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.749+0000 7fa2b3335700 1 --2- 172.21.15.156:0/3499578650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac06dee0 0x7fa2ac06e330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.749+0000 7fa2b3335700 1 -- 172.21.15.156:0/3499578650 >> 172.21.15.156:0/3499578650 conn(0x7fa2ac0d0000 msgr2=0x7fa2ac0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.749+0000 7fa2b3335700 1 -- 172.21.15.156:0/3499578650 shutdown_connections 2023-11-28T11:09:41.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.749+0000 7fa2b3335700 1 -- 172.21.15.156:0/3499578650 wait complete. 2023-11-28T11:09:41.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.751+0000 7fa2b3335700 1 Processor -- start 2023-11-28T11:09:41.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.751+0000 7fa2b3335700 1 -- start start 2023-11-28T11:09:41.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.751+0000 7fa2b3335700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2ac06dee0 0x7fa2ac1485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.751+0000 7fa2b3335700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac071040 0x7fa2ac15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.751+0000 7fa2b3335700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac160540 0x7fa2ac0adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.751+0000 7fa2b3335700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0b2e30 con 0x7fa2ac071040 2023-11-28T11:09:41.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.751+0000 7fa2b3335700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac15b780 con 0x7fa2ac06dee0 2023-11-28T11:09:41.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.751+0000 7fa2b3335700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac15ba60 con 0x7fa2ac160540 2023-11-28T11:09:41.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.751+0000 7fa2b08d0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac071040 0x7fa2ac15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.751+0000 7fa2b08d0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac071040 0x7fa2ac15ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34276/0 (socket says 172.21.15.156:34276) 2023-11-28T11:09:41.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.751+0000 7fa2b08d0700 1 -- 172.21.15.156:0/98847787 learned_addr learned my addr 172.21.15.156:0/98847787 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:41.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2b10d1700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2ac06dee0 0x7fa2ac1485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2b18d2700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac160540 0x7fa2ac0adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2b08d0700 1 -- 172.21.15.156:0/98847787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac160540 msgr2=0x7fa2ac0adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2b08d0700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac160540 0x7fa2ac0adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2b08d0700 1 -- 172.21.15.156:0/98847787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2ac06dee0 msgr2=0x7fa2ac1485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2b08d0700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2ac06dee0 0x7fa2ac1485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2b08d0700 1 -- 172.21.15.156:0/98847787 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a4009b30 con 0x7fa2ac071040 2023-11-28T11:09:41.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2b08d0700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac071040 0x7fa2ac15ace0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0090b0 tx=0x7fa29c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2a1ffb700 1 -- 172.21.15.156:0/98847787 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c0036a0 con 0x7fa2ac071040 2023-11-28T11:09:41.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2a1ffb700 1 -- 172.21.15.156:0/98847787 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29c01e070 con 0x7fa2ac071040 2023-11-28T11:09:41.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2b3335700 1 -- 172.21.15.156:0/98847787 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29c0094a0 con 0x7fa2ac071040 2023-11-28T11:09:41.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2a1ffb700 1 -- 172.21.15.156:0/98847787 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c003a80 con 0x7fa2ac071040 2023-11-28T11:09:41.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.752+0000 7fa2b3335700 1 -- 172.21.15.156:0/98847787 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac15ccd0 con 0x7fa2ac071040 2023-11-28T11:09:41.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.753+0000 7fa2b3335700 1 -- 172.21.15.156:0/98847787 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac06e870 con 0x7fa2ac071040 2023-11-28T11:09:41.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.754+0000 7fa2a1ffb700 1 -- 172.21.15.156:0/98847787 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa29c00f050 con 0x7fa2ac071040 2023-11-28T11:09:41.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.754+0000 7fa2a1ffb700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa2900682f0 0x7fa29006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.754+0000 7fa2a1ffb700 1 -- 172.21.15.156:0/98847787 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa29c087030 con 0x7fa2ac071040 2023-11-28T11:09:41.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.754+0000 7fa2b10d1700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa2900682f0 0x7fa29006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.755+0000 7fa2b10d1700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa2900682f0 0x7fa29006a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8007fd0 tx=0x7fa2a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.758+0000 7fa2a1ffb700 1 -- 172.21.15.156:0/98847787 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa29c0537a0 con 0x7fa2ac071040 2023-11-28T11:09:41.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.981+0000 7fa2b3335700 1 -- 172.21.15.156:0/98847787 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2ac06a270 con 0x7fa2ac071040 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.981+0000 7fa2a1ffb700 1 -- 172.21.15.156:0/98847787 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fa29c020070 con 0x7fa2ac071040 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1085440106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1085440106},{"type":"v1","addr":"172.21.15.186:6834","nonce":1085440106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1548245207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1548245207},{"type":"v1","addr":"172.21.15.156:6836","nonce":1548245207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4882,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3102077957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":3102077957},{"type":"v1","addr":"172.21.15.156:6837","nonce":3102077957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:15.391255+0000","modified":"2023-11-28T11:09:18.414744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3897746301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3897746301},{"type":"v1","addr":"172.21.15.186:6835","nonce":3897746301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:09:41.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.983+0000 7fa2b3335700 1 -- 172.21.15.156:0/98847787 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa2900682f0 msgr2=0x7fa29006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.983+0000 7fa2b3335700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa2900682f0 0x7fa29006a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8007fd0 tx=0x7fa2a8009040 comp rx=0 tx=0).stop 2023-11-28T11:09:41.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.984+0000 7fa2b3335700 1 -- 172.21.15.156:0/98847787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac071040 msgr2=0x7fa2ac15ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.984+0000 7fa2b3335700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac071040 0x7fa2ac15ace0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0090b0 tx=0x7fa29c004750 comp rx=0 tx=0).stop 2023-11-28T11:09:41.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.984+0000 7fa2b3335700 1 -- 172.21.15.156:0/98847787 shutdown_connections 2023-11-28T11:09:41.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.984+0000 7fa2b3335700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa2900682f0 0x7fa29006a7a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.984+0000 7fa2b3335700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2ac160540 0x7fa2ac0adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.984+0000 7fa2b3335700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2ac06dee0 0x7fa2ac1485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.984+0000 7fa2b3335700 1 --2- 172.21.15.156:0/98847787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2ac071040 0x7fa2ac15ace0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.984+0000 7fa2b3335700 1 -- 172.21.15.156:0/98847787 >> 172.21.15.156:0/98847787 conn(0x7fa2ac0d0000 msgr2=0x7fa2ac15e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.984+0000 7fa2b3335700 1 -- 172.21.15.156:0/98847787 shutdown_connections 2023-11-28T11:09:41.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:41.984+0000 7fa2b3335700 1 -- 172.21.15.156:0/98847787 wait complete. 2023-11-28T11:09:41.986 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T11:09:41.997 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:42.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.116+0000 7f0f39dd6700 1 Processor -- start 2023-11-28T11:09:42.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.117+0000 7f0f39dd6700 1 -- start start 2023-11-28T11:09:42.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.117+0000 7f0f39dd6700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0f3406dee0 0x7f0f3406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.117+0000 7f0f39dd6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34160540 0x7f0f341609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.117+0000 7f0f39dd6700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f3415d060 con 0x7f0f34071040 2023-11-28T11:09:42.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.117+0000 7f0f39dd6700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f3415d340 con 0x7f0f3406dee0 2023-11-28T11:09:42.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.117+0000 7f0f39dd6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3415d620 con 0x7f0f34160540 2023-11-28T11:09:42.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.117+0000 7f0f337fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f0f34071040 0x7f0f34071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49162/0 (socket says 172.21.15.156:49162) 2023-11-28T11:09:42.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.117+0000 7f0f337fe700 1 -- 172.21.15.156:0/72993418 learned_addr learned my addr 172.21.15.156:0/72993418 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:42.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f33fff700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34160540 0x7f0f341609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f32ffd700 1 --2- 172.21.15.156:0/72993418 >> v2:172.21.15.186:3300/0 conn(0x7f0f3406dee0 0x7f0f3406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893726582 0 0) 0x7f0f3415d060 con 0x7f0f34071040 2023-11-28T11:09:42.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f200035b0 con 0x7f0f34071040 2023-11-28T11:09:42.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1110815617 0 0) 0x7f0f200035b0 con 0x7f0f34071040 2023-11-28T11:09:42.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34160540 msgr2=0x7f0f341609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34160540 0x7f0f341609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 >> v2:172.21.15.186:3300/0 conn(0x7f0f3406dee0 msgr2=0x7f0f3406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 --2- 172.21.15.156:0/72993418 >> v2:172.21.15.186:3300/0 conn(0x7f0f3406dee0 0x7f0f3406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3415d900 con 0x7f0f34071040 2023-11-28T11:09:42.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f0f24001670 con 0x7f0f34071040 2023-11-28T11:09:42.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 >> v1:172.21.15.156:6789/0 conn(0x7f0f34071040 legacy=0x7f0f34071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34160540 0x7f0f20003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f20003f80 0x7f0f20004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f20004d90 0x7f0f20005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3415d340 con 0x7f0f20004d90 2023-11-28T11:09:42.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3415d620 con 0x7f0f20003f80 2023-11-28T11:09:42.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2000baf0 con 0x7f0f34160540 2023-11-28T11:09:42.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f33fff700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f20004d90 0x7f0f20005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f32ffd700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f20003f80 0x7f0f20004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f337fe700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34160540 0x7f0f20003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f33fff700 1 -- 172.21.15.156:0/72993418 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34160540 msgr2=0x7f0f20003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f33fff700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34160540 0x7f0f20003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.118+0000 7f0f33fff700 1 -- 172.21.15.156:0/72993418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f20003f80 msgr2=0x7f0f20004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.119+0000 7f0f33fff700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f20003f80 0x7f0f20004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.119+0000 7f0f33fff700 1 -- 172.21.15.156:0/72993418 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f200035b0 con 0x7f0f20004d90 2023-11-28T11:09:42.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.119+0000 7f0f32ffd700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f20003f80 0x7f0f20004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:42.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.119+0000 7f0f33fff700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f20004d90 0x7f0f20005200 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009f90 tx=0x7f0f20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820882e47172d2d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.119+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c00ec30 con 0x7f0f20004d90 2023-11-28T11:09:42.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.119+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2c00f040 con 0x7f0f20004d90 2023-11-28T11:09:42.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.119+0000 7f0f327fc700 1 -- 172.21.15.156:0/72993418 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c014480 con 0x7f0f20004d90 2023-11-28T11:09:42.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.119+0000 7f0f39dd6700 1 -- 172.21.15.156:0/72993418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f20004d90 msgr2=0x7f0f20005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.119+0000 7f0f39dd6700 1 --2- 172.21.15.156:0/72993418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f20004d90 0x7f0f20005200 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009f90 tx=0x7f0f20005880 comp rx=0 tx=0).stop 2023-11-28T11:09:42.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.119+0000 7f0f33fff700 1 -- 172.21.15.156:0/72993418 reap_dead start 2023-11-28T11:09:42.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.119+0000 7f0f39dd6700 1 -- 172.21.15.156:0/72993418 shutdown_connections 2023-11-28T11:09:42.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.119+0000 7f0f39dd6700 1 -- 172.21.15.156:0/72993418 >> 172.21.15.156:0/72993418 conn(0x7f0f340d0000 msgr2=0x7f0f340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.120+0000 7f0f39dd6700 1 -- 172.21.15.156:0/72993418 shutdown_connections 2023-11-28T11:09:42.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.120+0000 7f0f39dd6700 1 -- 172.21.15.156:0/72993418 wait complete. 2023-11-28T11:09:42.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.121+0000 7f0f39dd6700 1 Processor -- start 2023-11-28T11:09:42.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.121+0000 7f0f39dd6700 1 -- start start 2023-11-28T11:09:42.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f39dd6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34150430 0x7f0f340aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f39dd6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f340b2e30 0x7f0f340b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f39dd6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f34150800 0x7f0f340b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f39dd6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340b32f0 con 0x7f0f340b2e30 2023-11-28T11:09:42.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f39dd6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340b35d0 con 0x7f0f34150800 2023-11-28T11:09:42.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f39dd6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340b38b0 con 0x7f0f34150430 2023-11-28T11:09:42.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f32ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f340b2e30 0x7f0f340b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f32ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f340b2e30 0x7f0f340b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34298/0 (socket says 172.21.15.156:34298) 2023-11-28T11:09:42.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f32ffd700 1 -- 172.21.15.156:0/795430996 learned_addr learned my addr 172.21.15.156:0/795430996 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:42.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f337fe700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34150430 0x7f0f340aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f33fff700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f34150800 0x7f0f340b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f32ffd700 1 -- 172.21.15.156:0/795430996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34150430 msgr2=0x7f0f340aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f32ffd700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34150430 0x7f0f340aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f32ffd700 1 -- 172.21.15.156:0/795430996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f34150800 msgr2=0x7f0f340b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f32ffd700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f34150800 0x7f0f340b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f32ffd700 1 -- 172.21.15.156:0/795430996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3415ccd0 con 0x7f0f340b2e30 2023-11-28T11:09:42.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f32ffd700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f340b2e30 0x7f0f340b0a50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c011cd0 tx=0x7f0f1c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f2bfff700 1 -- 172.21.15.156:0/795430996 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c011e70 con 0x7f0f340b2e30 2023-11-28T11:09:42.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f2bfff700 1 -- 172.21.15.156:0/795430996 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c00fd40 con 0x7f0f340b2e30 2023-11-28T11:09:42.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f39dd6700 1 -- 172.21.15.156:0/795430996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3406e870 con 0x7f0f340b2e30 2023-11-28T11:09:42.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f2bfff700 1 -- 172.21.15.156:0/795430996 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c004240 con 0x7f0f340b2e30 2023-11-28T11:09:42.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.122+0000 7f0f39dd6700 1 -- 172.21.15.156:0/795430996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3406a270 con 0x7f0f340b2e30 2023-11-28T11:09:42.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.123+0000 7f0f39dd6700 1 -- 172.21.15.156:0/795430996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f340c46b0 con 0x7f0f340b2e30 2023-11-28T11:09:42.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.124+0000 7f0f2bfff700 1 -- 172.21.15.156:0/795430996 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0f1c024070 con 0x7f0f340b2e30 2023-11-28T11:09:42.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.124+0000 7f0f2bfff700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f140682f0 0x7f0f1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.124+0000 7f0f2bfff700 1 -- 172.21.15.156:0/795430996 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f1c086310 con 0x7f0f340b2e30 2023-11-28T11:09:42.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.124+0000 7f0f337fe700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f140682f0 0x7f0f1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.125+0000 7f0f337fe700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f140682f0 0x7f0f1406a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0f240014f0 tx=0x7f0f24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.128+0000 7f0f2bfff700 1 -- 172.21.15.156:0/795430996 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0f340c46b0 con 0x7f0f340b2e30 2023-11-28T11:09:42.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.312+0000 7f0f39dd6700 1 -- 172.21.15.156:0/795430996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f340c4690 con 0x7f0f340b2e30 2023-11-28T11:09:42.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.313+0000 7f0f2bfff700 1 -- 172.21.15.156:0/795430996 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f0f1c022090 con 0x7f0f340b2e30 2023-11-28T11:09:42.315 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:42.315 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":37,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:16.393008+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:13.663041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:14.347248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:42.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.315+0000 7f0f39dd6700 1 -- 172.21.15.156:0/795430996 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f140682f0 msgr2=0x7f0f1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.315+0000 7f0f39dd6700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f140682f0 0x7f0f1406a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0f240014f0 tx=0x7f0f24009040 comp rx=0 tx=0).stop 2023-11-28T11:09:42.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.315+0000 7f0f39dd6700 1 -- 172.21.15.156:0/795430996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f340b2e30 msgr2=0x7f0f340b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.315+0000 7f0f39dd6700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f340b2e30 0x7f0f340b0a50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c011cd0 tx=0x7f0f1c00f520 comp rx=0 tx=0).stop 2023-11-28T11:09:42.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.315+0000 7f0f39dd6700 1 -- 172.21.15.156:0/795430996 shutdown_connections 2023-11-28T11:09:42.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.315+0000 7f0f39dd6700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f34150800 0x7f0f340b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.315+0000 7f0f39dd6700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0f140682f0 0x7f0f1406a7a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.315+0000 7f0f39dd6700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f34150430 0x7f0f340aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.315+0000 7f0f39dd6700 1 --2- 172.21.15.156:0/795430996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f340b2e30 0x7f0f340b0a50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.315+0000 7f0f39dd6700 1 -- 172.21.15.156:0/795430996 >> 172.21.15.156:0/795430996 conn(0x7f0f340d0000 msgr2=0x7f0f340cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.315+0000 7f0f39dd6700 1 -- 172.21.15.156:0/795430996 shutdown_connections 2023-11-28T11:09:42.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.315+0000 7f0f39dd6700 1 -- 172.21.15.156:0/795430996 wait complete. 2023-11-28T11:09:42.329 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:09:42.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.449+0000 7f1f6fa21700 1 Processor -- start 2023-11-28T11:09:42.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.449+0000 7f1f6fa21700 1 -- start start 2023-11-28T11:09:42.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6fa21700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1f68071040 0x7f1f68071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6fa21700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f68160540 0x7f1f681609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6fa21700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f68071950 con 0x7f1f6806dee0 2023-11-28T11:09:42.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6fa21700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6815d060 con 0x7f1f68071040 2023-11-28T11:09:42.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6fa21700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6815d340 con 0x7f1f68160540 2023-11-28T11:09:42.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6d7bd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1f68071040 0x7f1f68071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6cfbc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f1f6806dee0 0x7f1f6806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49182/0 (socket says 172.21.15.156:49182) 2023-11-28T11:09:42.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6d7bd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1f68071040 0x7f1f68071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34304/0 (socket says 172.21.15.156:34304) 2023-11-28T11:09:42.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6cfbc700 1 -- 172.21.15.156:0/582288765 learned_addr learned my addr 172.21.15.156:0/582288765 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:42.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6d7bd700 1 -- 172.21.15.156:0/582288765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f68160540 msgr2=0x7f1f681609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6d7bd700 1 --2- 172.21.15.156:0/582288765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f68160540 0x7f1f681609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6d7bd700 1 -- 172.21.15.156:0/582288765 >> v1:172.21.15.156:6789/0 conn(0x7f1f6806dee0 legacy=0x7f1f6806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6d7bd700 1 -- 172.21.15.156:0/582288765 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6815d620 con 0x7f1f68071040 2023-11-28T11:09:42.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f5ffff700 1 -- 172.21.15.156:0/582288765 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800044431 0 0) 0x7f1f68071950 con 0x7f1f6806dee0 2023-11-28T11:09:42.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f6d7bd700 1 --2- 172.21.15.156:0/582288765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f68071040 0x7f1f68071410 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009840 tx=0x7f1f64009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4c6780c70e5390 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f5ffff700 1 -- 172.21.15.156:0/582288765 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64011830 con 0x7f1f68071040 2023-11-28T11:09:42.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f5ffff700 1 -- 172.21.15.156:0/582288765 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f64011990 con 0x7f1f68071040 2023-11-28T11:09:42.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.450+0000 7f1f5ffff700 1 -- 172.21.15.156:0/582288765 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64011d20 con 0x7f1f68071040 2023-11-28T11:09:42.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.451+0000 7f1f6fa21700 1 -- 172.21.15.156:0/582288765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f68071040 msgr2=0x7f1f68071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.451+0000 7f1f6fa21700 1 --2- 172.21.15.156:0/582288765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f68071040 0x7f1f68071410 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009840 tx=0x7f1f64009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:42.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.451+0000 7f1f6fa21700 1 -- 172.21.15.156:0/582288765 shutdown_connections 2023-11-28T11:09:42.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.451+0000 7f1f6fa21700 1 --2- 172.21.15.156:0/582288765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f68160540 0x7f1f681609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.451+0000 7f1f6fa21700 1 --2- 172.21.15.156:0/582288765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f68071040 0x7f1f68071410 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.451+0000 7f1f6fa21700 1 -- 172.21.15.156:0/582288765 >> 172.21.15.156:0/582288765 conn(0x7f1f680d0000 msgr2=0x7f1f680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.451+0000 7f1f6fa21700 1 -- 172.21.15.156:0/582288765 shutdown_connections 2023-11-28T11:09:42.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.451+0000 7f1f6fa21700 1 -- 172.21.15.156:0/582288765 wait complete. 2023-11-28T11:09:42.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.452+0000 7f1f6fa21700 1 Processor -- start 2023-11-28T11:09:42.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.453+0000 7f1f6fa21700 1 -- start start 2023-11-28T11:09:42.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.453+0000 7f1f6fa21700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6806dee0 0x7f1f680b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.453+0000 7f1f6fa21700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f68071040 0x7f1f680b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.453+0000 7f1f6fa21700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f68160540 0x7f1f681f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.453+0000 7f1f6fa21700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f680c59e0 con 0x7f1f6806dee0 2023-11-28T11:09:42.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.453+0000 7f1f6fa21700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f680b5e60 con 0x7f1f68071040 2023-11-28T11:09:42.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.453+0000 7f1f6fa21700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f680c4dd0 con 0x7f1f68160540 2023-11-28T11:09:42.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f6d7bd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6806dee0 0x7f1f680b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f6d7bd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6806dee0 0x7f1f680b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34310/0 (socket says 172.21.15.156:34310) 2023-11-28T11:09:42.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f6d7bd700 1 -- 172.21.15.156:0/2909284270 learned_addr learned my addr 172.21.15.156:0/2909284270 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:42.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f6d7bd700 1 -- 172.21.15.156:0/2909284270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f68160540 msgr2=0x7f1f681f2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f6d7bd700 1 --2- 172.21.15.156:0/2909284270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f68160540 0x7f1f681f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f6d7bd700 1 -- 172.21.15.156:0/2909284270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f68071040 msgr2=0x7f1f680b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f6d7bd700 1 --2- 172.21.15.156:0/2909284270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f68071040 0x7f1f680b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f6d7bd700 1 -- 172.21.15.156:0/2909284270 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640094a0 con 0x7f1f6806dee0 2023-11-28T11:09:42.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f6d7bd700 1 --2- 172.21.15.156:0/2909284270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6806dee0 0x7f1f680b2e30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1f640090b0 tx=0x7f1f64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f5dffb700 1 -- 172.21.15.156:0/2909284270 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f640036a0 con 0x7f1f6806dee0 2023-11-28T11:09:42.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f5dffb700 1 -- 172.21.15.156:0/2909284270 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6401e070 con 0x7f1f6806dee0 2023-11-28T11:09:42.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f6fa21700 1 -- 172.21.15.156:0/2909284270 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6815ccd0 con 0x7f1f6806dee0 2023-11-28T11:09:42.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f5dffb700 1 -- 172.21.15.156:0/2909284270 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64003a20 con 0x7f1f6806dee0 2023-11-28T11:09:42.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.454+0000 7f1f6fa21700 1 -- 172.21.15.156:0/2909284270 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6815d900 con 0x7f1f6806dee0 2023-11-28T11:09:42.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.455+0000 7f1f6fa21700 1 -- 172.21.15.156:0/2909284270 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6806a270 con 0x7f1f6806dee0 2023-11-28T11:09:42.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.456+0000 7f1f5dffb700 1 -- 172.21.15.156:0/2909284270 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1f6400f050 con 0x7f1f6806dee0 2023-11-28T11:09:42.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.456+0000 7f1f5dffb700 1 --2- 172.21.15.156:0/2909284270 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1f4c0682f0 0x7f1f4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.456+0000 7f1f5dffb700 1 -- 172.21.15.156:0/2909284270 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f64086de0 con 0x7f1f6806dee0 2023-11-28T11:09:42.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.456+0000 7f1f6cfbc700 1 --2- 172.21.15.156:0/2909284270 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1f4c0682f0 0x7f1f4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.457+0000 7f1f6cfbc700 1 --2- 172.21.15.156:0/2909284270 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1f4c0682f0 0x7f1f4c06a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f6815c6e0 tx=0x7f1f580079f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.460+0000 7f1f5dffb700 1 -- 172.21.15.156:0/2909284270 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1f640536f0 con 0x7f1f6806dee0 2023-11-28T11:09:42.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.687+0000 7f1f6fa21700 1 -- 172.21.15.156:0/2909284270 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1f6804e620 con 0x7f1f6806dee0 2023-11-28T11:09:42.821 INFO:tasks.ceph.mon.a.smithi156.stderr:2023-11-28T11:09:42.820+0000 7f6a4eb7c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:09:42.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.825+0000 7f1f5dffb700 1 -- 172.21.15.156:0/2909284270 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1f64020020 con 0x7f1f6806dee0 2023-11-28T11:09:42.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.828+0000 7f1f6fa21700 1 -- 172.21.15.156:0/2909284270 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1f4c0682f0 msgr2=0x7f1f4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.828+0000 7f1f6fa21700 1 --2- 172.21.15.156:0/2909284270 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1f4c0682f0 0x7f1f4c06a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f6815c6e0 tx=0x7f1f580079f0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.828+0000 7f1f6fa21700 1 -- 172.21.15.156:0/2909284270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6806dee0 msgr2=0x7f1f680b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.828+0000 7f1f6fa21700 1 --2- 172.21.15.156:0/2909284270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6806dee0 0x7f1f680b2e30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1f640090b0 tx=0x7f1f64004750 comp rx=0 tx=0).stop 2023-11-28T11:09:42.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.828+0000 7f1f6fa21700 1 -- 172.21.15.156:0/2909284270 shutdown_connections 2023-11-28T11:09:42.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.828+0000 7f1f6fa21700 1 --2- 172.21.15.156:0/2909284270 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1f4c0682f0 0x7f1f4c06a7a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.828+0000 7f1f6fa21700 1 --2- 172.21.15.156:0/2909284270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f68160540 0x7f1f681f2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.828+0000 7f1f6fa21700 1 --2- 172.21.15.156:0/2909284270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6806dee0 0x7f1f680b2e30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.828+0000 7f1f6fa21700 1 --2- 172.21.15.156:0/2909284270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f68071040 0x7f1f680b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.828+0000 7f1f6fa21700 1 -- 172.21.15.156:0/2909284270 >> 172.21.15.156:0/2909284270 conn(0x7f1f680d0000 msgr2=0x7f1f680cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.828+0000 7f1f6fa21700 1 -- 172.21.15.156:0/2909284270 shutdown_connections 2023-11-28T11:09:42.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.828+0000 7f1f6fa21700 1 -- 172.21.15.156:0/2909284270 wait complete. 2023-11-28T11:09:42.831 INFO:teuthology.orchestra.run.smithi156.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:09:42.832 INFO:tasks.ceph.mds.d.smithi186.stderr: -7250> 2023-11-28T11:09:18.918+0000 7f51bf4af700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:09:42.842 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:09:42.867 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-11-28T11:09:42.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.963+0000 7faee8dd8700 1 Processor -- start 2023-11-28T11:09:42.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.963+0000 7faee8dd8700 1 -- start start 2023-11-28T11:09:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.964+0000 7faee8dd8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faee406dee0 0x7faee406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.964+0000 7faee8dd8700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee415d060 con 0x7faee4071040 2023-11-28T11:09:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.964+0000 7faee8dd8700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee415d340 con 0x7faee4160540 2023-11-28T11:09:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.964+0000 7faee8dd8700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee415d620 con 0x7faee406dee0 2023-11-28T11:09:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.965+0000 7faee1d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faee406dee0 0x7faee406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.965+0000 7faee259c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7faee4071040 0x7faee4071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49196/0 (socket says 172.21.15.156:49196) 2023-11-28T11:09:42.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.965+0000 7faee259c700 1 -- 172.21.15.156:0/3265374269 learned_addr learned my addr 172.21.15.156:0/3265374269 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.965+0000 7faee1d9b700 1 -- 172.21.15.156:0/3265374269 >> v1:172.21.15.186:6789/0 conn(0x7faee4160540 legacy=0x7faee4160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.965+0000 7faee1d9b700 1 -- 172.21.15.156:0/3265374269 >> v1:172.21.15.156:6789/0 conn(0x7faee4071040 legacy=0x7faee4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.965+0000 7faee1d9b700 1 -- 172.21.15.156:0/3265374269 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee415d900 con 0x7faee406dee0 2023-11-28T11:09:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.965+0000 7faee159a700 1 -- 172.21.15.156:0/3265374269 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209710521 0 0) 0x7faee415d060 con 0x7faee4071040 2023-11-28T11:09:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.965+0000 7faee1d9b700 1 --2- 172.21.15.156:0/3265374269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee406dee0 0x7faee406e330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faed4009840 tx=0x7faed4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fbfc9de31df1fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.965+0000 7faee159a700 1 -- 172.21.15.156:0/3265374269 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4011830 con 0x7faee406dee0 2023-11-28T11:09:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.965+0000 7faee159a700 1 -- 172.21.15.156:0/3265374269 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed4011990 con 0x7faee406dee0 2023-11-28T11:09:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.965+0000 7faee159a700 1 -- 172.21.15.156:0/3265374269 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4011d20 con 0x7faee406dee0 2023-11-28T11:09:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.966+0000 7faee8dd8700 1 -- 172.21.15.156:0/3265374269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee406dee0 msgr2=0x7faee406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.966+0000 7faee8dd8700 1 --2- 172.21.15.156:0/3265374269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee406dee0 0x7faee406e330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faed4009840 tx=0x7faed4009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.966+0000 7faee8dd8700 1 -- 172.21.15.156:0/3265374269 shutdown_connections 2023-11-28T11:09:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.966+0000 7faee8dd8700 1 --2- 172.21.15.156:0/3265374269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee406dee0 0x7faee406e330 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.966+0000 7faee8dd8700 1 -- 172.21.15.156:0/3265374269 >> 172.21.15.156:0/3265374269 conn(0x7faee40d0000 msgr2=0x7faee40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.966+0000 7faee8dd8700 1 -- 172.21.15.156:0/3265374269 shutdown_connections 2023-11-28T11:09:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.966+0000 7faee8dd8700 1 -- 172.21.15.156:0/3265374269 wait complete. 2023-11-28T11:09:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.967+0000 7faee8dd8700 1 Processor -- start 2023-11-28T11:09:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.967+0000 7faee8dd8700 1 -- start start 2023-11-28T11:09:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee8dd8700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faee406dee0 0x7faee41f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee8dd8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4071040 0x7faee40c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee8dd8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee4160540 0x7faee40c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee8dd8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee40b4460 con 0x7faee4160540 2023-11-28T11:09:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee8dd8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee40b4740 con 0x7faee4071040 2023-11-28T11:09:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee8dd8700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee40b4a20 con 0x7faee406dee0 2023-11-28T11:09:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee4160540 0x7faee40c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee4160540 0x7faee40c5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34328/0 (socket says 172.21.15.156:34328) 2023-11-28T11:09:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee2d9d700 1 -- 172.21.15.156:0/2847303148 learned_addr learned my addr 172.21.15.156:0/2847303148 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee1d9b700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4071040 0x7faee40c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee259c700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faee406dee0 0x7faee41f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee2d9d700 1 -- 172.21.15.156:0/2847303148 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faee406dee0 msgr2=0x7faee41f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee2d9d700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faee406dee0 0x7faee41f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee2d9d700 1 -- 172.21.15.156:0/2847303148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4071040 msgr2=0x7faee40c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.968+0000 7faee2d9d700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4071040 0x7faee40c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.969+0000 7faee2d9d700 1 -- 172.21.15.156:0/2847303148 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed40094a0 con 0x7faee4160540 2023-11-28T11:09:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.969+0000 7faee2d9d700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee4160540 0x7faee40c5f20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7faedc00cba0 tx=0x7faedc00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.969+0000 7faed2ffd700 1 -- 172.21.15.156:0/2847303148 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc003c00 con 0x7faee4160540 2023-11-28T11:09:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.969+0000 7faed2ffd700 1 -- 172.21.15.156:0/2847303148 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faedc003d60 con 0x7faee4160540 2023-11-28T11:09:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.969+0000 7faee8dd8700 1 -- 172.21.15.156:0/2847303148 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee415ccd0 con 0x7faee4160540 2023-11-28T11:09:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.969+0000 7faed2ffd700 1 -- 172.21.15.156:0/2847303148 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc00aac0 con 0x7faee4160540 2023-11-28T11:09:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.969+0000 7faee8dd8700 1 -- 172.21.15.156:0/2847303148 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee406e870 con 0x7faee4160540 2023-11-28T11:09:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.970+0000 7faee8dd8700 1 -- 172.21.15.156:0/2847303148 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee406a270 con 0x7faee4160540 2023-11-28T11:09:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.970+0000 7faed2ffd700 1 -- 172.21.15.156:0/2847303148 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faedc010050 con 0x7faee4160540 2023-11-28T11:09:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.971+0000 7faed2ffd700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faec40682f0 0x7faec406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.971+0000 7faed2ffd700 1 -- 172.21.15.156:0/2847303148 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6148+0+0 (secure 0 0 0) 0x7faedc085af0 con 0x7faee4160540 2023-11-28T11:09:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.971+0000 7faee259c700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faec40682f0 0x7faec406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.971+0000 7faee259c700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faec40682f0 0x7faec406a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faed8000f60 tx=0x7faed8004a50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:42.975+0000 7faed2ffd700 1 -- 172.21.15.156:0/2847303148 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faedc0522f0 con 0x7faee4160540 2023-11-28T11:09:43.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.201+0000 7faee8dd8700 1 -- 172.21.15.156:0/2847303148 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faee40b45a0 con 0x7faee4160540 2023-11-28T11:09:43.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.836+0000 7faed2ffd700 1 -- 172.21.15.156:0/2847303148 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7faee40b45a0 con 0x7faee4160540 2023-11-28T11:09:43.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.838+0000 7faee8dd8700 1 -- 172.21.15.156:0/2847303148 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faec40682f0 msgr2=0x7faec406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.838+0000 7faee8dd8700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faec40682f0 0x7faec406a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faed8000f60 tx=0x7faed8004a50 comp rx=0 tx=0).stop 2023-11-28T11:09:43.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.838+0000 7faee8dd8700 1 -- 172.21.15.156:0/2847303148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee4160540 msgr2=0x7faee40c5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.838+0000 7faee8dd8700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee4160540 0x7faee40c5f20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7faedc00cba0 tx=0x7faedc00ceb0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.838+0000 7faee8dd8700 1 -- 172.21.15.156:0/2847303148 shutdown_connections 2023-11-28T11:09:43.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.838+0000 7faee8dd8700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee4160540 0x7faee40c5f20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.838+0000 7faee8dd8700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faec40682f0 0x7faec406a7a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.838+0000 7faee8dd8700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faee406dee0 0x7faee41f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.838+0000 7faee8dd8700 1 --2- 172.21.15.156:0/2847303148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4071040 0x7faee40c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.838+0000 7faee8dd8700 1 -- 172.21.15.156:0/2847303148 >> 172.21.15.156:0/2847303148 conn(0x7faee40d0000 msgr2=0x7faee415f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.838+0000 7faee8dd8700 1 -- 172.21.15.156:0/2847303148 shutdown_connections 2023-11-28T11:09:43.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.838+0000 7faee8dd8700 1 -- 172.21.15.156:0/2847303148 wait complete. 2023-11-28T11:09:43.856 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:09:43.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.984+0000 7f8b8e475700 1 Processor -- start 2023-11-28T11:09:43.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.985+0000 7f8b8e475700 1 -- start start 2023-11-28T11:09:43.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.985+0000 7f8b8e475700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b8806a980 0x7f8b88070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.985+0000 7f8b8e475700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8b880af430 0x7f8b8806d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.985+0000 7f8b8e475700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b880713c0 con 0x7f8b880af060 2023-11-28T11:09:43.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.985+0000 7f8b8e475700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8806dee0 con 0x7f8b880af430 2023-11-28T11:09:43.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.985+0000 7f8b8e475700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880716a0 con 0x7f8b8806a980 2023-11-28T11:09:43.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.985+0000 7f8b8ca12700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8b880af430 0x7f8b8806d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.985+0000 7f8b8ca12700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8b880af430 0x7f8b8806d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34336/0 (socket says 172.21.15.156:34336) 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.985+0000 7f8b87fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f8b880af060 0x7f8b8806a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49210/0 (socket says 172.21.15.156:49210) 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.985+0000 7f8b8ca12700 1 -- 172.21.15.156:0/766381003 learned_addr learned my addr 172.21.15.156:0/766381003 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.986+0000 7f8b877fe700 1 --2- 172.21.15.156:0/766381003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b8806a980 0x7f8b88070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.986+0000 7f8b8ca12700 1 -- 172.21.15.156:0/766381003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b8806a980 msgr2=0x7f8b88070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.986+0000 7f8b8ca12700 1 --2- 172.21.15.156:0/766381003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b8806a980 0x7f8b88070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.986+0000 7f8b8ca12700 1 -- 172.21.15.156:0/766381003 >> v1:172.21.15.156:6789/0 conn(0x7f8b880af060 legacy=0x7f8b8806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.986+0000 7f8b8ca12700 1 -- 172.21.15.156:0/766381003 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b88071980 con 0x7f8b880af430 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.986+0000 7f8b86ffd700 1 -- 172.21.15.156:0/766381003 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718690742 0 0) 0x7f8b880713c0 con 0x7f8b880af060 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.986+0000 7f8b8ca12700 1 --2- 172.21.15.156:0/766381003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b880af430 0x7f8b8806d9a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009e50 tx=0x7f8b8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec9b97360367997 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.986+0000 7f8b86ffd700 1 -- 172.21.15.156:0/766381003 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000e070 con 0x7f8b880af430 2023-11-28T11:09:43.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.986+0000 7f8b86ffd700 1 -- 172.21.15.156:0/766381003 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8000b9c0 con 0x7f8b880af430 2023-11-28T11:09:43.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.986+0000 7f8b86ffd700 1 -- 172.21.15.156:0/766381003 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000bd70 con 0x7f8b880af430 2023-11-28T11:09:43.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.986+0000 7f8b8e475700 1 -- 172.21.15.156:0/766381003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b880af430 msgr2=0x7f8b8806d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.987+0000 7f8b8e475700 1 --2- 172.21.15.156:0/766381003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b880af430 0x7f8b8806d9a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009e50 tx=0x7f8b8000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:43.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.987+0000 7f8b8e475700 1 -- 172.21.15.156:0/766381003 shutdown_connections 2023-11-28T11:09:43.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.987+0000 7f8b8e475700 1 --2- 172.21.15.156:0/766381003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b880af430 0x7f8b8806d9a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.987+0000 7f8b8e475700 1 --2- 172.21.15.156:0/766381003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b8806a980 0x7f8b88070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.987+0000 7f8b8e475700 1 -- 172.21.15.156:0/766381003 >> 172.21.15.156:0/766381003 conn(0x7f8b8806b380 msgr2=0x7f8b8806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.987+0000 7f8b8e475700 1 -- 172.21.15.156:0/766381003 shutdown_connections 2023-11-28T11:09:43.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.987+0000 7f8b8e475700 1 -- 172.21.15.156:0/766381003 wait complete. 2023-11-28T11:09:43.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.988+0000 7f8b8e475700 1 Processor -- start 2023-11-28T11:09:43.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.988+0000 7f8b8e475700 1 -- start start 2023-11-28T11:09:43.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.989+0000 7f8b8e475700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8806a980 0x7f8b8815fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.989+0000 7f8b8e475700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b880af060 0x7f8b88155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.989+0000 7f8b8e475700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b880af430 0x7f8b880b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.989+0000 7f8b8e475700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880b2840 con 0x7f8b8806a980 2023-11-28T11:09:43.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.989+0000 7f8b8e475700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880b0770 con 0x7f8b880af430 2023-11-28T11:09:43.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.989+0000 7f8b8e475700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880b0a10 con 0x7f8b880af060 2023-11-28T11:09:43.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.989+0000 7f8b87fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8806a980 0x7f8b8815fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.989+0000 7f8b87fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8806a980 0x7f8b8815fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34338/0 (socket says 172.21.15.156:34338) 2023-11-28T11:09:43.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.989+0000 7f8b87fff700 1 -- 172.21.15.156:0/4118747197 learned_addr learned my addr 172.21.15.156:0/4118747197 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:43.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.989+0000 7f8b877fe700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b880af060 0x7f8b88155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b8ca12700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b880af430 0x7f8b880b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b87fff700 1 -- 172.21.15.156:0/4118747197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b880af060 msgr2=0x7f8b88155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b87fff700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b880af060 0x7f8b88155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b87fff700 1 -- 172.21.15.156:0/4118747197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b880af430 msgr2=0x7f8b880b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b87fff700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b880af430 0x7f8b880b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b87fff700 1 -- 172.21.15.156:0/4118747197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b80009b30 con 0x7f8b8806a980 2023-11-28T11:09:43.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b877fe700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b880af060 0x7f8b88155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:43.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b87fff700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8806a980 0x7f8b8815fac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0011e0 tx=0x7f8b7c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b84ff9700 1 -- 172.21.15.156:0/4118747197 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c004c80 con 0x7f8b8806a980 2023-11-28T11:09:43.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b84ff9700 1 -- 172.21.15.156:0/4118747197 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7c004e00 con 0x7f8b8806a980 2023-11-28T11:09:43.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b8e475700 1 -- 172.21.15.156:0/4118747197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b88071f40 con 0x7f8b8806a980 2023-11-28T11:09:43.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b84ff9700 1 -- 172.21.15.156:0/4118747197 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c007310 con 0x7f8b8806a980 2023-11-28T11:09:43.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.990+0000 7f8b8e475700 1 -- 172.21.15.156:0/4118747197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b88071c60 con 0x7f8b8806a980 2023-11-28T11:09:43.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.991+0000 7f8b8e475700 1 -- 172.21.15.156:0/4118747197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b880ca660 con 0x7f8b8806a980 2023-11-28T11:09:43.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.991+0000 7f8b84ff9700 1 -- 172.21.15.156:0/4118747197 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8b7c01a020 con 0x7f8b8806a980 2023-11-28T11:09:43.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.991+0000 7f8b84ff9700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8b74068320 0x7f8b7406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.992+0000 7f8b84ff9700 1 -- 172.21.15.156:0/4118747197 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f8b7c0859e0 con 0x7f8b8806a980 2023-11-28T11:09:43.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.992+0000 7f8b877fe700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8b74068320 0x7f8b7406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.992+0000 7f8b877fe700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8b74068320 0x7f8b7406a7d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b880ca330 tx=0x7f8b78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:43.997+0000 7f8b84ff9700 1 -- 172.21.15.156:0/4118747197 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8b7c052150 con 0x7f8b8806a980 2023-11-28T11:09:44.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.179+0000 7f8b8e475700 1 -- 172.21.15.156:0/4118747197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b88068690 con 0x7f8b8806a980 2023-11-28T11:09:44.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.839+0000 7f8b84ff9700 1 -- 172.21.15.156:0/4118747197 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8b7c085020 con 0x7f8b8806a980 2023-11-28T11:09:44.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.846+0000 7f8b8e475700 1 -- 172.21.15.156:0/4118747197 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8b74068320 msgr2=0x7f8b7406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.846+0000 7f8b8e475700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8b74068320 0x7f8b7406a7d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b880ca330 tx=0x7f8b78008040 comp rx=0 tx=0).stop 2023-11-28T11:09:44.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.846+0000 7f8b8e475700 1 -- 172.21.15.156:0/4118747197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8806a980 msgr2=0x7f8b8815fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.846+0000 7f8b8e475700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8806a980 0x7f8b8815fac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0011e0 tx=0x7f8b7c00cae0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.846+0000 7f8b8e475700 1 -- 172.21.15.156:0/4118747197 shutdown_connections 2023-11-28T11:09:44.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.846+0000 7f8b8e475700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b880af430 0x7f8b880b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.846+0000 7f8b8e475700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8806a980 0x7f8b8815fac0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.846+0000 7f8b8e475700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8b74068320 0x7f8b7406a7d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.846+0000 7f8b8e475700 1 --2- 172.21.15.156:0/4118747197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b880af060 0x7f8b88155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.846+0000 7f8b8e475700 1 -- 172.21.15.156:0/4118747197 >> 172.21.15.156:0/4118747197 conn(0x7f8b8806b380 msgr2=0x7f8b880d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:44.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.846+0000 7f8b8e475700 1 -- 172.21.15.156:0/4118747197 shutdown_connections 2023-11-28T11:09:44.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.846+0000 7f8b8e475700 1 -- 172.21.15.156:0/4118747197 wait complete. 2023-11-28T11:09:44.849 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:09:44.863 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:09:44.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.991+0000 7f424d2dc700 1 Processor -- start 2023-11-28T11:09:44.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.992+0000 7f424d2dc700 1 -- start start 2023-11-28T11:09:44.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.992+0000 7f424d2dc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42480d0610 0x7f42480cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.992+0000 7f424d2dc700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424806de50 con 0x7f4248071c00 2023-11-28T11:09:44.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.992+0000 7f424d2dc700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4248071fd0 con 0x7f424806a270 2023-11-28T11:09:44.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.992+0000 7f424d2dc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42480d0a80 con 0x7f42480d0610 2023-11-28T11:09:44.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.992+0000 7f4246ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4248071c00 0x7f4248070d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49224/0 (socket says 172.21.15.156:49224) 2023-11-28T11:09:44.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.992+0000 7f4246ffd700 1 -- 172.21.15.156:0/87921233 learned_addr learned my addr 172.21.15.156:0/87921233 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:44.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f42467fc700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42480d0610 0x7f42480cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916397781 0 0) 0x7f423c001110 con 0x7f4248071c00 2023-11-28T11:09:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42300035b0 con 0x7f4248071c00 2023-11-28T11:09:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661680756 0 0) 0x7f4248071fd0 con 0x7f424806a270 2023-11-28T11:09:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f423c001110 con 0x7f424806a270 2023-11-28T11:09:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 736659352 0 0) 0x7f42300035b0 con 0x7f4248071c00 2023-11-28T11:09:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42480d0610 msgr2=0x7f42480cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42480d0610 0x7f42480cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 >> v1:172.21.15.186:6789/0 conn(0x7f424806a270 legacy=0x7f42480cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480cb440 con 0x7f4248071c00 2023-11-28T11:09:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f423c001800 con 0x7f4248071c00 2023-11-28T11:09:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 >> v1:172.21.15.156:6789/0 conn(0x7f4248071c00 legacy=0x7f4248070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4230003bc0 0x7f4230003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42480d0610 0x7f4230004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230004d90 0x7f4230005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42480d0a80 con 0x7f4230003bc0 2023-11-28T11:09:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423000bac0 con 0x7f4230004d90 2023-11-28T11:09:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423000bc30 con 0x7f42480d0610 2023-11-28T11:09:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4246ffd700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4230003bc0 0x7f4230003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f42467fc700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42480d0610 0x7f4230004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f42477fe700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230004d90 0x7f4230005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4246ffd700 1 -- 172.21.15.156:0/87921233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42480d0610 msgr2=0x7f4230004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4246ffd700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42480d0610 0x7f4230004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4246ffd700 1 -- 172.21.15.156:0/87921233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230004d90 msgr2=0x7f4230005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4246ffd700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230004d90 0x7f4230005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.993+0000 7f4246ffd700 1 -- 172.21.15.156:0/87921233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42300035b0 con 0x7f4230003bc0 2023-11-28T11:09:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.994+0000 7f42477fe700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230004d90 0x7f4230005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.994+0000 7f4246ffd700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4230003bc0 0x7f4230003fb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f423c00c210 tx=0x7f4230005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1ddd8ebd02fb5a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.994+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c00ccf0 con 0x7f4230003bc0 2023-11-28T11:09:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.994+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c00ce50 con 0x7f4230003bc0 2023-11-28T11:09:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.994+0000 7f4245ffb700 1 -- 172.21.15.156:0/87921233 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c005ba0 con 0x7f4230003bc0 2023-11-28T11:09:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.994+0000 7f424d2dc700 1 -- 172.21.15.156:0/87921233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4230003bc0 msgr2=0x7f4230003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.994+0000 7f424d2dc700 1 --2- 172.21.15.156:0/87921233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4230003bc0 0x7f4230003fb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f423c00c210 tx=0x7f4230005880 comp rx=0 tx=0).stop 2023-11-28T11:09:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.994+0000 7f42477fe700 1 -- 172.21.15.156:0/87921233 reap_dead start 2023-11-28T11:09:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.994+0000 7f424d2dc700 1 -- 172.21.15.156:0/87921233 shutdown_connections 2023-11-28T11:09:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.994+0000 7f424d2dc700 1 -- 172.21.15.156:0/87921233 >> 172.21.15.156:0/87921233 conn(0x7f424806b380 msgr2=0x7f424806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:45.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.994+0000 7f424d2dc700 1 -- 172.21.15.156:0/87921233 shutdown_connections 2023-11-28T11:09:45.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.994+0000 7f424d2dc700 1 -- 172.21.15.156:0/87921233 wait complete. 2023-11-28T11:09:45.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.996+0000 7f424d2dc700 1 Processor -- start 2023-11-28T11:09:45.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.996+0000 7f424d2dc700 1 -- start start 2023-11-28T11:09:45.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.996+0000 7f424d2dc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4230003bc0 0x7f42480b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.996+0000 7f424d2dc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42481542b0 0x7f4248154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.996+0000 7f424d2dc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42480cee10 0x7f42480cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.996+0000 7f424d2dc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248154c60 con 0x7f42480cee10 2023-11-28T11:09:45.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.996+0000 7f424d2dc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248158400 con 0x7f42481542b0 2023-11-28T11:09:45.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.996+0000 7f424d2dc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42481586c0 con 0x7f4230003bc0 2023-11-28T11:09:45.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f42477fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42480cee10 0x7f42480cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f42477fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42480cee10 0x7f42480cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34358/0 (socket says 172.21.15.156:34358) 2023-11-28T11:09:45.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f42477fe700 1 -- 172.21.15.156:0/3840348647 learned_addr learned my addr 172.21.15.156:0/3840348647 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:45.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f42467fc700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42481542b0 0x7f4248154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f4246ffd700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4230003bc0 0x7f42480b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f42477fe700 1 -- 172.21.15.156:0/3840348647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4230003bc0 msgr2=0x7f42480b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f42477fe700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4230003bc0 0x7f42480b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f42477fe700 1 -- 172.21.15.156:0/3840348647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42481542b0 msgr2=0x7f4248154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f42477fe700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42481542b0 0x7f4248154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f42477fe700 1 -- 172.21.15.156:0/3840348647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424806db70 con 0x7f42480cee10 2023-11-28T11:09:45.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f42477fe700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42480cee10 0x7f42480cf280 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4240001a50 tx=0x7f424000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f422f7fe700 1 -- 172.21.15.156:0/3840348647 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240003c10 con 0x7f42480cee10 2023-11-28T11:09:45.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f422f7fe700 1 -- 172.21.15.156:0/3840348647 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240003d90 con 0x7f42480cee10 2023-11-28T11:09:45.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f424d2dc700 1 -- 172.21.15.156:0/3840348647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424806d890 con 0x7f42480cee10 2023-11-28T11:09:45.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f422f7fe700 1 -- 172.21.15.156:0/3840348647 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240005310 con 0x7f42480cee10 2023-11-28T11:09:45.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.997+0000 7f424d2dc700 1 -- 172.21.15.156:0/3840348647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42480cb6c0 con 0x7f42480cee10 2023-11-28T11:09:45.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.998+0000 7f424d2dc700 1 -- 172.21.15.156:0/3840348647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4248068690 con 0x7f42480cee10 2023-11-28T11:09:45.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.998+0000 7f422f7fe700 1 -- 172.21.15.156:0/3840348647 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4240005470 con 0x7f42480cee10 2023-11-28T11:09:45.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.999+0000 7f422f7fe700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f42340682f0 0x7f423406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.999+0000 7f422f7fe700 1 -- 172.21.15.156:0/3840348647 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f4240085cb0 con 0x7f42480cee10 2023-11-28T11:09:45.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.999+0000 7f4246ffd700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f42340682f0 0x7f423406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:44.999+0000 7f4246ffd700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f42340682f0 0x7f423406a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f423c015040 tx=0x7f423c00a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.004+0000 7f422f7fe700 1 -- 172.21.15.156:0/3840348647 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4240052670 con 0x7f42480cee10 2023-11-28T11:09:45.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.185+0000 7f424d2dc700 1 -- 172.21.15.156:0/3840348647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f424806bbd0 con 0x7f42480cee10 2023-11-28T11:09:45.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.846+0000 7f422f7fe700 1 -- 172.21.15.156:0/3840348647 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4240001ca0 con 0x7f42480cee10 2023-11-28T11:09:45.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.850+0000 7f424d2dc700 1 -- 172.21.15.156:0/3840348647 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f42340682f0 msgr2=0x7f423406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.850+0000 7f424d2dc700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f42340682f0 0x7f423406a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f423c015040 tx=0x7f423c00a000 comp rx=0 tx=0).stop 2023-11-28T11:09:45.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.850+0000 7f424d2dc700 1 -- 172.21.15.156:0/3840348647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42480cee10 msgr2=0x7f42480cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.850+0000 7f424d2dc700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42480cee10 0x7f42480cf280 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4240001a50 tx=0x7f424000ee90 comp rx=0 tx=0).stop 2023-11-28T11:09:45.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.851+0000 7f424d2dc700 1 -- 172.21.15.156:0/3840348647 shutdown_connections 2023-11-28T11:09:45.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.851+0000 7f424d2dc700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42480cee10 0x7f42480cf280 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.851+0000 7f424d2dc700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f42340682f0 0x7f423406a7a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.851+0000 7f424d2dc700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4230003bc0 0x7f42480b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.851+0000 7f424d2dc700 1 --2- 172.21.15.156:0/3840348647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42481542b0 0x7f4248154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.851+0000 7f424d2dc700 1 -- 172.21.15.156:0/3840348647 >> 172.21.15.156:0/3840348647 conn(0x7f424806b380 msgr2=0x7f42480b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:45.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.854+0000 7f424d2dc700 1 -- 172.21.15.156:0/3840348647 shutdown_connections 2023-11-28T11:09:45.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.854+0000 7f424d2dc700 1 -- 172.21.15.156:0/3840348647 wait complete. 2023-11-28T11:09:45.856 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' removed 2023-11-28T11:09:45.870 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:09:45.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.994+0000 7f674e8a8700 1 Processor -- start 2023-11-28T11:09:45.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.994+0000 7f674e8a8700 1 -- start start 2023-11-28T11:09:45.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f674e8a8700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f67480cd8c0 0x7f6748140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f674e8a8700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f674806f850 0x7f674806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f674e8a8700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674813ff00 con 0x7f67480cdc90 2023-11-28T11:09:45.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f674e8a8700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67481404c0 con 0x7f67480cd8c0 2023-11-28T11:09:45.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f674e8a8700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674815aa80 con 0x7f674806f850 2023-11-28T11:09:45.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f674ce45700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f674806f850 0x7f674806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f6747fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f67480cd8c0 0x7f6748140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f6747fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f67480cd8c0 0x7f6748140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:45378/0 (socket says 172.21.15.156:45378) 2023-11-28T11:09:45.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f674ce45700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f674806f850 0x7f674806fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:38230/0 (socket says 172.21.15.156:38230) 2023-11-28T11:09:45.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f67477fe700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f67480cdc90 0x7f67480b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:59602/0 (socket says 172.21.15.156:59602) 2023-11-28T11:09:45.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f6747fff700 1 -- 172.21.15.156:0/4074569467 learned_addr learned my addr 172.21.15.156:0/4074569467 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:45.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f674ce45700 1 -- 172.21.15.156:0/4074569467 >> v2:172.21.15.186:3300/0 conn(0x7f67480cd8c0 msgr2=0x7f6748140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:45.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f674ce45700 1 --2- 172.21.15.156:0/4074569467 >> v2:172.21.15.186:3300/0 conn(0x7f67480cd8c0 0x7f6748140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f674ce45700 1 -- 172.21.15.156:0/4074569467 >> v1:172.21.15.186:6789/0 conn(0x7f67480cdc90 legacy=0x7f67480b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f674ce45700 1 -- 172.21.15.156:0/4074569467 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67480b3040 con 0x7f674806f850 2023-11-28T11:09:45.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.995+0000 7f6747fff700 1 --2- 172.21.15.156:0/4074569467 >> v2:172.21.15.186:3300/0 conn(0x7f67480cd8c0 0x7f6748140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:45.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.996+0000 7f674ce45700 1 --2- 172.21.15.156:0/4074569467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f674806f850 0x7f674806fcc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6740009d90 tx=0x7f674000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a932bec5e0ec4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.996+0000 7f6746ffd700 1 -- 172.21.15.156:0/4074569467 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674000e070 con 0x7f674806f850 2023-11-28T11:09:46.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.996+0000 7f6746ffd700 1 -- 172.21.15.156:0/4074569467 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674000bb20 con 0x7f674806f850 2023-11-28T11:09:46.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.996+0000 7f6746ffd700 1 -- 172.21.15.156:0/4074569467 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740012430 con 0x7f674806f850 2023-11-28T11:09:46.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.996+0000 7f674e8a8700 1 -- 172.21.15.156:0/4074569467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f674806f850 msgr2=0x7f674806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.996+0000 7f674e8a8700 1 --2- 172.21.15.156:0/4074569467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f674806f850 0x7f674806fcc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6740009d90 tx=0x7f674000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:46.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.996+0000 7f674e8a8700 1 -- 172.21.15.156:0/4074569467 shutdown_connections 2023-11-28T11:09:46.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.996+0000 7f674e8a8700 1 --2- 172.21.15.156:0/4074569467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f674806f850 0x7f674806fcc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.996+0000 7f674e8a8700 1 --2- 172.21.15.156:0/4074569467 >> v2:172.21.15.186:3300/0 conn(0x7f67480cd8c0 0x7f6748140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.996+0000 7f674e8a8700 1 -- 172.21.15.156:0/4074569467 >> 172.21.15.156:0/4074569467 conn(0x7f674806aed0 msgr2=0x7f674806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.996+0000 7f674e8a8700 1 -- 172.21.15.156:0/4074569467 shutdown_connections 2023-11-28T11:09:46.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.996+0000 7f674e8a8700 1 -- 172.21.15.156:0/4074569467 wait complete. 2023-11-28T11:09:46.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.998+0000 7f674e8a8700 1 Processor -- start 2023-11-28T11:09:46.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.998+0000 7f674e8a8700 1 -- start start 2023-11-28T11:09:46.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.998+0000 7f674e8a8700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f674806f850 0x7f6748143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.998+0000 7f674e8a8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67480cd8c0 0x7f6748140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.998+0000 7f674e8a8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67480cdc90 0x7f674813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.998+0000 7f674e8a8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6748143040 con 0x7f67480cd8c0 2023-11-28T11:09:46.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.998+0000 7f674e8a8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6748143320 con 0x7f67480cdc90 2023-11-28T11:09:46.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.998+0000 7f674e8a8700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6748143600 con 0x7f674806f850 2023-11-28T11:09:46.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f67477fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67480cd8c0 0x7f6748140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f67477fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67480cd8c0 0x7f6748140ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34370/0 (socket says 172.21.15.156:34370) 2023-11-28T11:09:46.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f67477fe700 1 -- 172.21.15.156:0/1828187071 learned_addr learned my addr 172.21.15.156:0/1828187071 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:46.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f674ce45700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67480cdc90 0x7f674813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f6747fff700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f674806f850 0x7f6748143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f67477fe700 1 -- 172.21.15.156:0/1828187071 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f674806f850 msgr2=0x7f6748143e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f67477fe700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f674806f850 0x7f6748143e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f67477fe700 1 -- 172.21.15.156:0/1828187071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67480cdc90 msgr2=0x7f674813ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f67477fe700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67480cdc90 0x7f674813ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f67477fe700 1 -- 172.21.15.156:0/1828187071 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c009500 con 0x7f67480cd8c0 2023-11-28T11:09:46.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f67477fe700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67480cd8c0 0x7f6748140ff0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f673800c4d0 tx=0x7f673800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f6744ff9700 1 -- 172.21.15.156:0/1828187071 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738005cb0 con 0x7f67480cd8c0 2023-11-28T11:09:46.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f6744ff9700 1 -- 172.21.15.156:0/1828187071 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6738005e10 con 0x7f67480cd8c0 2023-11-28T11:09:46.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f674e8a8700 1 -- 172.21.15.156:0/1828187071 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6740009a70 con 0x7f67480cd8c0 2023-11-28T11:09:46.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f6744ff9700 1 -- 172.21.15.156:0/1828187071 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67380072d0 con 0x7f67480cd8c0 2023-11-28T11:09:46.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:45.999+0000 7f674e8a8700 1 -- 172.21.15.156:0/1828187071 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67480b35f0 con 0x7f67480cd8c0 2023-11-28T11:09:46.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.000+0000 7f674e8a8700 1 -- 172.21.15.156:0/1828187071 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67480b3310 con 0x7f67480cd8c0 2023-11-28T11:09:46.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.001+0000 7f6744ff9700 1 -- 172.21.15.156:0/1828187071 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6738019030 con 0x7f67480cd8c0 2023-11-28T11:09:46.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.001+0000 7f6744ff9700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67340682f0 0x7f673406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.001+0000 7f6744ff9700 1 -- 172.21.15.156:0/1828187071 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f67380849b0 con 0x7f67480cd8c0 2023-11-28T11:09:46.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.001+0000 7f6747fff700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67340682f0 0x7f673406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.002+0000 7f6747fff700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67340682f0 0x7f673406a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f673c006fd0 tx=0x7f673c009290 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.005+0000 7f6744ff9700 1 -- 172.21.15.156:0/1828187071 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f673804b5e0 con 0x7f67480cd8c0 2023-11-28T11:09:46.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.187+0000 7f674e8a8700 1 -- 172.21.15.156:0/1828187071 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6748140150 con 0x7f67480cd8c0 2023-11-28T11:09:46.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.190+0000 7f6744ff9700 1 -- 172.21.15.156:0/1828187071 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f67380170a0 con 0x7f67480cd8c0 2023-11-28T11:09:46.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.192+0000 7f674e8a8700 1 -- 172.21.15.156:0/1828187071 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67340682f0 msgr2=0x7f673406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.192+0000 7f674e8a8700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67340682f0 0x7f673406a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f673c006fd0 tx=0x7f673c009290 comp rx=0 tx=0).stop 2023-11-28T11:09:46.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.192+0000 7f674e8a8700 1 -- 172.21.15.156:0/1828187071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67480cd8c0 msgr2=0x7f6748140ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.192+0000 7f674e8a8700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67480cd8c0 0x7f6748140ff0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f673800c4d0 tx=0x7f673800c7e0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.193+0000 7f674e8a8700 1 -- 172.21.15.156:0/1828187071 shutdown_connections 2023-11-28T11:09:46.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.193+0000 7f674e8a8700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67480cdc90 0x7f674813ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.193+0000 7f674e8a8700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67340682f0 0x7f673406a7a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.193+0000 7f674e8a8700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f674806f850 0x7f6748143e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.193+0000 7f674e8a8700 1 --2- 172.21.15.156:0/1828187071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67480cd8c0 0x7f6748140ff0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.193+0000 7f674e8a8700 1 -- 172.21.15.156:0/1828187071 >> 172.21.15.156:0/1828187071 conn(0x7f674806aed0 msgr2=0x7f67480cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.193+0000 7f674e8a8700 1 -- 172.21.15.156:0/1828187071 shutdown_connections 2023-11-28T11:09:46.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.193+0000 7f674e8a8700 1 -- 172.21.15.156:0/1828187071 wait complete. 2023-11-28T11:09:46.206 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T11:09:46.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.326+0000 7f770717d700 1 Processor -- start 2023-11-28T11:09:46.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.326+0000 7f770717d700 1 -- start start 2023-11-28T11:09:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.327+0000 7f770717d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f77000cb0e0 0x7f7700065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.327+0000 7f770717d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7700066580 con 0x7f77000cec70 2023-11-28T11:09:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.327+0000 7f770717d700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77000cded0 con 0x7f770006ec90 2023-11-28T11:09:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.327+0000 7f770717d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77000ce1b0 con 0x7f77000cb0e0 2023-11-28T11:09:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.327+0000 7f76fffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f77000cb0e0 0x7f7700065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.327+0000 7f770571a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f77000cec70 0x7f770006d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49258/0 (socket says 172.21.15.156:49258) 2023-11-28T11:09:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.327+0000 7f770571a700 1 -- 172.21.15.156:0/2210461018 learned_addr learned my addr 172.21.15.156:0/2210461018 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.327+0000 7f76fffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f77000cb0e0 0x7f7700065ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34378/0 (socket says 172.21.15.156:34378) 2023-11-28T11:09:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.327+0000 7f76fffff700 1 -- 172.21.15.156:0/2210461018 >> v1:172.21.15.186:6789/0 conn(0x7f770006ec90 legacy=0x7f77000ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.327+0000 7f76fffff700 1 -- 172.21.15.156:0/2210461018 >> v1:172.21.15.156:6789/0 conn(0x7f77000cec70 legacy=0x7f770006d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.327+0000 7f76fffff700 1 -- 172.21.15.156:0/2210461018 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77000ce490 con 0x7f77000cb0e0 2023-11-28T11:09:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.327+0000 7f76ff7fe700 1 -- 172.21.15.156:0/2210461018 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116145005 0 0) 0x7f7700066580 con 0x7f77000cec70 2023-11-28T11:09:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.328+0000 7f76fffff700 1 --2- 172.21.15.156:0/2210461018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77000cb0e0 0x7f7700065ef0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009840 tx=0x7f76ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517b485c23ab58c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.328+0000 7f76ff7fe700 1 -- 172.21.15.156:0/2210461018 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec011830 con 0x7f77000cb0e0 2023-11-28T11:09:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.328+0000 7f76ff7fe700 1 -- 172.21.15.156:0/2210461018 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec011990 con 0x7f77000cb0e0 2023-11-28T11:09:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.328+0000 7f76ff7fe700 1 -- 172.21.15.156:0/2210461018 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec011d20 con 0x7f77000cb0e0 2023-11-28T11:09:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.328+0000 7f770717d700 1 -- 172.21.15.156:0/2210461018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77000cb0e0 msgr2=0x7f7700065ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.328+0000 7f770717d700 1 --2- 172.21.15.156:0/2210461018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77000cb0e0 0x7f7700065ef0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009840 tx=0x7f76ec009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.328+0000 7f770717d700 1 -- 172.21.15.156:0/2210461018 shutdown_connections 2023-11-28T11:09:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.328+0000 7f770717d700 1 --2- 172.21.15.156:0/2210461018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77000cb0e0 0x7f7700065ef0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.328+0000 7f770717d700 1 -- 172.21.15.156:0/2210461018 >> 172.21.15.156:0/2210461018 conn(0x7f77000b12b0 msgr2=0x7f77000b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.328+0000 7f770717d700 1 -- 172.21.15.156:0/2210461018 shutdown_connections 2023-11-28T11:09:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.328+0000 7f770717d700 1 -- 172.21.15.156:0/2210461018 wait complete. 2023-11-28T11:09:46.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.329+0000 7f770717d700 1 Processor -- start 2023-11-28T11:09:46.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.329+0000 7f770717d700 1 -- start start 2023-11-28T11:09:46.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.330+0000 7f770717d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f770006ec90 0x7f7700153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.330+0000 7f770717d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77000cb0e0 0x7f77000adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.330+0000 7f770717d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77000cec70 0x7f77001f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.330+0000 7f770717d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77001f3770 con 0x7f77000cb0e0 2023-11-28T11:09:46.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.330+0000 7f770717d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77000b4ca0 con 0x7f770006ec90 2023-11-28T11:09:46.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.330+0000 7f770717d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77000acce0 con 0x7f77000cec70 2023-11-28T11:09:46.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f76fffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77000cb0e0 0x7f77000adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f76fffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77000cb0e0 0x7f77000adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34384/0 (socket says 172.21.15.156:34384) 2023-11-28T11:09:46.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f76fffff700 1 -- 172.21.15.156:0/893812529 learned_addr learned my addr 172.21.15.156:0/893812529 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:46.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f7704f19700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f770006ec90 0x7f7700153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f76fffff700 1 -- 172.21.15.156:0/893812529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77000cec70 msgr2=0x7f77001f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f76fffff700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77000cec70 0x7f77001f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f76fffff700 1 -- 172.21.15.156:0/893812529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f770006ec90 msgr2=0x7f7700153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f76fffff700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f770006ec90 0x7f7700153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f76fffff700 1 -- 172.21.15.156:0/893812529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec0094a0 con 0x7f77000cb0e0 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f7704f19700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f770006ec90 0x7f7700153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f76fffff700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77000cb0e0 0x7f77000adaa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0090b0 tx=0x7f76ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f76fd7fa700 1 -- 172.21.15.156:0/893812529 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec0036a0 con 0x7f77000cb0e0 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f76fd7fa700 1 -- 172.21.15.156:0/893812529 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec01f070 con 0x7f77000cb0e0 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f770717d700 1 -- 172.21.15.156:0/893812529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77000cb6c0 con 0x7f77000cb0e0 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f76fd7fa700 1 -- 172.21.15.156:0/893812529 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec003a00 con 0x7f77000cb0e0 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.331+0000 7f770717d700 1 -- 172.21.15.156:0/893812529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77000ce770 con 0x7f77000cb0e0 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.332+0000 7f770717d700 1 -- 172.21.15.156:0/893812529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770006f060 con 0x7f77000cb0e0 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.333+0000 7f76fd7fa700 1 -- 172.21.15.156:0/893812529 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f76ec00f050 con 0x7f77000cb0e0 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.333+0000 7f76fd7fa700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f76e40682f0 0x7f76e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.333+0000 7f76fd7fa700 1 -- 172.21.15.156:0/893812529 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f76ec021070 con 0x7f77000cb0e0 2023-11-28T11:09:46.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.333+0000 7f7704f19700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f76e40682f0 0x7f76e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.333+0000 7f7704f19700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f76e40682f0 0x7f76e406a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f77000b37c0 tx=0x7f76f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.337+0000 7f76fd7fa700 1 -- 172.21.15.156:0/893812529 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f76ec053fc0 con 0x7f77000cb0e0 2023-11-28T11:09:46.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.595+0000 7f770717d700 1 -- 172.21.15.156:0/893812529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f77000cf110 con 0x7f77000cb0e0 2023-11-28T11:09:46.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.848+0000 7f76fd7fa700 1 -- 172.21.15.156:0/893812529 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v97) v1 ==== 142+0+0 (secure 0 0 0) 0x7f76ec0575e0 con 0x7f77000cb0e0 2023-11-28T11:09:46.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.850+0000 7f770717d700 1 -- 172.21.15.156:0/893812529 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f76e40682f0 msgr2=0x7f76e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.850+0000 7f770717d700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f76e40682f0 0x7f76e406a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f77000b37c0 tx=0x7f76f4009040 comp rx=0 tx=0).stop 2023-11-28T11:09:46.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.850+0000 7f770717d700 1 -- 172.21.15.156:0/893812529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77000cb0e0 msgr2=0x7f77000adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.850+0000 7f770717d700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77000cb0e0 0x7f77000adaa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0090b0 tx=0x7f76ec004750 comp rx=0 tx=0).stop 2023-11-28T11:09:46.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.850+0000 7f770717d700 1 -- 172.21.15.156:0/893812529 shutdown_connections 2023-11-28T11:09:46.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.850+0000 7f770717d700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f76e40682f0 0x7f76e406a7a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.850+0000 7f770717d700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77000cec70 0x7f77001f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.850+0000 7f770717d700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f770006ec90 0x7f7700153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.850+0000 7f770717d700 1 --2- 172.21.15.156:0/893812529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77000cb0e0 0x7f77000adaa0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.850+0000 7f770717d700 1 -- 172.21.15.156:0/893812529 >> 172.21.15.156:0/893812529 conn(0x7f77000b12b0 msgr2=0x7f770015ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.850+0000 7f770717d700 1 -- 172.21.15.156:0/893812529 shutdown_connections 2023-11-28T11:09:46.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.850+0000 7f770717d700 1 -- 172.21.15.156:0/893812529 wait complete. 2023-11-28T11:09:46.864 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-28T11:09:46.864 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2023-11-28T11:09:46.864 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:09:46.864 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T11:09:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.984+0000 7fcc86554700 1 Processor -- start 2023-11-28T11:09:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.984+0000 7fcc86554700 1 -- start start 2023-11-28T11:09:46.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc86554700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcc80071040 0x7fcc80071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc86554700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc80071950 con 0x7fcc8006dee0 2023-11-28T11:09:46.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc86554700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8015d060 con 0x7fcc80160540 2023-11-28T11:09:46.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc86554700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8015d340 con 0x7fcc80071040 2023-11-28T11:09:46.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7f7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fcc8006dee0 0x7fcc8006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49268/0 (socket says 172.21.15.156:49268) 2023-11-28T11:09:46.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7f7fe700 1 -- 172.21.15.156:0/410253942 learned_addr learned my addr 172.21.15.156:0/410253942 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:46.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7ffff700 1 --2- 172.21.15.156:0/410253942 >> v2:172.21.15.186:3300/0 conn(0x7fcc80071040 0x7fcc80071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056518020 0 0) 0x7fcc80071950 con 0x7fcc8006dee0 2023-11-28T11:09:46.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc680035b0 con 0x7fcc8006dee0 2023-11-28T11:09:46.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130213114 0 0) 0x7fcc8015d060 con 0x7fcc80160540 2023-11-28T11:09:46.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc80071950 con 0x7fcc80160540 2023-11-28T11:09:46.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3407159848 0 0) 0x7fcc680035b0 con 0x7fcc8006dee0 2023-11-28T11:09:46.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 >> v2:172.21.15.186:3300/0 conn(0x7fcc80071040 msgr2=0x7fcc80071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:46.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7effd700 1 --2- 172.21.15.156:0/410253942 >> v2:172.21.15.186:3300/0 conn(0x7fcc80071040 0x7fcc80071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 >> v1:172.21.15.186:6789/0 conn(0x7fcc80160540 legacy=0x7fcc80160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8015d620 con 0x7fcc8006dee0 2023-11-28T11:09:46.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.985+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fcc70001670 con 0x7fcc8006dee0 2023-11-28T11:09:46.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 >> v1:172.21.15.156:6789/0 conn(0x7fcc8006dee0 legacy=0x7fcc8006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7effd700 1 --2- 172.21.15.156:0/410253942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc68003b90 0x7fcc68003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7effd700 1 --2- 172.21.15.156:0/410253942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc68004850 0x7fcc68004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7effd700 1 --2- 172.21.15.156:0/410253942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc68005270 0x7fcc6800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8015d340 con 0x7fcc68004850 2023-11-28T11:09:46.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6800c1b0 con 0x7fcc68005270 2023-11-28T11:09:46.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6800c370 con 0x7fcc68003b90 2023-11-28T11:09:46.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7f7fe700 1 --2- 172.21.15.156:0/410253942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc68004850 0x7fcc68004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc84af1700 1 --2- 172.21.15.156:0/410253942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc68005270 0x7fcc6800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7ffff700 1 --2- 172.21.15.156:0/410253942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc68003b90 0x7fcc68003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7f7fe700 1 -- 172.21.15.156:0/410253942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc68003b90 msgr2=0x7fcc68003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:46.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7f7fe700 1 --2- 172.21.15.156:0/410253942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc68003b90 0x7fcc68003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7f7fe700 1 -- 172.21.15.156:0/410253942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc68005270 msgr2=0x7fcc6800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:46.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7f7fe700 1 --2- 172.21.15.156:0/410253942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc68005270 0x7fcc6800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7f7fe700 1 -- 172.21.15.156:0/410253942 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680035b0 con 0x7fcc68004850 2023-11-28T11:09:46.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc84af1700 1 --2- 172.21.15.156:0/410253942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc68005270 0x7fcc6800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:46.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc84af1700 1 -- 172.21.15.156:0/410253942 reap_dead start 2023-11-28T11:09:46.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7f7fe700 1 --2- 172.21.15.156:0/410253942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc68004850 0x7fcc68004cc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000c210 tx=0x7fcc68005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907af9a1314beb48 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7000cd50 con 0x7fcc68004850 2023-11-28T11:09:46.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc70001af0 con 0x7fcc68004850 2023-11-28T11:09:46.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.986+0000 7fcc7effd700 1 -- 172.21.15.156:0/410253942 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70005ba0 con 0x7fcc68004850 2023-11-28T11:09:46.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.987+0000 7fcc86554700 1 -- 172.21.15.156:0/410253942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc68004850 msgr2=0x7fcc68004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.987+0000 7fcc86554700 1 --2- 172.21.15.156:0/410253942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc68004850 0x7fcc68004cc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000c210 tx=0x7fcc68005880 comp rx=0 tx=0).stop 2023-11-28T11:09:46.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.987+0000 7fcc86554700 1 -- 172.21.15.156:0/410253942 shutdown_connections 2023-11-28T11:09:46.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.987+0000 7fcc86554700 1 --2- 172.21.15.156:0/410253942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc68004850 0x7fcc68004cc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.987+0000 7fcc86554700 1 -- 172.21.15.156:0/410253942 >> 172.21.15.156:0/410253942 conn(0x7fcc800d0000 msgr2=0x7fcc800d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.987+0000 7fcc86554700 1 -- 172.21.15.156:0/410253942 shutdown_connections 2023-11-28T11:09:46.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.987+0000 7fcc86554700 1 -- 172.21.15.156:0/410253942 wait complete. 2023-11-28T11:09:46.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.988+0000 7fcc86554700 1 Processor -- start 2023-11-28T11:09:46.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.988+0000 7fcc86554700 1 -- start start 2023-11-28T11:09:46.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc86554700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc68004850 0x7fcc80148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc86554700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc801460f0 0x7fcc80145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc86554700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc800cf470 0x7fcc800cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc86554700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc800cfe30 con 0x7fcc801460f0 2023-11-28T11:09:46.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc86554700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc800b45f0 con 0x7fcc800cf470 2023-11-28T11:09:46.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc86554700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc800c6a90 con 0x7fcc68004850 2023-11-28T11:09:46.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc7f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc801460f0 0x7fcc80145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc7f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc801460f0 0x7fcc80145810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34404/0 (socket says 172.21.15.156:34404) 2023-11-28T11:09:46.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc7f7fe700 1 -- 172.21.15.156:0/534005880 learned_addr learned my addr 172.21.15.156:0/534005880 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:46.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc7ffff700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc68004850 0x7fcc80148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc84af1700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc800cf470 0x7fcc800cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc7f7fe700 1 -- 172.21.15.156:0/534005880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc68004850 msgr2=0x7fcc80148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc7f7fe700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc68004850 0x7fcc80148b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc7f7fe700 1 -- 172.21.15.156:0/534005880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc800cf470 msgr2=0x7fcc800cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc7f7fe700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc800cf470 0x7fcc800cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc7f7fe700 1 -- 172.21.15.156:0/534005880 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8015ccd0 con 0x7fcc801460f0 2023-11-28T11:09:46.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.989+0000 7fcc7f7fe700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc801460f0 0x7fcc80145810 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800c180 tx=0x7fcc70001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.990+0000 7fcc7cff9700 1 -- 172.21.15.156:0/534005880 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc700072d0 con 0x7fcc801460f0 2023-11-28T11:09:46.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.990+0000 7fcc7cff9700 1 -- 172.21.15.156:0/534005880 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc70007430 con 0x7fcc801460f0 2023-11-28T11:09:46.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.990+0000 7fcc7cff9700 1 -- 172.21.15.156:0/534005880 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc700077e0 con 0x7fcc801460f0 2023-11-28T11:09:47.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.990+0000 7fcc86554700 1 -- 172.21.15.156:0/534005880 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8015d900 con 0x7fcc801460f0 2023-11-28T11:09:47.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.990+0000 7fcc86554700 1 -- 172.21.15.156:0/534005880 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8006a270 con 0x7fcc801460f0 2023-11-28T11:09:47.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.991+0000 7fcc86554700 1 -- 172.21.15.156:0/534005880 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8006ee60 con 0x7fcc801460f0 2023-11-28T11:09:47.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.991+0000 7fcc7cff9700 1 -- 172.21.15.156:0/534005880 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcc70010070 con 0x7fcc801460f0 2023-11-28T11:09:47.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.991+0000 7fcc7cff9700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcc6c0682f0 0x7fcc6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.991+0000 7fcc7cff9700 1 -- 172.21.15.156:0/534005880 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcc70086050 con 0x7fcc801460f0 2023-11-28T11:09:47.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.991+0000 7fcc7ffff700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcc6c0682f0 0x7fcc6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.992+0000 7fcc7ffff700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcc6c0682f0 0x7fcc6c06a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009b00 tx=0x7fcc74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:46.996+0000 7fcc7cff9700 1 -- 172.21.15.156:0/534005880 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcc70052bb0 con 0x7fcc801460f0 2023-11-28T11:09:47.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.254+0000 7fcc86554700 1 -- 172.21.15.156:0/534005880 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fcc800720a0 con 0x7fcc801460f0 2023-11-28T11:09:47.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.849+0000 7fcc7cff9700 1 -- 172.21.15.156:0/534005880 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v98) v1 ==== 152+0+0 (secure 0 0 0) 0x7fcc700561d0 con 0x7fcc801460f0 2023-11-28T11:09:47.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.851+0000 7fcc86554700 1 -- 172.21.15.156:0/534005880 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcc6c0682f0 msgr2=0x7fcc6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.851+0000 7fcc86554700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcc6c0682f0 0x7fcc6c06a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009b00 tx=0x7fcc74008040 comp rx=0 tx=0).stop 2023-11-28T11:09:47.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.851+0000 7fcc86554700 1 -- 172.21.15.156:0/534005880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc801460f0 msgr2=0x7fcc80145810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.851+0000 7fcc86554700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc801460f0 0x7fcc80145810 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800c180 tx=0x7fcc70001980 comp rx=0 tx=0).stop 2023-11-28T11:09:47.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.851+0000 7fcc86554700 1 -- 172.21.15.156:0/534005880 shutdown_connections 2023-11-28T11:09:47.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.851+0000 7fcc86554700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc800cf470 0x7fcc800cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.851+0000 7fcc86554700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcc6c0682f0 0x7fcc6c06a7a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.851+0000 7fcc86554700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc68004850 0x7fcc80148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.851+0000 7fcc86554700 1 --2- 172.21.15.156:0/534005880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc801460f0 0x7fcc80145810 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.851+0000 7fcc86554700 1 -- 172.21.15.156:0/534005880 >> 172.21.15.156:0/534005880 conn(0x7fcc800d0000 msgr2=0x7fcc800cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:47.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.852+0000 7fcc86554700 1 -- 172.21.15.156:0/534005880 shutdown_connections 2023-11-28T11:09:47.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.852+0000 7fcc86554700 1 -- 172.21.15.156:0/534005880 wait complete. 2023-11-28T11:09:47.866 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:09:47.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.989+0000 7fd814184700 1 Processor -- start 2023-11-28T11:09:47.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.990+0000 7fd814184700 1 -- start start 2023-11-28T11:09:47.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.990+0000 7fd814184700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd80c0ce040 0x7fd80c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.990+0000 7fd814184700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80c06b420 con 0x7fd80c0d0a00 2023-11-28T11:09:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.990+0000 7fd814184700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80c06b700 con 0x7fd80c0b2930 2023-11-28T11:09:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.990+0000 7fd814184700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd80c06ff60 con 0x7fd80c0ce040 2023-11-28T11:09:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd81171f700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd80c0d0a00 0x7fd80c06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49288/0 (socket says 172.21.15.156:49288) 2023-11-28T11:09:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd81171f700 1 -- 172.21.15.156:0/3008390223 learned_addr learned my addr 172.21.15.156:0/3008390223 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888608700 0 0) 0x7fd80c06b420 con 0x7fd80c0d0a00 2023-11-28T11:09:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7f80035b0 con 0x7fd80c0d0a00 2023-11-28T11:09:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd811f20700 1 --2- 172.21.15.156:0/3008390223 >> v2:172.21.15.186:3300/0 conn(0x7fd80c0ce040 0x7fd80c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2952926514 0 0) 0x7fd7f80035b0 con 0x7fd80c0d0a00 2023-11-28T11:09:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 >> v2:172.21.15.186:3300/0 conn(0x7fd80c0ce040 msgr2=0x7fd80c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 --2- 172.21.15.156:0/3008390223 >> v2:172.21.15.186:3300/0 conn(0x7fd80c0ce040 0x7fd80c15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 >> v1:172.21.15.186:6789/0 conn(0x7fd80c0b2930 legacy=0x7fd80c06c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c06b9e0 con 0x7fd80c0d0a00 2023-11-28T11:09:47.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fd808001670 con 0x7fd80c0d0a00 2023-11-28T11:09:47.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 >> v1:172.21.15.156:6789/0 conn(0x7fd80c0d0a00 legacy=0x7fd80c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 --2- 172.21.15.156:0/3008390223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7f8003a00 0x7fd7f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 --2- 172.21.15.156:0/3008390223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7f8004850 0x7fd7f8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 --2- 172.21.15.156:0/3008390223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 0x7fd7f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c06ff60 con 0x7fd7f8005200 2023-11-28T11:09:47.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f800c190 con 0x7fd7f8004850 2023-11-28T11:09:47.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f800c350 con 0x7fd7f8003a00 2023-11-28T11:09:47.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd812721700 1 --2- 172.21.15.156:0/3008390223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 0x7fd7f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd81171f700 1 --2- 172.21.15.156:0/3008390223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7f8004850 0x7fd7f8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.991+0000 7fd811f20700 1 --2- 172.21.15.156:0/3008390223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7f8003a00 0x7fd7f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd812721700 1 -- 172.21.15.156:0/3008390223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7f8003a00 msgr2=0x7fd7f8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd812721700 1 --2- 172.21.15.156:0/3008390223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7f8003a00 0x7fd7f8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd812721700 1 -- 172.21.15.156:0/3008390223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7f8004850 msgr2=0x7fd7f8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd812721700 1 --2- 172.21.15.156:0/3008390223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7f8004850 0x7fd7f8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd812721700 1 -- 172.21.15.156:0/3008390223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80035b0 con 0x7fd7f8005200 2023-11-28T11:09:47.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd812721700 1 -- 172.21.15.156:0/3008390223 reap_dead start 2023-11-28T11:09:47.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd812721700 1 --2- 172.21.15.156:0/3008390223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 0x7fd7f800ba10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd80400c160 tx=0x7fd7f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eafde8bfdba774 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804001db0 con 0x7fd7f8005200 2023-11-28T11:09:47.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80400cca0 con 0x7fd7f8005200 2023-11-28T11:09:47.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd810f1e700 1 -- 172.21.15.156:0/3008390223 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804003c20 con 0x7fd7f8005200 2023-11-28T11:09:47.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd814184700 1 -- 172.21.15.156:0/3008390223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 msgr2=0x7fd7f800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd814184700 1 --2- 172.21.15.156:0/3008390223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 0x7fd7f800ba10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd80400c160 tx=0x7fd7f8005880 comp rx=0 tx=0).stop 2023-11-28T11:09:47.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd814184700 1 -- 172.21.15.156:0/3008390223 shutdown_connections 2023-11-28T11:09:47.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd814184700 1 --2- 172.21.15.156:0/3008390223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 0x7fd7f800ba10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd814184700 1 -- 172.21.15.156:0/3008390223 >> 172.21.15.156:0/3008390223 conn(0x7fd80c06aed0 msgr2=0x7fd80c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:47.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd814184700 1 -- 172.21.15.156:0/3008390223 shutdown_connections 2023-11-28T11:09:47.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.992+0000 7fd814184700 1 -- 172.21.15.156:0/3008390223 wait complete. 2023-11-28T11:09:47.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.994+0000 7fd814184700 1 Processor -- start 2023-11-28T11:09:47.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.994+0000 7fd814184700 1 -- start start 2023-11-28T11:09:48.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.994+0000 7fd814184700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 0x7fd80c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.994+0000 7fd814184700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd80c1f50e0 0x7fd80c0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.994+0000 7fd814184700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd80c15ed00 0x7fd80c15f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.994+0000 7fd814184700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c0c5260 con 0x7fd7f8005200 2023-11-28T11:09:48.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.994+0000 7fd814184700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c15f6e0 con 0x7fd80c15ed00 2023-11-28T11:09:48.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.994+0000 7fd814184700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c0c84c0 con 0x7fd80c1f50e0 2023-11-28T11:09:48.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.994+0000 7fd811f20700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 0x7fd80c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.994+0000 7fd811f20700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 0x7fd80c0cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34422/0 (socket says 172.21.15.156:34422) 2023-11-28T11:09:48.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.994+0000 7fd811f20700 1 -- 172.21.15.156:0/3636678275 learned_addr learned my addr 172.21.15.156:0/3636678275 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:48.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd811f20700 1 -- 172.21.15.156:0/3636678275 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd80c1f50e0 msgr2=0x7fd80c0c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd811f20700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd80c1f50e0 0x7fd80c0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd812721700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd80c15ed00 0x7fd80c15f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd811f20700 1 -- 172.21.15.156:0/3636678275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd80c15ed00 msgr2=0x7fd80c15f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd811f20700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd80c15ed00 0x7fd80c15f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd811f20700 1 -- 172.21.15.156:0/3636678275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c06bfa0 con 0x7fd7f8005200 2023-11-28T11:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd812721700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd80c15ed00 0x7fd80c15f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd811f20700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 0x7fd80c0cc8c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0115e0 tx=0x7fd7fc0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd8027fc700 1 -- 172.21.15.156:0/3636678275 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc004440 con 0x7fd7f8005200 2023-11-28T11:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd8027fc700 1 -- 172.21.15.156:0/3636678275 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc0045a0 con 0x7fd7f8005200 2023-11-28T11:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd814184700 1 -- 172.21.15.156:0/3636678275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80c06bcc0 con 0x7fd7f8005200 2023-11-28T11:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd8027fc700 1 -- 172.21.15.156:0/3636678275 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0038c0 con 0x7fd7f8005200 2023-11-28T11:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.995+0000 7fd814184700 1 -- 172.21.15.156:0/3636678275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80c15df10 con 0x7fd7f8005200 2023-11-28T11:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.996+0000 7fd814184700 1 -- 172.21.15.156:0/3636678275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80c068690 con 0x7fd7f8005200 2023-11-28T11:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.996+0000 7fd8027fc700 1 -- 172.21.15.156:0/3636678275 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd7fc025070 con 0x7fd7f8005200 2023-11-28T11:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.997+0000 7fd8027fc700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7f00682f0 0x7fd7f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.997+0000 7fd8027fc700 1 -- 172.21.15.156:0/3636678275 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd7fc085d50 con 0x7fd7f8005200 2023-11-28T11:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.997+0000 7fd81171f700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7f00682f0 0x7fd7f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:47.997+0000 7fd81171f700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7f00682f0 0x7fd7f006a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd8080011e0 tx=0x7fd808001170 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.001+0000 7fd8027fc700 1 -- 172.21.15.156:0/3636678275 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd7fc0528b0 con 0x7fd7f8005200 2023-11-28T11:09:48.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.184+0000 7fd814184700 1 -- 172.21.15.156:0/3636678275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd80c0cede0 con 0x7fd7f8005200 2023-11-28T11:09:48.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.188+0000 7fd8027fc700 1 -- 172.21.15.156:0/3636678275 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd7fc055ed0 con 0x7fd7f8005200 2023-11-28T11:09:48.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.190+0000 7fd814184700 1 -- 172.21.15.156:0/3636678275 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7f00682f0 msgr2=0x7fd7f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.190+0000 7fd814184700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7f00682f0 0x7fd7f006a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd8080011e0 tx=0x7fd808001170 comp rx=0 tx=0).stop 2023-11-28T11:09:48.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.190+0000 7fd814184700 1 -- 172.21.15.156:0/3636678275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 msgr2=0x7fd80c0cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.190+0000 7fd814184700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 0x7fd80c0cc8c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0115e0 tx=0x7fd7fc0119a0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.190+0000 7fd814184700 1 -- 172.21.15.156:0/3636678275 shutdown_connections 2023-11-28T11:09:48.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.190+0000 7fd814184700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd80c15ed00 0x7fd80c15f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.190+0000 7fd814184700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7f8005200 0x7fd80c0cc8c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.190+0000 7fd814184700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7f00682f0 0x7fd7f006a7a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.190+0000 7fd814184700 1 --2- 172.21.15.156:0/3636678275 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd80c1f50e0 0x7fd80c0c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.190+0000 7fd814184700 1 -- 172.21.15.156:0/3636678275 >> 172.21.15.156:0/3636678275 conn(0x7fd80c06aed0 msgr2=0x7fd80c15c260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:48.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.190+0000 7fd814184700 1 -- 172.21.15.156:0/3636678275 shutdown_connections 2023-11-28T11:09:48.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.190+0000 7fd814184700 1 -- 172.21.15.156:0/3636678275 wait complete. 2023-11-28T11:09:48.203 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:09:48.203 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:09:48.243 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:09:48.288 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:09:48.288 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:09:48.288 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:48.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2550 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:09:48.306 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:09:48.306 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:48.348 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:09:48.348 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:09:48.348 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:48.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2550 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:09:48.365 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:09:48.365 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:48.407 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:09:48.407 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:09:48.407 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:48.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.526+0000 7f8cc4dd9700 1 Processor -- start 2023-11-28T11:09:48.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.526+0000 7f8cc4dd9700 1 -- start start 2023-11-28T11:09:48.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cc4dd9700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8cc00cd8c0 0x7f8cc0140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cc4dd9700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8cc006f850 0x7f8cc006fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cc4dd9700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc013ff00 con 0x7f8cc00cdc90 2023-11-28T11:09:48.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cc4dd9700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc01404c0 con 0x7f8cc006f850 2023-11-28T11:09:48.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cc4dd9700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc015aa80 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cbed9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8cc006f850 0x7f8cc006fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cbed9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8cc006f850 0x7f8cc006fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34434/0 (socket says 172.21.15.156:34434) 2023-11-28T11:09:48.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cbed9d700 1 -- 172.21.15.156:0/3826726992 learned_addr learned my addr 172.21.15.156:0/3826726992 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:48.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cbe59c700 1 --2- 172.21.15.156:0/3826726992 >> v2:172.21.15.186:3300/0 conn(0x7f8cc00cd8c0 0x7f8cc0140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cbed9d700 1 -- 172.21.15.156:0/3826726992 >> v2:172.21.15.186:3300/0 conn(0x7f8cc00cd8c0 msgr2=0x7f8cc0140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:48.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cbed9d700 1 --2- 172.21.15.156:0/3826726992 >> v2:172.21.15.186:3300/0 conn(0x7f8cc00cd8c0 0x7f8cc0140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cbed9d700 1 -- 172.21.15.156:0/3826726992 >> v1:172.21.15.186:6789/0 conn(0x7f8cc00cdc90 legacy=0x7f8cc00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:48.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cbed9d700 1 -- 172.21.15.156:0/3826726992 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00b3040 con 0x7f8cc006f850 2023-11-28T11:09:48.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.527+0000 7f8cbe59c700 1 --2- 172.21.15.156:0/3826726992 >> v2:172.21.15.186:3300/0 conn(0x7f8cc00cd8c0 0x7f8cc0140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:48.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.528+0000 7f8cbed9d700 1 --2- 172.21.15.156:0/3826726992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc006f850 0x7f8cc006fcf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0070a20 tx=0x7f8cb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f68d335b87860f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.528+0000 7f8cbd59a700 1 -- 172.21.15.156:0/3826726992 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800e070 con 0x7f8cc006f850 2023-11-28T11:09:48.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.528+0000 7f8cbd59a700 1 -- 172.21.15.156:0/3826726992 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb8003a20 con 0x7f8cc006f850 2023-11-28T11:09:48.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.528+0000 7f8cbd59a700 1 -- 172.21.15.156:0/3826726992 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8003dd0 con 0x7f8cc006f850 2023-11-28T11:09:48.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.528+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3826726992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc006f850 msgr2=0x7f8cc006fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.528+0000 7f8cc4dd9700 1 --2- 172.21.15.156:0/3826726992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc006f850 0x7f8cc006fcf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0070a20 tx=0x7f8cb800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:48.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.528+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3826726992 shutdown_connections 2023-11-28T11:09:48.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.528+0000 7f8cc4dd9700 1 --2- 172.21.15.156:0/3826726992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc006f850 0x7f8cc006fcf0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.528+0000 7f8cc4dd9700 1 --2- 172.21.15.156:0/3826726992 >> v2:172.21.15.186:3300/0 conn(0x7f8cc00cd8c0 0x7f8cc0140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.528+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3826726992 >> 172.21.15.156:0/3826726992 conn(0x7f8cc006aed0 msgr2=0x7f8cc006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:48.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.528+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3826726992 shutdown_connections 2023-11-28T11:09:48.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.528+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3826726992 wait complete. 2023-11-28T11:09:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.530+0000 7f8cc4dd9700 1 Processor -- start 2023-11-28T11:09:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.530+0000 7f8cc4dd9700 1 -- start start 2023-11-28T11:09:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.530+0000 7f8cc4dd9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cc006f850 0x7f8cc0151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.530+0000 7f8cc4dd9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc00cd8c0 0x7f8cc00b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.530+0000 7f8cc4dd9700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cc00cdc90 0x7f8cc0158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.530+0000 7f8cc4dd9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00b5350 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.530+0000 7f8cc4dd9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00b4d90 con 0x7f8cc006f850 2023-11-28T11:09:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.530+0000 7f8cc4dd9700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00b5070 con 0x7f8cc00cdc90 2023-11-28T11:09:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cbdd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc00cd8c0 0x7f8cc00b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cbdd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc00cd8c0 0x7f8cc00b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34438/0 (socket says 172.21.15.156:34438) 2023-11-28T11:09:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cbdd9b700 1 -- 172.21.15.156:0/3024753288 learned_addr learned my addr 172.21.15.156:0/3024753288 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cbe59c700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cc006f850 0x7f8cc0151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cbed9d700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cc00cdc90 0x7f8cc0158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cbdd9b700 1 -- 172.21.15.156:0/3024753288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cc00cdc90 msgr2=0x7f8cc0158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cbdd9b700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cc00cdc90 0x7f8cc0158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cbdd9b700 1 -- 172.21.15.156:0/3024753288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cc006f850 msgr2=0x7f8cc0151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cbdd9b700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cc006f850 0x7f8cc0151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cbdd9b700 1 -- 172.21.15.156:0/3024753288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb8009b30 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cbe59c700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cc006f850 0x7f8cc0151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cbdd9b700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc00cd8c0 0x7f8cc00b42d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000b450 tx=0x7f8cb000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8caeffd700 1 -- 172.21.15.156:0/3024753288 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0011830 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8caeffd700 1 -- 172.21.15.156:0/3024753288 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb0011990 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8caeffd700 1 -- 172.21.15.156:0/3024753288 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0011d40 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3024753288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc00b35f0 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.531+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3024753288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc00b3310 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.532+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3024753288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc0140150 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.533+0000 7f8caeffd700 1 -- 172.21.15.156:0/3024753288 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8cb000f050 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.533+0000 7f8caeffd700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8ca00682f0 0x7f8ca006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.533+0000 7f8caeffd700 1 -- 172.21.15.156:0/3024753288 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8cb0085530 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.533+0000 7f8cbe59c700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8ca00682f0 0x7f8ca006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.533+0000 7f8cbe59c700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8ca00682f0 0x7f8ca006a7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00cf5a0 tx=0x7f8cb4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.537+0000 7f8caeffd700 1 -- 172.21.15.156:0/3024753288 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8cb0052090 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.761+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3024753288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cc00c5b10 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.761+0000 7f8caeffd700 1 -- 172.21.15.156:0/3024753288 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f8cb0015df0 con 0x7f8cc00cd8c0 2023-11-28T11:09:48.763 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:48.763 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/1085440106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":1085440106},{"type":"v1","addr":"172.21.15.186:6834","nonce":1085440106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1548245207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1548245207},{"type":"v1","addr":"172.21.15.156:6836","nonce":1548245207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4882,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3102077957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":3102077957},{"type":"v1","addr":"172.21.15.156:6837","nonce":3102077957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5191,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1606663607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":1606663607},{"type":"v1","addr":"172.21.15.186:6835","nonce":1606663607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-11-28T11:09:48.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.763+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3024753288 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8ca00682f0 msgr2=0x7f8ca006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.763+0000 7f8cc4dd9700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8ca00682f0 0x7f8ca006a7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00cf5a0 tx=0x7f8cb4008040 comp rx=0 tx=0).stop 2023-11-28T11:09:48.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.763+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3024753288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc00cd8c0 msgr2=0x7f8cc00b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.763+0000 7f8cc4dd9700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc00cd8c0 0x7f8cc00b42d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000b450 tx=0x7f8cb000b810 comp rx=0 tx=0).stop 2023-11-28T11:09:48.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.763+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3024753288 shutdown_connections 2023-11-28T11:09:48.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.763+0000 7f8cc4dd9700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8ca00682f0 0x7f8ca006a7a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.763+0000 7f8cc4dd9700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cc00cdc90 0x7f8cc0158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.763+0000 7f8cc4dd9700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cc006f850 0x7f8cc0151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.763+0000 7f8cc4dd9700 1 --2- 172.21.15.156:0/3024753288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cc00cd8c0 0x7f8cc00b42d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.763+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3024753288 >> 172.21.15.156:0/3024753288 conn(0x7f8cc006aed0 msgr2=0x7f8cc013f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:48.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.763+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3024753288 shutdown_connections 2023-11-28T11:09:48.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:48.764+0000 7f8cc4dd9700 1 -- 172.21.15.156:0/3024753288 wait complete. 2023-11-28T11:09:48.768 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 18 2023-11-28T11:09:48.777 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:09:48.838 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:09:48.838 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:09:48.839 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:48.839 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:09:48.840 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:09:48.841 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:09:48.841 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:48.842 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:09:48.842 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:09:48.842 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:09:48.843 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:48.843 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:09:48.843 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:09:48.844 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:09:48.845 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:48.848 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:09:48.848 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:09:48.850 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:09:48.850 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:09:48.858 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:09:48.858 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:09:48.860 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:09:48.860 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:09:48.892 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:09:48.892 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:48.893 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:09:48.893 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:48.899 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:09:48.900 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:48.900 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:09:48.900 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:48.901 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:09:48.971 INFO:tasks.ceph.mds.a.smithi156.stdout:starting mds.a at 2023-11-28T11:09:48.971 INFO:tasks.ceph.mds.c.smithi156.stdout:starting mds.c at 2023-11-28T11:09:48.984 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2023-11-28T11:09:48.984 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-11-28T11:09:49.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.030+0000 7f6a7383b700 1 Processor -- start 2023-11-28T11:09:49.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.031+0000 7f6a7383b700 1 -- start start 2023-11-28T11:09:49.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.031+0000 7f6a7383b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a6c161340 0x7f6a6c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.031+0000 7f6a7383b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6a6c0d2320 0x7f6a6c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.031+0000 7f6a7383b700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c0d1950 con 0x7f6a6c15c9b0 2023-11-28T11:09:49.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.031+0000 7f6a7383b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c0b5800 con 0x7f6a6c0d2320 2023-11-28T11:09:49.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.031+0000 7f6a7383b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0b5300 con 0x7f6a6c161340 2023-11-28T11:09:49.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.032+0000 7f6a71dd8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6a6c0d2320 0x7f6a6c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.032+0000 7f6a71dd8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6a6c0d2320 0x7f6a6c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34472/0 (socket says 172.21.15.156:34472) 2023-11-28T11:09:49.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.032+0000 7f6a71dd8700 1 -- 172.21.15.156:0/3957597521 learned_addr learned my addr 172.21.15.156:0/3957597521 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:49.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.032+0000 7f6a71dd8700 1 -- 172.21.15.156:0/3957597521 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a6c161340 msgr2=0x7f6a6c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:49.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.032+0000 7f6a71dd8700 1 --2- 172.21.15.156:0/3957597521 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a6c161340 0x7f6a6c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.032+0000 7f6a71dd8700 1 -- 172.21.15.156:0/3957597521 >> v1:172.21.15.186:6789/0 conn(0x7f6a6c15c9b0 legacy=0x7f6a6c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:49.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.032+0000 7f6a71dd8700 1 -- 172.21.15.156:0/3957597521 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c0b5520 con 0x7f6a6c0d2320 2023-11-28T11:09:49.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.032+0000 7f6a71dd8700 1 --2- 172.21.15.156:0/3957597521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a6c0d2320 0x7f6a6c0cd7c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009e80 tx=0x7f6a5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f87aa7a09766618 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.032+0000 7f6a6bfff700 1 -- 172.21.15.156:0/3957597521 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00e070 con 0x7f6a6c0d2320 2023-11-28T11:09:49.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.032+0000 7f6a6bfff700 1 -- 172.21.15.156:0/3957597521 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c003a20 con 0x7f6a6c0d2320 2023-11-28T11:09:49.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.032+0000 7f6a6bfff700 1 -- 172.21.15.156:0/3957597521 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c003dd0 con 0x7f6a6c0d2320 2023-11-28T11:09:49.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.033+0000 7f6a7383b700 1 -- 172.21.15.156:0/3957597521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a6c0d2320 msgr2=0x7f6a6c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.033+0000 7f6a7383b700 1 --2- 172.21.15.156:0/3957597521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a6c0d2320 0x7f6a6c0cd7c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009e80 tx=0x7f6a5c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:49.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.033+0000 7f6a7383b700 1 -- 172.21.15.156:0/3957597521 shutdown_connections 2023-11-28T11:09:49.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.033+0000 7f6a7383b700 1 --2- 172.21.15.156:0/3957597521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a6c0d2320 0x7f6a6c0cd7c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.033+0000 7f6a7383b700 1 --2- 172.21.15.156:0/3957597521 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a6c161340 0x7f6a6c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.033+0000 7f6a7383b700 1 -- 172.21.15.156:0/3957597521 >> 172.21.15.156:0/3957597521 conn(0x7f6a6c072600 msgr2=0x7f6a6c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:49.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.033+0000 7f6a7383b700 1 -- 172.21.15.156:0/3957597521 shutdown_connections 2023-11-28T11:09:49.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.033+0000 7f6a7383b700 1 -- 172.21.15.156:0/3957597521 wait complete. 2023-11-28T11:09:49.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.034+0000 7f6a7383b700 1 Processor -- start 2023-11-28T11:09:49.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.034+0000 7f6a7383b700 1 -- start start 2023-11-28T11:09:49.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.035+0000 7f6a7383b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a6c0d2320 0x7f6a6c152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.035+0000 7f6a7383b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a6c15c9b0 0x7f6a6c146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.035+0000 7f6a7383b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a6c161340 0x7f6a6c0b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.035+0000 7f6a7383b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0c6a40 con 0x7f6a6c15c9b0 2023-11-28T11:09:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.035+0000 7f6a7383b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c150b80 con 0x7f6a6c0d2320 2023-11-28T11:09:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.035+0000 7f6a7383b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c153140 con 0x7f6a6c161340 2023-11-28T11:09:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.035+0000 7f6a70dd6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a6c15c9b0 0x7f6a6c146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.035+0000 7f6a70dd6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a6c15c9b0 0x7f6a6c146510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34476/0 (socket says 172.21.15.156:34476) 2023-11-28T11:09:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.035+0000 7f6a70dd6700 1 -- 172.21.15.156:0/2841225464 learned_addr learned my addr 172.21.15.156:0/2841225464 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a71dd8700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a6c161340 0x7f6a6c0b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a715d7700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a6c0d2320 0x7f6a6c152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a70dd6700 1 -- 172.21.15.156:0/2841225464 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a6c161340 msgr2=0x7f6a6c0b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a70dd6700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a6c161340 0x7f6a6c0b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a70dd6700 1 -- 172.21.15.156:0/2841225464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a6c0d2320 msgr2=0x7f6a6c152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a70dd6700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a6c0d2320 0x7f6a6c152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a70dd6700 1 -- 172.21.15.156:0/2841225464 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c009b30 con 0x7f6a6c15c9b0 2023-11-28T11:09:49.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a715d7700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a6c0d2320 0x7f6a6c152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:49.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a70dd6700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a6c15c9b0 0x7f6a6c146510 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b530 tx=0x7f6a6000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a69ffb700 1 -- 172.21.15.156:0/2841225464 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60011830 con 0x7f6a6c15c9b0 2023-11-28T11:09:49.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a69ffb700 1 -- 172.21.15.156:0/2841225464 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a600119b0 con 0x7f6a6c15c9b0 2023-11-28T11:09:49.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a7383b700 1 -- 172.21.15.156:0/2841225464 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c0d15e0 con 0x7f6a6c15c9b0 2023-11-28T11:09:49.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a69ffb700 1 -- 172.21.15.156:0/2841225464 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60011d60 con 0x7f6a6c15c9b0 2023-11-28T11:09:49.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.036+0000 7f6a7383b700 1 -- 172.21.15.156:0/2841225464 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c0d1300 con 0x7f6a6c15c9b0 2023-11-28T11:09:49.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.037+0000 7f6a7383b700 1 -- 172.21.15.156:0/2841225464 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c15cd80 con 0x7f6a6c15c9b0 2023-11-28T11:09:49.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.038+0000 7f6a69ffb700 1 -- 172.21.15.156:0/2841225464 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a60021070 con 0x7f6a6c15c9b0 2023-11-28T11:09:49.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.038+0000 7f6a69ffb700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6a580682f0 0x7f6a5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.039+0000 7f6a69ffb700 1 -- 172.21.15.156:0/2841225464 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6a60085fb0 con 0x7f6a6c15c9b0 2023-11-28T11:09:49.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.039+0000 7f6a715d7700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6a580682f0 0x7f6a5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.039+0000 7f6a715d7700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6a580682f0 0x7f6a5806a7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6a64001750 tx=0x7f6a64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.042+0000 7f6a69ffb700 1 -- 172.21.15.156:0/2841225464 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6a60052ac0 con 0x7f6a6c15c9b0 2023-11-28T11:09:49.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:49.244+0000 7f6a7383b700 1 -- 172.21.15.156:0/2841225464 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6a6c0745c0 con 0x7f6a6c15c9b0 2023-11-28T11:09:50.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.190+0000 7f6a69ffb700 1 -- 172.21.15.156:0/2841225464 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v45) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6a600560e0 con 0x7f6a6c15c9b0 2023-11-28T11:09:50.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.193+0000 7f6a7383b700 1 -- 172.21.15.156:0/2841225464 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6a580682f0 msgr2=0x7f6a5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.193+0000 7f6a7383b700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6a580682f0 0x7f6a5806a7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6a64001750 tx=0x7f6a64009040 comp rx=0 tx=0).stop 2023-11-28T11:09:50.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.193+0000 7f6a7383b700 1 -- 172.21.15.156:0/2841225464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a6c15c9b0 msgr2=0x7f6a6c146510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.193+0000 7f6a7383b700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a6c15c9b0 0x7f6a6c146510 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b530 tx=0x7f6a6000b8f0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.193+0000 7f6a7383b700 1 -- 172.21.15.156:0/2841225464 shutdown_connections 2023-11-28T11:09:50.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.193+0000 7f6a7383b700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6a580682f0 0x7f6a5806a7a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.193+0000 7f6a7383b700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a6c161340 0x7f6a6c0b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.193+0000 7f6a7383b700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a6c0d2320 0x7f6a6c152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.193+0000 7f6a7383b700 1 --2- 172.21.15.156:0/2841225464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a6c15c9b0 0x7f6a6c146510 secure :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b530 tx=0x7f6a6000b8f0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.193+0000 7f6a7383b700 1 -- 172.21.15.156:0/2841225464 >> 172.21.15.156:0/2841225464 conn(0x7f6a6c072600 msgr2=0x7f6a6c15bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.193+0000 7f6a7383b700 1 -- 172.21.15.156:0/2841225464 shutdown_connections 2023-11-28T11:09:50.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.193+0000 7f6a7383b700 1 -- 172.21.15.156:0/2841225464 wait complete. 2023-11-28T11:09:50.196 INFO:teuthology.orchestra.run.smithi156.stderr: removed all blocklist entries 2023-11-28T11:09:50.208 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:09:50.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.330+0000 7fba41a05700 1 Processor -- start 2023-11-28T11:09:50.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.330+0000 7fba41a05700 1 -- start start 2023-11-28T11:09:50.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.330+0000 7fba41a05700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba3c071040 0x7fba3c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.330+0000 7fba41a05700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fba3c160540 0x7fba3c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba41a05700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3c071950 con 0x7fba3c06dee0 2023-11-28T11:09:50.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba41a05700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba3c15d060 con 0x7fba3c160540 2023-11-28T11:09:50.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba41a05700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c15d340 con 0x7fba3c071040 2023-11-28T11:09:50.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba3b7fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fba3c160540 0x7fba3c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba3affd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba3c071040 0x7fba3c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba3a7fc700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fba3c06dee0 0x7fba3c06e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:59724/0 (socket says 172.21.15.156:59724) 2023-11-28T11:09:50.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba3affd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba3c071040 0x7fba3c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:38350/0 (socket says 172.21.15.156:38350) 2023-11-28T11:09:50.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba3a7fc700 1 -- 172.21.15.156:0/2211425070 learned_addr learned my addr 172.21.15.156:0/2211425070 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:50.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba39ffb700 1 -- 172.21.15.156:0/2211425070 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507569660 0 0) 0x7fba3c071950 con 0x7fba3c06dee0 2023-11-28T11:09:50.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba39ffb700 1 -- 172.21.15.156:0/2211425070 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba240035b0 con 0x7fba3c06dee0 2023-11-28T11:09:50.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba3affd700 1 -- 172.21.15.156:0/2211425070 >> v2:172.21.15.186:3300/0 conn(0x7fba3c160540 msgr2=0x7fba3c1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:50.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba3affd700 1 --2- 172.21.15.156:0/2211425070 >> v2:172.21.15.186:3300/0 conn(0x7fba3c160540 0x7fba3c1609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba3affd700 1 -- 172.21.15.156:0/2211425070 >> v1:172.21.15.186:6789/0 conn(0x7fba3c06dee0 legacy=0x7fba3c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.331+0000 7fba3affd700 1 -- 172.21.15.156:0/2211425070 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c15d620 con 0x7fba3c071040 2023-11-28T11:09:50.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.332+0000 7fba3affd700 1 --2- 172.21.15.156:0/2211425070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba3c071040 0x7fba3c071410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009840 tx=0x7fba2c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b64fd6878df0da4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.332+0000 7fba39ffb700 1 -- 172.21.15.156:0/2211425070 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c011830 con 0x7fba3c071040 2023-11-28T11:09:50.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.332+0000 7fba39ffb700 1 -- 172.21.15.156:0/2211425070 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c011990 con 0x7fba3c071040 2023-11-28T11:09:50.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.332+0000 7fba39ffb700 1 -- 172.21.15.156:0/2211425070 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c011d20 con 0x7fba3c071040 2023-11-28T11:09:50.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.332+0000 7fba41a05700 1 -- 172.21.15.156:0/2211425070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba3c071040 msgr2=0x7fba3c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.332+0000 7fba41a05700 1 --2- 172.21.15.156:0/2211425070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba3c071040 0x7fba3c071410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009840 tx=0x7fba2c009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:50.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.332+0000 7fba41a05700 1 -- 172.21.15.156:0/2211425070 shutdown_connections 2023-11-28T11:09:50.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.332+0000 7fba41a05700 1 --2- 172.21.15.156:0/2211425070 >> v2:172.21.15.186:3300/0 conn(0x7fba3c160540 0x7fba3c1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.332+0000 7fba41a05700 1 --2- 172.21.15.156:0/2211425070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba3c071040 0x7fba3c071410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.332+0000 7fba41a05700 1 -- 172.21.15.156:0/2211425070 >> 172.21.15.156:0/2211425070 conn(0x7fba3c0d0000 msgr2=0x7fba3c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.332+0000 7fba41a05700 1 -- 172.21.15.156:0/2211425070 shutdown_connections 2023-11-28T11:09:50.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.332+0000 7fba41a05700 1 -- 172.21.15.156:0/2211425070 wait complete. 2023-11-28T11:09:50.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.333+0000 7fba41a05700 1 Processor -- start 2023-11-28T11:09:50.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.334+0000 7fba41a05700 1 -- start start 2023-11-28T11:09:50.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.334+0000 7fba41a05700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba3c06dee0 0x7fba3c0b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.334+0000 7fba41a05700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba3c071040 0x7fba3c0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.334+0000 7fba41a05700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba3c160540 0x7fba3c0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.334+0000 7fba41a05700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0b59d0 con 0x7fba3c06dee0 2023-11-28T11:09:50.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.334+0000 7fba41a05700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0b5540 con 0x7fba3c160540 2023-11-28T11:09:50.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.334+0000 7fba41a05700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0b4930 con 0x7fba3c071040 2023-11-28T11:09:50.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba3affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba3c06dee0 0x7fba3c0b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba3affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba3c06dee0 0x7fba3c0b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34492/0 (socket says 172.21.15.156:34492) 2023-11-28T11:09:50.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba3affd700 1 -- 172.21.15.156:0/2562901638 learned_addr learned my addr 172.21.15.156:0/2562901638 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:50.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba3a7fc700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba3c071040 0x7fba3c0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba3b7fe700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba3c160540 0x7fba3c0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba3affd700 1 -- 172.21.15.156:0/2562901638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba3c071040 msgr2=0x7fba3c0b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba3affd700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba3c071040 0x7fba3c0b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba3affd700 1 -- 172.21.15.156:0/2562901638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba3c160540 msgr2=0x7fba3c0b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba3affd700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba3c160540 0x7fba3c0b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba3affd700 1 -- 172.21.15.156:0/2562901638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c0094a0 con 0x7fba3c06dee0 2023-11-28T11:09:50.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba3b7fe700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba3c160540 0x7fba3c0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:50.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba3affd700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba3c06dee0 0x7fba3c0b40e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009fd0 tx=0x7fba2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba23fff700 1 -- 172.21.15.156:0/2562901638 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0036a0 con 0x7fba3c06dee0 2023-11-28T11:09:50.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba23fff700 1 -- 172.21.15.156:0/2562901638 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c01e070 con 0x7fba3c06dee0 2023-11-28T11:09:50.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba41a05700 1 -- 172.21.15.156:0/2562901638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3c15ccd0 con 0x7fba3c06dee0 2023-11-28T11:09:50.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba23fff700 1 -- 172.21.15.156:0/2562901638 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c003a20 con 0x7fba3c06dee0 2023-11-28T11:09:50.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.335+0000 7fba41a05700 1 -- 172.21.15.156:0/2562901638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3c15d900 con 0x7fba3c06dee0 2023-11-28T11:09:50.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.336+0000 7fba41a05700 1 -- 172.21.15.156:0/2562901638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3c06a270 con 0x7fba3c06dee0 2023-11-28T11:09:50.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.337+0000 7fba23fff700 1 -- 172.21.15.156:0/2562901638 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fba2c00f050 con 0x7fba3c06dee0 2023-11-28T11:09:50.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.337+0000 7fba23fff700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fba280682f0 0x7fba2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.337+0000 7fba23fff700 1 -- 172.21.15.156:0/2562901638 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fba2c086960 con 0x7fba3c06dee0 2023-11-28T11:09:50.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.337+0000 7fba3a7fc700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fba280682f0 0x7fba2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.338+0000 7fba3a7fc700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fba280682f0 0x7fba2806a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0c3ef0 tx=0x7fba30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.341+0000 7fba23fff700 1 -- 172.21.15.156:0/2562901638 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fba2c053510 con 0x7fba3c06dee0 2023-11-28T11:09:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.595+0000 7fba41a05700 1 -- 172.21.15.156:0/2562901638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fba3c0b5820 con 0x7fba3c06dee0 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.595+0000 7fba23fff700 1 -- 172.21.15.156:0/2562901638 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fba2c020020 con 0x7fba3c06dee0 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi156.stdout: "auth_dump": [ 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.a", 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlKLGIAhAAFJHYsC+LCoq5Qq1eh+MsgA==", 2023-11-28T11:09:50.597 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.598 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:50.598 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:09:50.598 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:09:50.598 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.598 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.598 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.598 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.598 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.b", 2023-11-28T11:09:50.598 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlr6ZnDxAAnJd4CtD5MUHauaV9GVqqVA==", 2023-11-28T11:09:50.599 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.599 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:50.599 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:09:50.599 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:09:50.599 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.599 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.599 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.599 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.600 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.c", 2023-11-28T11:09:50.600 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlygniCRAAI6YYq5P02LiCGC4/tdGwqg==", 2023-11-28T11:09:50.600 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.600 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:50.600 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:09:50.600 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:09:50.600 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.600 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.600 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.600 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.600 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.d", 2023-11-28T11:09:50.600 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlc4gEFxAA6J82vSOm+mN6hvRs7Ec8SA==", 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.0", 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDsyWVlhbi+ORAAG5EKPXefX8NjPR7TNi6Wgg==", 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:50.601 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:50.602 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.602 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.602 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.602 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.602 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.1", 2023-11-28T11:09:50.602 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDvyWVlu+nEOBAAmvcXFjC+MNeWkDsZsI30PA==", 2023-11-28T11:09:50.602 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.602 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:50.602 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:50.602 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.602 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.603 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.603 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.603 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.2", 2023-11-28T11:09:50.603 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDyyWVlYMdIOBAABzlpv33IMXmgXqwROZ1n7A==", 2023-11-28T11:09:50.603 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.603 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:50.603 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:50.603 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.603 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.603 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.603 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.603 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.3", 2023-11-28T11:09:50.604 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD1yWVltreINxAAcPoQjsD28WEyEYN7kZZEUg==", 2023-11-28T11:09:50.604 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.604 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:50.604 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:50.604 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.604 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.604 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.604 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.604 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.4", 2023-11-28T11:09:50.604 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD8yWVlliukARAA3YVAwd8OePh6WT7H0Jykog==", 2023-11-28T11:09:50.604 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.605 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:50.605 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:50.605 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.605 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.605 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.605 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.605 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.5", 2023-11-28T11:09:50.605 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD/yWVl6nilARAAfFTZi6p2kdxqOgtWonRRKA==", 2023-11-28T11:09:50.605 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.605 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:50.605 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:50.605 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.606 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.606 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.606 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.606 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.6", 2023-11-28T11:09:50.606 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQACymVleb4IARAAXz8dDUDjem6eqAczZ5omDg==", 2023-11-28T11:09:50.606 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.606 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:50.606 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:50.606 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.606 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.606 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.7", 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAFymVlloyRABAAMa2gdE3weqqPWgiCI28jbQ==", 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.0", 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVl5LhsGxAA7WC9xFoSeBWdUPsmO5QaSg==", 2023-11-28T11:09:50.607 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.608 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:50.608 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow r", 2023-11-28T11:09:50.608 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T11:09:50.608 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.608 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.608 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.608 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.1", 2023-11-28T11:09:50.608 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlXV40IhAAhx1t1uwqXPA2g7GmoSXVmA==", 2023-11-28T11:09:50.608 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.608 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:50.608 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T11:09:50.608 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.2", 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlnJDEHhAAEZX8g8t6Gbm6InUWOLqetw==", 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.609 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.610 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.610 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.3", 2023-11-28T11:09:50.610 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlPtmDJRAAoEvqSTpIIIWm42myVE3jVw==", 2023-11-28T11:09:50.610 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.610 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:09:50.610 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T11:09:50.613 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T11:09:50.613 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T11:09:50.613 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.613 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.613 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.613 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.admin", 2023-11-28T11:09:50.613 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDqyWVl8mP6HRAAudHt/W3jSGkgKdYHCgM2uQ==", 2023-11-28T11:09:50.614 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.614 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T11:09:50.614 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:09:50.614 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:09:50.614 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.614 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.614 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.614 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.614 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mgr.x", 2023-11-28T11:09:50.614 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDqyWVly3m5OhAAWirKz8b4mHzNgLftULkWQg==", 2023-11-28T11:09:50.614 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.614 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile mgr", 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mgr.y", 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDqyWVlI+zRMxAANSXxnBj90PtANlg25RwW1A==", 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile mgr", 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.615 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:50.616 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2023-11-28T11:09:50.616 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:09:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.597+0000 7fba41a05700 1 -- 172.21.15.156:0/2562901638 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fba280682f0 msgr2=0x7fba2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.597+0000 7fba41a05700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fba280682f0 0x7fba2806a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0c3ef0 tx=0x7fba30009040 comp rx=0 tx=0).stop 2023-11-28T11:09:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.597+0000 7fba41a05700 1 -- 172.21.15.156:0/2562901638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba3c06dee0 msgr2=0x7fba3c0b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.597+0000 7fba41a05700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba3c06dee0 0x7fba3c0b40e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009fd0 tx=0x7fba2c004750 comp rx=0 tx=0).stop 2023-11-28T11:09:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.598+0000 7fba41a05700 1 -- 172.21.15.156:0/2562901638 shutdown_connections 2023-11-28T11:09:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.598+0000 7fba41a05700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba3c160540 0x7fba3c0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.598+0000 7fba41a05700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba3c06dee0 0x7fba3c0b40e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.598+0000 7fba41a05700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fba280682f0 0x7fba2806a7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.598+0000 7fba41a05700 1 --2- 172.21.15.156:0/2562901638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba3c071040 0x7fba3c0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.598+0000 7fba41a05700 1 -- 172.21.15.156:0/2562901638 >> 172.21.15.156:0/2562901638 conn(0x7fba3c0d0000 msgr2=0x7fba3c0cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.598+0000 7fba41a05700 1 -- 172.21.15.156:0/2562901638 shutdown_connections 2023-11-28T11:09:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.598+0000 7fba41a05700 1 -- 172.21.15.156:0/2562901638 wait complete. 2023-11-28T11:09:50.619 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:50.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.738+0000 7f7a300f0700 1 Processor -- start 2023-11-28T11:09:50.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.738+0000 7f7a300f0700 1 -- start start 2023-11-28T11:09:50.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a300f0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7a2806dee0 0x7f7a2806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a300f0700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2815d060 con 0x7f7a28160540 2023-11-28T11:09:50.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a300f0700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2815d340 con 0x7f7a28071040 2023-11-28T11:09:50.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a300f0700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a2815d620 con 0x7f7a2806dee0 2023-11-28T11:09:50.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a2d68b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7a2806dee0 0x7f7a2806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a2e68d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7a28160540 0x7f7a28160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49382/0 (socket says 172.21.15.156:49382) 2023-11-28T11:09:50.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a2e68d700 1 -- 172.21.15.156:0/1330359483 learned_addr learned my addr 172.21.15.156:0/1330359483 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:50.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a2d68b700 1 -- 172.21.15.156:0/1330359483 >> v1:172.21.15.186:6789/0 conn(0x7f7a28071040 legacy=0x7f7a28071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a2d68b700 1 -- 172.21.15.156:0/1330359483 >> v1:172.21.15.156:6789/0 conn(0x7f7a28160540 legacy=0x7f7a28160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a2d68b700 1 -- 172.21.15.156:0/1330359483 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2815d900 con 0x7f7a2806dee0 2023-11-28T11:09:50.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a2ce8a700 1 -- 172.21.15.156:0/1330359483 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610339811 0 0) 0x7f7a2815d060 con 0x7f7a28160540 2023-11-28T11:09:50.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a2d68b700 1 --2- 172.21.15.156:0/1330359483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a2806dee0 0x7f7a2806e330 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009840 tx=0x7f7a18009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cd763ae4e7bb7e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.739+0000 7f7a2ce8a700 1 -- 172.21.15.156:0/1330359483 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18011830 con 0x7f7a2806dee0 2023-11-28T11:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.740+0000 7f7a2ce8a700 1 -- 172.21.15.156:0/1330359483 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a18011990 con 0x7f7a2806dee0 2023-11-28T11:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.740+0000 7f7a2ce8a700 1 -- 172.21.15.156:0/1330359483 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18011d20 con 0x7f7a2806dee0 2023-11-28T11:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.740+0000 7f7a300f0700 1 -- 172.21.15.156:0/1330359483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a2806dee0 msgr2=0x7f7a2806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.740+0000 7f7a300f0700 1 --2- 172.21.15.156:0/1330359483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a2806dee0 0x7f7a2806e330 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009840 tx=0x7f7a18009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.740+0000 7f7a300f0700 1 -- 172.21.15.156:0/1330359483 shutdown_connections 2023-11-28T11:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.740+0000 7f7a300f0700 1 --2- 172.21.15.156:0/1330359483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a2806dee0 0x7f7a2806e330 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.740+0000 7f7a300f0700 1 -- 172.21.15.156:0/1330359483 >> 172.21.15.156:0/1330359483 conn(0x7f7a280d0000 msgr2=0x7f7a280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.740+0000 7f7a300f0700 1 -- 172.21.15.156:0/1330359483 shutdown_connections 2023-11-28T11:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.740+0000 7f7a300f0700 1 -- 172.21.15.156:0/1330359483 wait complete. 2023-11-28T11:09:50.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.741+0000 7f7a300f0700 1 Processor -- start 2023-11-28T11:09:50.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.741+0000 7f7a300f0700 1 -- start start 2023-11-28T11:09:50.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a300f0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a2806dee0 0x7f7a28153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a300f0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a28071040 0x7f7a280b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a300f0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28160540 0x7f7a2815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a300f0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a28159120 con 0x7f7a28160540 2023-11-28T11:09:50.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a300f0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a280b0920 con 0x7f7a28071040 2023-11-28T11:09:50.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a300f0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a280b0bd0 con 0x7f7a2806dee0 2023-11-28T11:09:50.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a2e68d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28160540 0x7f7a2815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a2e68d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28160540 0x7f7a2815e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34510/0 (socket says 172.21.15.156:34510) 2023-11-28T11:09:50.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a2e68d700 1 -- 172.21.15.156:0/593182630 learned_addr learned my addr 172.21.15.156:0/593182630 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:50.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a2d68b700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a28071040 0x7f7a280b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a2de8c700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a2806dee0 0x7f7a28153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a2e68d700 1 -- 172.21.15.156:0/593182630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a2806dee0 msgr2=0x7f7a28153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a2e68d700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a2806dee0 0x7f7a28153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a2e68d700 1 -- 172.21.15.156:0/593182630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a28071040 msgr2=0x7f7a280b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a2e68d700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a28071040 0x7f7a280b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a2e68d700 1 -- 172.21.15.156:0/593182630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a180094a0 con 0x7f7a28160540 2023-11-28T11:09:50.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.742+0000 7f7a2d68b700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a28071040 0x7f7a280b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:50.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.743+0000 7f7a2e68d700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28160540 0x7f7a2815e290 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000cba0 tx=0x7f7a2000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.743+0000 7f7a1e7fc700 1 -- 172.21.15.156:0/593182630 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000aad0 con 0x7f7a28160540 2023-11-28T11:09:50.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.743+0000 7f7a1e7fc700 1 -- 172.21.15.156:0/593182630 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2000ac50 con 0x7f7a28160540 2023-11-28T11:09:50.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.743+0000 7f7a300f0700 1 -- 172.21.15.156:0/593182630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2815ccd0 con 0x7f7a28160540 2023-11-28T11:09:50.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.743+0000 7f7a1e7fc700 1 -- 172.21.15.156:0/593182630 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a200070d0 con 0x7f7a28160540 2023-11-28T11:09:50.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.743+0000 7f7a300f0700 1 -- 172.21.15.156:0/593182630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2806e870 con 0x7f7a28160540 2023-11-28T11:09:50.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.744+0000 7f7a300f0700 1 -- 172.21.15.156:0/593182630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2806a270 con 0x7f7a28160540 2023-11-28T11:09:50.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.744+0000 7f7a1e7fc700 1 -- 172.21.15.156:0/593182630 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a2001a030 con 0x7f7a28160540 2023-11-28T11:09:50.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.745+0000 7f7a1e7fc700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7a0c0682f0 0x7f7a0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.745+0000 7f7a2de8c700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7a0c0682f0 0x7f7a0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.745+0000 7f7a1e7fc700 1 -- 172.21.15.156:0/593182630 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7a20085570 con 0x7f7a28160540 2023-11-28T11:09:50.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.745+0000 7f7a2de8c700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7a0c0682f0 0x7f7a0c06a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7a24000f60 tx=0x7f7a24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.749+0000 7f7a1e7fc700 1 -- 172.21.15.156:0/593182630 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7a20052120 con 0x7f7a28160540 2023-11-28T11:09:50.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.935+0000 7f7a300f0700 1 -- 172.21.15.156:0/593182630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a280720a0 con 0x7f7a28160540 2023-11-28T11:09:50.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.936+0000 7f7a1e7fc700 1 -- 172.21.15.156:0/593182630 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+14315 (secure 0 0 0) 0x7f7a20018090 con 0x7f7a28160540 2023-11-28T11:09:50.937 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:50.937 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":45,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:50.188088+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:50.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.938+0000 7f7a300f0700 1 -- 172.21.15.156:0/593182630 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7a0c0682f0 msgr2=0x7f7a0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.938+0000 7f7a300f0700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7a0c0682f0 0x7f7a0c06a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7a24000f60 tx=0x7f7a24009040 comp rx=0 tx=0).stop 2023-11-28T11:09:50.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.938+0000 7f7a300f0700 1 -- 172.21.15.156:0/593182630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28160540 msgr2=0x7f7a2815e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.938+0000 7f7a300f0700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28160540 0x7f7a2815e290 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000cba0 tx=0x7f7a2000ceb0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.938+0000 7f7a300f0700 1 -- 172.21.15.156:0/593182630 shutdown_connections 2023-11-28T11:09:50.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.938+0000 7f7a300f0700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a28160540 0x7f7a2815e290 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.938+0000 7f7a300f0700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7a0c0682f0 0x7f7a0c06a7a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.938+0000 7f7a300f0700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a2806dee0 0x7f7a28153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.938+0000 7f7a300f0700 1 --2- 172.21.15.156:0/593182630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a28071040 0x7f7a280b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.938+0000 7f7a300f0700 1 -- 172.21.15.156:0/593182630 >> 172.21.15.156:0/593182630 conn(0x7f7a280d0000 msgr2=0x7f7a2815f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.938+0000 7f7a300f0700 1 -- 172.21.15.156:0/593182630 shutdown_connections 2023-11-28T11:09:50.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:50.938+0000 7f7a300f0700 1 -- 172.21.15.156:0/593182630 wait complete. 2023-11-28T11:09:50.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:09:50.953 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:50.976 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:09:50.998 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:09:51.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.078+0000 7f2e28a92700 1 Processor -- start 2023-11-28T11:09:51.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.078+0000 7f2e28a92700 1 -- start start 2023-11-28T11:09:51.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.079+0000 7f2e28a92700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2e240cd8c0 0x7f2e24140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.079+0000 7f2e28a92700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2e240cdc90 0x7f2e240b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.079+0000 7f2e28a92700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2413ff00 con 0x7f2e240b2e70 2023-11-28T11:09:51.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.079+0000 7f2e28a92700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e241404c0 con 0x7f2e240cd8c0 2023-11-28T11:09:51.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.079+0000 7f2e28a92700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2415aa80 con 0x7f2e240cdc90 2023-11-28T11:09:51.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e2259c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2e240cd8c0 0x7f2e24140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e22d9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2e240b2e70 0x7f2e2406f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49396/0 (socket says 172.21.15.156:49396) 2023-11-28T11:09:51.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e22d9d700 1 -- 172.21.15.156:0/3281416400 learned_addr learned my addr 172.21.15.156:0/3281416400 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:51.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e21d9b700 1 --2- 172.21.15.156:0/3281416400 >> v2:172.21.15.186:3300/0 conn(0x7f2e240cdc90 0x7f2e240b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e2259c700 1 -- 172.21.15.156:0/3281416400 >> v2:172.21.15.186:3300/0 conn(0x7f2e240cdc90 msgr2=0x7f2e240b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:51.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e2259c700 1 --2- 172.21.15.156:0/3281416400 >> v2:172.21.15.186:3300/0 conn(0x7f2e240cdc90 0x7f2e240b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e2259c700 1 -- 172.21.15.156:0/3281416400 >> v1:172.21.15.156:6789/0 conn(0x7f2e240b2e70 legacy=0x7f2e2406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e2259c700 1 -- 172.21.15.156:0/3281416400 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e240b3310 con 0x7f2e240cd8c0 2023-11-28T11:09:51.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e2159a700 1 -- 172.21.15.156:0/3281416400 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22113047 0 0) 0x7f2e2413ff00 con 0x7f2e240b2e70 2023-11-28T11:09:51.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e2259c700 1 --2- 172.21.15.156:0/3281416400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e240cd8c0 0x7f2e24140950 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2e24071c40 tx=0x7f2e0c009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827d2a0444b0828c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e2159a700 1 -- 172.21.15.156:0/3281416400 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c005e30 con 0x7f2e240cd8c0 2023-11-28T11:09:51.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e2159a700 1 -- 172.21.15.156:0/3281416400 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c006030 con 0x7f2e240cd8c0 2023-11-28T11:09:51.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.080+0000 7f2e2159a700 1 -- 172.21.15.156:0/3281416400 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c0063c0 con 0x7f2e240cd8c0 2023-11-28T11:09:51.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.081+0000 7f2e28a92700 1 -- 172.21.15.156:0/3281416400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e240cd8c0 msgr2=0x7f2e24140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.081+0000 7f2e28a92700 1 --2- 172.21.15.156:0/3281416400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e240cd8c0 0x7f2e24140950 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2e24071c40 tx=0x7f2e0c009b00 comp rx=0 tx=0).stop 2023-11-28T11:09:51.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.081+0000 7f2e28a92700 1 -- 172.21.15.156:0/3281416400 shutdown_connections 2023-11-28T11:09:51.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.081+0000 7f2e28a92700 1 --2- 172.21.15.156:0/3281416400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e240cd8c0 0x7f2e24140950 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.081+0000 7f2e28a92700 1 --2- 172.21.15.156:0/3281416400 >> v2:172.21.15.186:3300/0 conn(0x7f2e240cdc90 0x7f2e240b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.081+0000 7f2e28a92700 1 -- 172.21.15.156:0/3281416400 >> 172.21.15.156:0/3281416400 conn(0x7f2e2406aed0 msgr2=0x7f2e2406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.081+0000 7f2e28a92700 1 -- 172.21.15.156:0/3281416400 shutdown_connections 2023-11-28T11:09:51.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.081+0000 7f2e28a92700 1 -- 172.21.15.156:0/3281416400 wait complete. 2023-11-28T11:09:51.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.082+0000 7f2e28a92700 1 Processor -- start 2023-11-28T11:09:51.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.082+0000 7f2e28a92700 1 -- start start 2023-11-28T11:09:51.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e28a92700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e240b2e70 0x7f2e2406d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e28a92700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e240cd8c0 0x7f2e24153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e28a92700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e240cdc90 0x7f2e241586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e28a92700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24158260 con 0x7f2e240b2e70 2023-11-28T11:09:51.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e28a92700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e241544c0 con 0x7f2e240cd8c0 2023-11-28T11:09:51.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e28a92700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2415d570 con 0x7f2e240cdc90 2023-11-28T11:09:51.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e2259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e240b2e70 0x7f2e2406d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e2259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e240b2e70 0x7f2e2406d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34520/0 (socket says 172.21.15.156:34520) 2023-11-28T11:09:51.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e2259c700 1 -- 172.21.15.156:0/787151608 learned_addr learned my addr 172.21.15.156:0/787151608 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:51.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e21d9b700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e240cd8c0 0x7f2e24153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e22d9d700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e240cdc90 0x7f2e241586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e2259c700 1 -- 172.21.15.156:0/787151608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e240cdc90 msgr2=0x7f2e241586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e2259c700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e240cdc90 0x7f2e241586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e2259c700 1 -- 172.21.15.156:0/787151608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e240cd8c0 msgr2=0x7f2e24153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e2259c700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e240cd8c0 0x7f2e24153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.083+0000 7f2e2259c700 1 -- 172.21.15.156:0/787151608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c009430 con 0x7f2e240b2e70 2023-11-28T11:09:51.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.084+0000 7f2e2259c700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e240b2e70 0x7f2e2406d010 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c000c00 tx=0x7f2e0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.084+0000 7f2e1affd700 1 -- 172.21.15.156:0/787151608 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c006930 con 0x7f2e240b2e70 2023-11-28T11:09:51.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.084+0000 7f2e1affd700 1 -- 172.21.15.156:0/787151608 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c020070 con 0x7f2e240b2e70 2023-11-28T11:09:51.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.084+0000 7f2e28a92700 1 -- 172.21.15.156:0/787151608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e24140150 con 0x7f2e240b2e70 2023-11-28T11:09:51.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.084+0000 7f2e1affd700 1 -- 172.21.15.156:0/787151608 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c004280 con 0x7f2e240b2e70 2023-11-28T11:09:51.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.084+0000 7f2e28a92700 1 -- 172.21.15.156:0/787151608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e240b35f0 con 0x7f2e240b2e70 2023-11-28T11:09:51.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.085+0000 7f2e28a92700 1 -- 172.21.15.156:0/787151608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e240ce100 con 0x7f2e240b2e70 2023-11-28T11:09:51.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.085+0000 7f2e1affd700 1 -- 172.21.15.156:0/787151608 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2e0c02b020 con 0x7f2e240b2e70 2023-11-28T11:09:51.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.085+0000 7f2e1affd700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e040682f0 0x7f2e0406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.085+0000 7f2e1affd700 1 -- 172.21.15.156:0/787151608 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2e0c087050 con 0x7f2e240b2e70 2023-11-28T11:09:51.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.086+0000 7f2e21d9b700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e040682f0 0x7f2e0406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.086+0000 7f2e21d9b700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e040682f0 0x7f2e0406a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e24154880 tx=0x7f2e14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.090+0000 7f2e1affd700 1 -- 172.21.15.156:0/787151608 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2e0c053c00 con 0x7f2e240b2e70 2023-11-28T11:09:51.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.274+0000 7f2e28a92700 1 -- 172.21.15.156:0/787151608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e24158540 con 0x7f2e240b2e70 2023-11-28T11:09:51.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.274+0000 7f2e1affd700 1 -- 172.21.15.156:0/787151608 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e0c057220 con 0x7f2e240b2e70 2023-11-28T11:09:51.275 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:09:51.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.276+0000 7f2e28a92700 1 -- 172.21.15.156:0/787151608 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e040682f0 msgr2=0x7f2e0406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.276+0000 7f2e28a92700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e040682f0 0x7f2e0406a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e24154880 tx=0x7f2e14008040 comp rx=0 tx=0).stop 2023-11-28T11:09:51.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.276+0000 7f2e28a92700 1 -- 172.21.15.156:0/787151608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e240b2e70 msgr2=0x7f2e2406d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.276+0000 7f2e28a92700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e240b2e70 0x7f2e2406d010 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c000c00 tx=0x7f2e0c004750 comp rx=0 tx=0).stop 2023-11-28T11:09:51.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.276+0000 7f2e28a92700 1 -- 172.21.15.156:0/787151608 shutdown_connections 2023-11-28T11:09:51.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.276+0000 7f2e28a92700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2e040682f0 0x7f2e0406a7a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.276+0000 7f2e28a92700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e240cdc90 0x7f2e241586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.276+0000 7f2e28a92700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e240b2e70 0x7f2e2406d010 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.276+0000 7f2e28a92700 1 --2- 172.21.15.156:0/787151608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e240cd8c0 0x7f2e24153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.276+0000 7f2e28a92700 1 -- 172.21.15.156:0/787151608 >> 172.21.15.156:0/787151608 conn(0x7f2e2406aed0 msgr2=0x7f2e240cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.276+0000 7f2e28a92700 1 -- 172.21.15.156:0/787151608 shutdown_connections 2023-11-28T11:09:51.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.276+0000 7f2e28a92700 1 -- 172.21.15.156:0/787151608 wait complete. 2023-11-28T11:09:51.289 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:51.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.453+0000 7f6d49b8d700 1 Processor -- start 2023-11-28T11:09:51.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.453+0000 7f6d49b8d700 1 -- start start 2023-11-28T11:09:51.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.453+0000 7f6d49b8d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440cd8c0 0x7f6d44140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.453+0000 7f6d49b8d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6d440cdc90 0x7f6d440b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.453+0000 7f6d49b8d700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4413ff00 con 0x7f6d440b2e70 2023-11-28T11:09:51.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.453+0000 7f6d49b8d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d441404c0 con 0x7f6d440cdc90 2023-11-28T11:09:51.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.453+0000 7f6d49b8d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4415aa80 con 0x7f6d440cd8c0 2023-11-28T11:09:51.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.453+0000 7f6d42ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6d440cdc90 0x7f6d440b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.453+0000 7f6d42ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6d440cdc90 0x7f6d440b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34528/0 (socket says 172.21.15.156:34528) 2023-11-28T11:09:51.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.453+0000 7f6d42ffd700 1 -- 172.21.15.156:0/365183777 learned_addr learned my addr 172.21.15.156:0/365183777 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:51.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.454+0000 7f6d42ffd700 1 -- 172.21.15.156:0/365183777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440cd8c0 msgr2=0x7f6d44140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:51.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.454+0000 7f6d437fe700 1 --2- 172.21.15.156:0/365183777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440cd8c0 0x7f6d44140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.454+0000 7f6d42ffd700 1 --2- 172.21.15.156:0/365183777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440cd8c0 0x7f6d44140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.454+0000 7f6d42ffd700 1 -- 172.21.15.156:0/365183777 >> v1:172.21.15.186:6789/0 conn(0x7f6d440b2e70 legacy=0x7f6d4406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:51.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.454+0000 7f6d42ffd700 1 -- 172.21.15.156:0/365183777 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d440b3310 con 0x7f6d440cdc90 2023-11-28T11:09:51.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.454+0000 7f6d42ffd700 1 --2- 172.21.15.156:0/365183777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d440cdc90 0x7f6d440b2930 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6d340098f0 tx=0x7f6d34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2800f6a48e7e2562 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.454+0000 7f6d427fc700 1 -- 172.21.15.156:0/365183777 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34004230 con 0x7f6d440cdc90 2023-11-28T11:09:51.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.454+0000 7f6d427fc700 1 -- 172.21.15.156:0/365183777 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d34004390 con 0x7f6d440cdc90 2023-11-28T11:09:51.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.454+0000 7f6d427fc700 1 -- 172.21.15.156:0/365183777 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d340036a0 con 0x7f6d440cdc90 2023-11-28T11:09:51.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.455+0000 7f6d49b8d700 1 -- 172.21.15.156:0/365183777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d440cdc90 msgr2=0x7f6d440b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.455+0000 7f6d49b8d700 1 --2- 172.21.15.156:0/365183777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d440cdc90 0x7f6d440b2930 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6d340098f0 tx=0x7f6d34009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:51.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.455+0000 7f6d49b8d700 1 -- 172.21.15.156:0/365183777 shutdown_connections 2023-11-28T11:09:51.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.455+0000 7f6d49b8d700 1 --2- 172.21.15.156:0/365183777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440cd8c0 0x7f6d44140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.455+0000 7f6d49b8d700 1 --2- 172.21.15.156:0/365183777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d440cdc90 0x7f6d440b2930 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.455+0000 7f6d49b8d700 1 -- 172.21.15.156:0/365183777 >> 172.21.15.156:0/365183777 conn(0x7f6d4406aed0 msgr2=0x7f6d4406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.455+0000 7f6d49b8d700 1 -- 172.21.15.156:0/365183777 shutdown_connections 2023-11-28T11:09:51.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.455+0000 7f6d49b8d700 1 -- 172.21.15.156:0/365183777 wait complete. 2023-11-28T11:09:51.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.456+0000 7f6d49b8d700 1 Processor -- start 2023-11-28T11:09:51.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.456+0000 7f6d49b8d700 1 -- start start 2023-11-28T11:09:51.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d49b8d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440b2e70 0x7f6d4406d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d49b8d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d440cd8c0 0x7f6d440c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d49b8d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d440cdc90 0x7f6d4413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d49b8d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d441546c0 con 0x7f6d440cd8c0 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d49b8d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d44158ce0 con 0x7f6d440cdc90 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d49b8d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d440c8c30 con 0x7f6d440b2e70 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d42ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d440cd8c0 0x7f6d440c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d42ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d440cd8c0 0x7f6d440c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34534/0 (socket says 172.21.15.156:34534) 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d42ffd700 1 -- 172.21.15.156:0/3655999768 learned_addr learned my addr 172.21.15.156:0/3655999768 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d43fff700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d440cdc90 0x7f6d4413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d437fe700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440b2e70 0x7f6d4406d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d42ffd700 1 -- 172.21.15.156:0/3655999768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440b2e70 msgr2=0x7f6d4406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d42ffd700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440b2e70 0x7f6d4406d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d42ffd700 1 -- 172.21.15.156:0/3655999768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d440cdc90 msgr2=0x7f6d4413ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d42ffd700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d440cdc90 0x7f6d4413ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d42ffd700 1 -- 172.21.15.156:0/3655999768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d34009550 con 0x7f6d440cd8c0 2023-11-28T11:09:51.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d437fe700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440b2e70 0x7f6d4406d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:51.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d42ffd700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d440cd8c0 0x7f6d440c8150 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009130 tx=0x7f6d34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d3bfff700 1 -- 172.21.15.156:0/3655999768 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3400b860 con 0x7f6d440cd8c0 2023-11-28T11:09:51.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d3bfff700 1 -- 172.21.15.156:0/3655999768 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3400b9c0 con 0x7f6d440cd8c0 2023-11-28T11:09:51.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.457+0000 7f6d49b8d700 1 -- 172.21.15.156:0/3655999768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d44140150 con 0x7f6d440cd8c0 2023-11-28T11:09:51.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.458+0000 7f6d3bfff700 1 -- 172.21.15.156:0/3655999768 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3400bd70 con 0x7f6d440cd8c0 2023-11-28T11:09:51.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.458+0000 7f6d49b8d700 1 -- 172.21.15.156:0/3655999768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d440b35f0 con 0x7f6d440cd8c0 2023-11-28T11:09:51.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.458+0000 7f6d49b8d700 1 -- 172.21.15.156:0/3655999768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d440ce100 con 0x7f6d440cd8c0 2023-11-28T11:09:51.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.459+0000 7f6d3bfff700 1 -- 172.21.15.156:0/3655999768 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6d34017020 con 0x7f6d440cd8c0 2023-11-28T11:09:51.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.459+0000 7f6d3bfff700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6d240682f0 0x7f6d2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.459+0000 7f6d3bfff700 1 -- 172.21.15.156:0/3655999768 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6d34087230 con 0x7f6d440cd8c0 2023-11-28T11:09:51.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.459+0000 7f6d437fe700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6d240682f0 0x7f6d2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.460+0000 7f6d437fe700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6d240682f0 0x7f6d2406a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d4406f040 tx=0x7f6d2c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.464+0000 7f6d3bfff700 1 -- 172.21.15.156:0/3655999768 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6d34053de0 con 0x7f6d440cd8c0 2023-11-28T11:09:51.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.644+0000 7f6d49b8d700 1 -- 172.21.15.156:0/3655999768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6d44068690 con 0x7f6d440cd8c0 2023-11-28T11:09:51.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.644+0000 7f6d3bfff700 1 -- 172.21.15.156:0/3655999768 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6d34057400 con 0x7f6d440cd8c0 2023-11-28T11:09:51.646 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:09:51.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.646+0000 7f6d49b8d700 1 -- 172.21.15.156:0/3655999768 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6d240682f0 msgr2=0x7f6d2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.646+0000 7f6d49b8d700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6d240682f0 0x7f6d2406a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d4406f040 tx=0x7f6d2c008040 comp rx=0 tx=0).stop 2023-11-28T11:09:51.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.646+0000 7f6d49b8d700 1 -- 172.21.15.156:0/3655999768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d440cd8c0 msgr2=0x7f6d440c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.646+0000 7f6d49b8d700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d440cd8c0 0x7f6d440c8150 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009130 tx=0x7f6d34004750 comp rx=0 tx=0).stop 2023-11-28T11:09:51.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.646+0000 7f6d49b8d700 1 -- 172.21.15.156:0/3655999768 shutdown_connections 2023-11-28T11:09:51.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.646+0000 7f6d49b8d700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d440cdc90 0x7f6d4413ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.646+0000 7f6d49b8d700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6d240682f0 0x7f6d2406a7a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.646+0000 7f6d49b8d700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d440b2e70 0x7f6d4406d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.646+0000 7f6d49b8d700 1 --2- 172.21.15.156:0/3655999768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d440cd8c0 0x7f6d440c8150 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.646+0000 7f6d49b8d700 1 -- 172.21.15.156:0/3655999768 >> 172.21.15.156:0/3655999768 conn(0x7f6d4406aed0 msgr2=0x7f6d440b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.646+0000 7f6d49b8d700 1 -- 172.21.15.156:0/3655999768 shutdown_connections 2023-11-28T11:09:51.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.646+0000 7f6d49b8d700 1 -- 172.21.15.156:0/3655999768 wait complete. 2023-11-28T11:09:51.660 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:09:51.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.780+0000 7fbb5175d700 1 Processor -- start 2023-11-28T11:09:51.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.780+0000 7fbb5175d700 1 -- start start 2023-11-28T11:09:51.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.780+0000 7fbb5175d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbb4c06ba90 0x7fbb4c0cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb5175d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4c0cf5e0 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb5175d700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4c0cecb0 con 0x7fbb4c15ca50 2023-11-28T11:09:51.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb5175d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4c0cd470 con 0x7fbb4c06ba90 2023-11-28T11:09:51.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb4a7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbb4c06ba90 0x7fbb4c0cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb4b7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fbb4c0ccc40 0x7fbb4c0cfa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49422/0 (socket says 172.21.15.156:49422) 2023-11-28T11:09:51.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb4a7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbb4c06ba90 0x7fbb4c0cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34542/0 (socket says 172.21.15.156:34542) 2023-11-28T11:09:51.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb4b7fe700 1 -- 172.21.15.156:0/2504815490 learned_addr learned my addr 172.21.15.156:0/2504815490 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:51.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb4a7fc700 1 -- 172.21.15.156:0/2504815490 >> v1:172.21.15.186:6789/0 conn(0x7fbb4c15ca50 legacy=0x7fbb4c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb4a7fc700 1 -- 172.21.15.156:0/2504815490 >> v1:172.21.15.156:6789/0 conn(0x7fbb4c0ccc40 legacy=0x7fbb4c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb4a7fc700 1 -- 172.21.15.156:0/2504815490 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c0cef90 con 0x7fbb4c06ba90 2023-11-28T11:09:51.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb49ffb700 1 -- 172.21.15.156:0/2504815490 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399231897 0 0) 0x7fbb4c0cf5e0 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb4a7fc700 1 --2- 172.21.15.156:0/2504815490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c06ba90 0x7fbb4c0cc700 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009840 tx=0x7fbb40009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fd9d68a2dc51ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb49ffb700 1 -- 172.21.15.156:0/2504815490 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40011830 con 0x7fbb4c06ba90 2023-11-28T11:09:51.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb49ffb700 1 -- 172.21.15.156:0/2504815490 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb40011990 con 0x7fbb4c06ba90 2023-11-28T11:09:51.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.781+0000 7fbb49ffb700 1 -- 172.21.15.156:0/2504815490 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40011d20 con 0x7fbb4c06ba90 2023-11-28T11:09:51.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.782+0000 7fbb5175d700 1 -- 172.21.15.156:0/2504815490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c06ba90 msgr2=0x7fbb4c0cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.782+0000 7fbb5175d700 1 --2- 172.21.15.156:0/2504815490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c06ba90 0x7fbb4c0cc700 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009840 tx=0x7fbb40009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:51.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.782+0000 7fbb5175d700 1 -- 172.21.15.156:0/2504815490 shutdown_connections 2023-11-28T11:09:51.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.782+0000 7fbb5175d700 1 --2- 172.21.15.156:0/2504815490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c06ba90 0x7fbb4c0cc700 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.782+0000 7fbb5175d700 1 -- 172.21.15.156:0/2504815490 >> 172.21.15.156:0/2504815490 conn(0x7fbb4c09a230 msgr2=0x7fbb4c08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.782+0000 7fbb5175d700 1 -- 172.21.15.156:0/2504815490 shutdown_connections 2023-11-28T11:09:51.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.782+0000 7fbb5175d700 1 -- 172.21.15.156:0/2504815490 wait complete. 2023-11-28T11:09:51.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.783+0000 7fbb5175d700 1 Processor -- start 2023-11-28T11:09:51.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.783+0000 7fbb5175d700 1 -- start start 2023-11-28T11:09:51.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb5175d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb4c06ba90 0x7fbb4c0c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb5175d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c0ccc40 0x7fbb4c0c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb5175d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb4c15ca50 0x7fbb4c0b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb5175d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c15e380 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb5175d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c15e660 con 0x7fbb4c06ba90 2023-11-28T11:09:51.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb5175d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c15e940 con 0x7fbb4c15ca50 2023-11-28T11:09:51.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb4a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c0ccc40 0x7fbb4c0c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb4a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c0ccc40 0x7fbb4c0c5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34548/0 (socket says 172.21.15.156:34548) 2023-11-28T11:09:51.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb4a7fc700 1 -- 172.21.15.156:0/1775814577 learned_addr learned my addr 172.21.15.156:0/1775814577 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:51.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb4affd700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb4c06ba90 0x7fbb4c0c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb4b7fe700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb4c15ca50 0x7fbb4c0b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb4a7fc700 1 -- 172.21.15.156:0/1775814577 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb4c15ca50 msgr2=0x7fbb4c0b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb4a7fc700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb4c15ca50 0x7fbb4c0b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb4a7fc700 1 -- 172.21.15.156:0/1775814577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb4c06ba90 msgr2=0x7fbb4c0c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb4a7fc700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb4c06ba90 0x7fbb4c0c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.784+0000 7fbb4a7fc700 1 -- 172.21.15.156:0/1775814577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb400094a0 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.785+0000 7fbb4a7fc700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c0ccc40 0x7fbb4c0c5620 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbb400090b0 tx=0x7fbb40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.785+0000 7fbb337fe700 1 -- 172.21.15.156:0/1775814577 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4001a7e0 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.785+0000 7fbb337fe700 1 -- 172.21.15.156:0/1775814577 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4001a940 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.785+0000 7fbb337fe700 1 -- 172.21.15.156:0/1775814577 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4001acb0 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.785+0000 7fbb5175d700 1 -- 172.21.15.156:0/1775814577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c0cd060 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.785+0000 7fbb5175d700 1 -- 172.21.15.156:0/1775814577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c0cf270 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.786+0000 7fbb5175d700 1 -- 172.21.15.156:0/1775814577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4c0cd1f0 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.786+0000 7fbb337fe700 1 -- 172.21.15.156:0/1775814577 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb4002b030 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.786+0000 7fbb337fe700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbb380682f0 0x7fbb3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.786+0000 7fbb337fe700 1 -- 172.21.15.156:0/1775814577 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbb40086b90 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.786+0000 7fbb4affd700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbb380682f0 0x7fbb3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.787+0000 7fbb4affd700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbb380682f0 0x7fbb3806a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c000f60 tx=0x7fbb3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.791+0000 7fbb337fe700 1 -- 172.21.15.156:0/1775814577 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbb400537f0 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.979+0000 7fbb5175d700 1 -- 172.21.15.156:0/1775814577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbb4c0745c0 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.979+0000 7fbb337fe700 1 -- 172.21.15.156:0/1775814577 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) v1 ==== 84+0+126 (secure 0 0 0) 0x7fbb40029090 con 0x7fbb4c0ccc40 2023-11-28T11:09:51.981 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:51.981 INFO:teuthology.orchestra.run.smithi156.stdout:[ 2023-11-28T11:09:51.981 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:51.981 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 1, 2023-11-28T11:09:51.982 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": ".mgr" 2023-11-28T11:09:51.982 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:09:51.982 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:09:51.982 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 2, 2023-11-28T11:09:51.982 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": "rbd" 2023-11-28T11:09:51.982 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:09:51.982 INFO:teuthology.orchestra.run.smithi156.stdout:] 2023-11-28T11:09:51.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.981+0000 7fbb5175d700 1 -- 172.21.15.156:0/1775814577 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbb380682f0 msgr2=0x7fbb3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.981+0000 7fbb5175d700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbb380682f0 0x7fbb3806a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c000f60 tx=0x7fbb3c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:51.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.981+0000 7fbb5175d700 1 -- 172.21.15.156:0/1775814577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c0ccc40 msgr2=0x7fbb4c0c5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.981+0000 7fbb5175d700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c0ccc40 0x7fbb4c0c5620 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbb400090b0 tx=0x7fbb40004750 comp rx=0 tx=0).stop 2023-11-28T11:09:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.981+0000 7fbb5175d700 1 -- 172.21.15.156:0/1775814577 shutdown_connections 2023-11-28T11:09:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.981+0000 7fbb5175d700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbb380682f0 0x7fbb3806a7a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.981+0000 7fbb5175d700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb4c15ca50 0x7fbb4c0b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.981+0000 7fbb5175d700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb4c06ba90 0x7fbb4c0c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.981+0000 7fbb5175d700 1 --2- 172.21.15.156:0/1775814577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb4c0ccc40 0x7fbb4c0c5620 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.981+0000 7fbb5175d700 1 -- 172.21.15.156:0/1775814577 >> 172.21.15.156:0/1775814577 conn(0x7fbb4c09a230 msgr2=0x7fbb4c15bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.982+0000 7fbb5175d700 1 -- 172.21.15.156:0/1775814577 shutdown_connections 2023-11-28T11:09:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:51.982+0000 7fbb5175d700 1 -- 172.21.15.156:0/1775814577 wait complete. 2023-11-28T11:09:51.995 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:09:51.995 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:09:52.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.118+0000 7f2b05cb6700 1 Processor -- start 2023-11-28T11:09:52.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.118+0000 7f2b05cb6700 1 -- start start 2023-11-28T11:09:52.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.118+0000 7f2b05cb6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b000b6ae0 0x7f2b000cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.118+0000 7f2b05cb6700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b0015af70 con 0x7f2b0015aba0 2023-11-28T11:09:52.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.118+0000 7f2b05cb6700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b000cc870 con 0x7f2b000cfa20 2023-11-28T11:09:52.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.118+0000 7f2b05cb6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b000cfec0 con 0x7f2b000b6ae0 2023-11-28T11:09:52.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.118+0000 7f2aff7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2b0015aba0 0x7f2b000b3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49432/0 (socket says 172.21.15.156:49432) 2023-11-28T11:09:52.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.118+0000 7f2aff7fe700 1 -- 172.21.15.156:0/573889359 learned_addr learned my addr 172.21.15.156:0/573889359 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:52.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140771021 0 0) 0x7f2b0015af70 con 0x7f2b0015aba0 2023-11-28T11:09:52.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ae80035b0 con 0x7f2b0015aba0 2023-11-28T11:09:52.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afeffd700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b000b6ae0 0x7f2b000cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3908595536 0 0) 0x7f2ae80035b0 con 0x7f2b0015aba0 2023-11-28T11:09:52.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b000b6ae0 msgr2=0x7f2b000cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b000b6ae0 0x7f2b000cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 >> v1:172.21.15.186:6789/0 conn(0x7f2b000cfa20 legacy=0x7f2b000d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b000ccaa0 con 0x7f2b0015aba0 2023-11-28T11:09:52.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f2af0001670 con 0x7f2b0015aba0 2023-11-28T11:09:52.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 >> v1:172.21.15.156:6789/0 conn(0x7f2b0015aba0 legacy=0x7f2b000b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae8003a30 0x7f2ae8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ae8004850 0x7f2ae8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ae8005200 0x7f2ae800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b000cfec0 con 0x7f2ae8003a30 2023-11-28T11:09:52.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae800bf50 con 0x7f2ae8004850 2023-11-28T11:09:52.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae800c090 con 0x7f2ae8005200 2023-11-28T11:09:52.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2aff7fe700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae8003a30 0x7f2ae8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afeffd700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ae8004850 0x7f2ae8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2afffff700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ae8005200 0x7f2ae800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2aff7fe700 1 -- 172.21.15.156:0/573889359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ae8005200 msgr2=0x7f2ae800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.119+0000 7f2aff7fe700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ae8005200 0x7f2ae800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2aff7fe700 1 -- 172.21.15.156:0/573889359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ae8004850 msgr2=0x7f2ae8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2aff7fe700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ae8004850 0x7f2ae8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2aff7fe700 1 -- 172.21.15.156:0/573889359 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae80035b0 con 0x7f2ae8003a30 2023-11-28T11:09:52.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2afffff700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ae8005200 0x7f2ae800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:52.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2aff7fe700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae8003a30 0x7f2ae8003e40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2af000c270 tx=0x7f2ae8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5382e9022114bcd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0007c10 con 0x7f2ae8003a30 2023-11-28T11:09:52.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000cdd0 con 0x7f2ae8003a30 2023-11-28T11:09:52.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2afe7fc700 1 -- 172.21.15.156:0/573889359 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0005ba0 con 0x7f2ae8003a30 2023-11-28T11:09:52.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2b05cb6700 1 -- 172.21.15.156:0/573889359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae8003a30 msgr2=0x7f2ae8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2b05cb6700 1 --2- 172.21.15.156:0/573889359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae8003a30 0x7f2ae8003e40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2af000c270 tx=0x7f2ae8005880 comp rx=0 tx=0).stop 2023-11-28T11:09:52.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2afffff700 1 -- 172.21.15.156:0/573889359 reap_dead start 2023-11-28T11:09:52.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2b05cb6700 1 -- 172.21.15.156:0/573889359 shutdown_connections 2023-11-28T11:09:52.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2b05cb6700 1 -- 172.21.15.156:0/573889359 >> 172.21.15.156:0/573889359 conn(0x7f2b0006df70 msgr2=0x7f2b0006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2b05cb6700 1 -- 172.21.15.156:0/573889359 shutdown_connections 2023-11-28T11:09:52.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.120+0000 7f2b05cb6700 1 -- 172.21.15.156:0/573889359 wait complete. 2023-11-28T11:09:52.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.122+0000 7f2b05cb6700 1 Processor -- start 2023-11-28T11:09:52.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.122+0000 7f2b05cb6700 1 -- start start 2023-11-28T11:09:52.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.122+0000 7f2b05cb6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ae8003a30 0x7f2b001524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.122+0000 7f2b05cb6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b0006c670 0x7f2b000ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.122+0000 7f2b05cb6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b0006cae0 0x7f2b000caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.122+0000 7f2b05cb6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0006b380 con 0x7f2b0006c670 2023-11-28T11:09:52.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.122+0000 7f2b05cb6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b001529f0 con 0x7f2ae8003a30 2023-11-28T11:09:52.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.122+0000 7f2b05cb6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0006b5d0 con 0x7f2b0006cae0 2023-11-28T11:09:52.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2afeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b0006c670 0x7f2b000ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2afeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b0006c670 0x7f2b000ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34568/0 (socket says 172.21.15.156:34568) 2023-11-28T11:09:52.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2afeffd700 1 -- 172.21.15.156:0/1074354823 learned_addr learned my addr 172.21.15.156:0/1074354823 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:52.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2aff7fe700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ae8003a30 0x7f2b001524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2afffff700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b0006cae0 0x7f2b000caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2afeffd700 1 -- 172.21.15.156:0/1074354823 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b0006cae0 msgr2=0x7f2b000caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2afeffd700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b0006cae0 0x7f2b000caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2afeffd700 1 -- 172.21.15.156:0/1074354823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ae8003a30 msgr2=0x7f2b001524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2afeffd700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ae8003a30 0x7f2b001524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2afeffd700 1 -- 172.21.15.156:0/1074354823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b000cd060 con 0x7f2b0006c670 2023-11-28T11:09:52.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2afeffd700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b0006c670 0x7f2b000ca9b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2af4011aa0 tx=0x7f2af4011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2ae7fff700 1 -- 172.21.15.156:0/1074354823 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4011ed0 con 0x7f2b0006c670 2023-11-28T11:09:52.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2ae7fff700 1 -- 172.21.15.156:0/1074354823 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af400f940 con 0x7f2b0006c670 2023-11-28T11:09:52.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2b05cb6700 1 -- 172.21.15.156:0/1074354823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b000ccd80 con 0x7f2b0006c670 2023-11-28T11:09:52.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2ae7fff700 1 -- 172.21.15.156:0/1074354823 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af400fcf0 con 0x7f2b0006c670 2023-11-28T11:09:52.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.123+0000 7f2b05cb6700 1 -- 172.21.15.156:0/1074354823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b000cd340 con 0x7f2b0006c670 2023-11-28T11:09:52.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.124+0000 7f2b05cb6700 1 -- 172.21.15.156:0/1074354823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b0006c4c0 con 0x7f2b0006c670 2023-11-28T11:09:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.124+0000 7f2ae7fff700 1 -- 172.21.15.156:0/1074354823 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2af4014440 con 0x7f2b0006c670 2023-11-28T11:09:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.125+0000 7f2ae7fff700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2aec0682f0 0x7f2aec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.125+0000 7f2aff7fe700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2aec0682f0 0x7f2aec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.125+0000 7f2ae7fff700 1 -- 172.21.15.156:0/1074354823 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2af4086040 con 0x7f2b0006c670 2023-11-28T11:09:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.125+0000 7f2aff7fe700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2aec0682f0 0x7f2aec06a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2af0001930 tx=0x7f2af0007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.129+0000 7f2ae7fff700 1 -- 172.21.15.156:0/1074354823 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2af4052c50 con 0x7f2b0006c670 2023-11-28T11:09:52.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.313+0000 7f2b05cb6700 1 -- 172.21.15.156:0/1074354823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2b00068690 con 0x7f2b0006c670 2023-11-28T11:09:52.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.867+0000 7f2ae7fff700 1 -- 172.21.15.156:0/1074354823 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v46) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2af4056270 con 0x7f2b0006c670 2023-11-28T11:09:52.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.870+0000 7f2b05cb6700 1 -- 172.21.15.156:0/1074354823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2aec0682f0 msgr2=0x7f2aec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.870+0000 7f2b05cb6700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2aec0682f0 0x7f2aec06a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2af0001930 tx=0x7f2af0007000 comp rx=0 tx=0).stop 2023-11-28T11:09:52.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.870+0000 7f2b05cb6700 1 -- 172.21.15.156:0/1074354823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b0006c670 msgr2=0x7f2b000ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.870+0000 7f2b05cb6700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b0006c670 0x7f2b000ca9b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2af4011aa0 tx=0x7f2af4011db0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.870+0000 7f2b05cb6700 1 -- 172.21.15.156:0/1074354823 shutdown_connections 2023-11-28T11:09:52.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.870+0000 7f2b05cb6700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2aec0682f0 0x7f2aec06a7a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.870+0000 7f2b05cb6700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b0006cae0 0x7f2b000caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.870+0000 7f2b05cb6700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ae8003a30 0x7f2b001524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.870+0000 7f2b05cb6700 1 --2- 172.21.15.156:0/1074354823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b0006c670 0x7f2b000ca9b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.870+0000 7f2b05cb6700 1 -- 172.21.15.156:0/1074354823 >> 172.21.15.156:0/1074354823 conn(0x7f2b0006df70 msgr2=0x7f2b00158000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.871+0000 7f2b05cb6700 1 -- 172.21.15.156:0/1074354823 shutdown_connections 2023-11-28T11:09:52.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:52.871+0000 7f2b05cb6700 1 -- 172.21.15.156:0/1074354823 wait complete. 2023-11-28T11:09:52.873 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' created 2023-11-28T11:09:52.888 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:09:53.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.017+0000 7f83797d0700 1 Processor -- start 2023-11-28T11:09:53.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.017+0000 7f83797d0700 1 -- start start 2023-11-28T11:09:53.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.018+0000 7f83797d0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f837415a5d0 0x7f83740711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.018+0000 7f83797d0700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f83740caf80 0x7f83740cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.018+0000 7f83797d0700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83740cfdb0 con 0x7f83740cb3f0 2023-11-28T11:09:53.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.018+0000 7f83797d0700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83740cf1a0 con 0x7f83740caf80 2023-11-28T11:09:53.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.018+0000 7f83797d0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83740cf480 con 0x7f837415a5d0 2023-11-28T11:09:53.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.018+0000 7f83737fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f83740cb3f0 0x7f837406dd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49456/0 (socket says 172.21.15.156:49456) 2023-11-28T11:09:53.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.018+0000 7f83737fe700 1 -- 172.21.15.156:0/647016023 learned_addr learned my addr 172.21.15.156:0/647016023 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:53.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.018+0000 7f8372ffd700 1 --2- 172.21.15.156:0/647016023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f837415a5d0 0x7f83740711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.018+0000 7f83727fc700 1 --2- 172.21.15.156:0/647016023 >> v2:172.21.15.186:3300/0 conn(0x7f83740caf80 0x7f83740cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.019+0000 7f8371ffb700 1 -- 172.21.15.156:0/647016023 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783753095 0 0) 0x7f83740cfdb0 con 0x7f83740cb3f0 2023-11-28T11:09:53.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.019+0000 7f8371ffb700 1 -- 172.21.15.156:0/647016023 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f835c0035b0 con 0x7f83740cb3f0 2023-11-28T11:09:53.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.019+0000 7f8372ffd700 1 -- 172.21.15.156:0/647016023 >> v2:172.21.15.186:3300/0 conn(0x7f83740caf80 msgr2=0x7f83740cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:53.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.019+0000 7f8372ffd700 1 --2- 172.21.15.156:0/647016023 >> v2:172.21.15.186:3300/0 conn(0x7f83740caf80 0x7f83740cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.019+0000 7f8372ffd700 1 -- 172.21.15.156:0/647016023 >> v1:172.21.15.156:6789/0 conn(0x7f83740cb3f0 legacy=0x7f837406dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.019+0000 7f8372ffd700 1 -- 172.21.15.156:0/647016023 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83740cf760 con 0x7f837415a5d0 2023-11-28T11:09:53.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.019+0000 7f83727fc700 1 --2- 172.21.15.156:0/647016023 >> v2:172.21.15.186:3300/0 conn(0x7f83740caf80 0x7f83740cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:53.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.019+0000 7f8372ffd700 1 --2- 172.21.15.156:0/647016023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f837415a5d0 0x7f83740711b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8368009840 tx=0x7f8368009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=941513a01724efb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:53.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.019+0000 7f8371ffb700 1 -- 172.21.15.156:0/647016023 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368011830 con 0x7f837415a5d0 2023-11-28T11:09:53.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.019+0000 7f8371ffb700 1 -- 172.21.15.156:0/647016023 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8368011990 con 0x7f837415a5d0 2023-11-28T11:09:53.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.019+0000 7f8371ffb700 1 -- 172.21.15.156:0/647016023 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368011d00 con 0x7f837415a5d0 2023-11-28T11:09:53.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.020+0000 7f83797d0700 1 -- 172.21.15.156:0/647016023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f837415a5d0 msgr2=0x7f83740711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.020+0000 7f83797d0700 1 --2- 172.21.15.156:0/647016023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f837415a5d0 0x7f83740711b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8368009840 tx=0x7f8368009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:53.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.020+0000 7f83797d0700 1 -- 172.21.15.156:0/647016023 shutdown_connections 2023-11-28T11:09:53.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.020+0000 7f83797d0700 1 --2- 172.21.15.156:0/647016023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f837415a5d0 0x7f83740711b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.020+0000 7f83797d0700 1 --2- 172.21.15.156:0/647016023 >> v2:172.21.15.186:3300/0 conn(0x7f83740caf80 0x7f83740cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.020+0000 7f83797d0700 1 -- 172.21.15.156:0/647016023 >> 172.21.15.156:0/647016023 conn(0x7f83740b0c30 msgr2=0x7f83740b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:53.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.020+0000 7f83797d0700 1 -- 172.21.15.156:0/647016023 shutdown_connections 2023-11-28T11:09:53.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.020+0000 7f83797d0700 1 -- 172.21.15.156:0/647016023 wait complete. 2023-11-28T11:09:53.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.021+0000 7f83797d0700 1 Processor -- start 2023-11-28T11:09:53.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.021+0000 7f83797d0700 1 -- start start 2023-11-28T11:09:53.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83797d0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83740caf80 0x7f83740c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83797d0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83740cb3f0 0x7f8374159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83797d0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837415a5d0 0x7f837414e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83797d0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837414ebb0 con 0x7f83740cb3f0 2023-11-28T11:09:53.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83797d0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b2f80 con 0x7f837415a5d0 2023-11-28T11:09:53.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83797d0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b3de0 con 0x7f83740caf80 2023-11-28T11:09:53.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83727fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83740cb3f0 0x7f8374159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83727fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83740cb3f0 0x7f8374159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34582/0 (socket says 172.21.15.156:34582) 2023-11-28T11:09:53.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83727fc700 1 -- 172.21.15.156:0/1152805362 learned_addr learned my addr 172.21.15.156:0/1152805362 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:53.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f8372ffd700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83740caf80 0x7f83740c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83737fe700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837415a5d0 0x7f837414e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83727fc700 1 -- 172.21.15.156:0/1152805362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83740caf80 msgr2=0x7f83740c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83727fc700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83740caf80 0x7f83740c36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83727fc700 1 -- 172.21.15.156:0/1152805362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837415a5d0 msgr2=0x7f837414e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83727fc700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837415a5d0 0x7f837414e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83727fc700 1 -- 172.21.15.156:0/1152805362 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83640094a0 con 0x7f83740cb3f0 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f83727fc700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83740cb3f0 0x7f8374159a30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f836400ec10 tx=0x7f8364009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.022+0000 7f835b7fe700 1 -- 172.21.15.156:0/1152805362 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836400c940 con 0x7f83740cb3f0 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.023+0000 7f835b7fe700 1 -- 172.21.15.156:0/1152805362 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836400caa0 con 0x7f83740cb3f0 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.023+0000 7f83797d0700 1 -- 172.21.15.156:0/1152805362 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83680094a0 con 0x7f83740cb3f0 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.023+0000 7f835b7fe700 1 -- 172.21.15.156:0/1152805362 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836400ce10 con 0x7f83740cb3f0 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.023+0000 7f83797d0700 1 -- 172.21.15.156:0/1152805362 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837415a9a0 con 0x7f83740cb3f0 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.023+0000 7f83797d0700 1 -- 172.21.15.156:0/1152805362 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83740cfa40 con 0x7f83740cb3f0 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.024+0000 7f835b7fe700 1 -- 172.21.15.156:0/1152805362 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8364023070 con 0x7f83740cb3f0 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.024+0000 7f835b7fe700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f83600682f0 0x7f836006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.025+0000 7f8372ffd700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f83600682f0 0x7f836006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.025+0000 7f835b7fe700 1 -- 172.21.15.156:0/1152805362 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f8364085f10 con 0x7f83740cb3f0 2023-11-28T11:09:53.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.025+0000 7f8372ffd700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f83600682f0 0x7f836006a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8368009fd0 tx=0x7f8368008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:53.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.029+0000 7f835b7fe700 1 -- 172.21.15.156:0/1152805362 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8364052870 con 0x7f83740cb3f0 2023-11-28T11:09:53.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.213+0000 7f83797d0700 1 -- 172.21.15.156:0/1152805362 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8374068690 con 0x7f83740cb3f0 2023-11-28T11:09:53.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.870+0000 7f835b7fe700 1 -- 172.21.15.156:0/1152805362 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v47) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8364055e90 con 0x7f83740cb3f0 2023-11-28T11:09:53.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.874+0000 7f83797d0700 1 -- 172.21.15.156:0/1152805362 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f83600682f0 msgr2=0x7f836006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.874+0000 7f83797d0700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f83600682f0 0x7f836006a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8368009fd0 tx=0x7f8368008030 comp rx=0 tx=0).stop 2023-11-28T11:09:53.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.874+0000 7f83797d0700 1 -- 172.21.15.156:0/1152805362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83740cb3f0 msgr2=0x7f8374159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.874+0000 7f83797d0700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83740cb3f0 0x7f8374159a30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f836400ec10 tx=0x7f8364009ec0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.876+0000 7f83797d0700 1 -- 172.21.15.156:0/1152805362 shutdown_connections 2023-11-28T11:09:53.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.876+0000 7f83797d0700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837415a5d0 0x7f837414e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.876+0000 7f83797d0700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f83600682f0 0x7f836006a7a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.876+0000 7f83797d0700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83740caf80 0x7f83740c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.876+0000 7f83797d0700 1 --2- 172.21.15.156:0/1152805362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83740cb3f0 0x7f8374159a30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.876+0000 7f83797d0700 1 -- 172.21.15.156:0/1152805362 >> 172.21.15.156:0/1152805362 conn(0x7f83740b0c30 msgr2=0x7f8374071170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:53.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.877+0000 7f83797d0700 1 -- 172.21.15.156:0/1152805362 shutdown_connections 2023-11-28T11:09:53.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:53.877+0000 7f83797d0700 1 -- 172.21.15.156:0/1152805362 wait complete. 2023-11-28T11:09:53.892 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' created 2023-11-28T11:09:53.906 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:09:54.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.032+0000 7fd49a8d0700 1 Processor -- start 2023-11-28T11:09:54.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.032+0000 7fd49a8d0700 1 -- start start 2023-11-28T11:09:54.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.032+0000 7fd49a8d0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd49406dee0 0x7fd49406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.032+0000 7fd49a8d0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd494160540 0x7fd4941609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.032+0000 7fd49a8d0700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49415d060 con 0x7fd494071040 2023-11-28T11:09:54.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.032+0000 7fd49a8d0700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49415d340 con 0x7fd49406dee0 2023-11-28T11:09:54.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.032+0000 7fd49a8d0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49415d620 con 0x7fd494160540 2023-11-28T11:09:54.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd4937fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd49406dee0 0x7fd49406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd4937fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd49406dee0 0x7fd49406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34590/0 (socket says 172.21.15.156:34590) 2023-11-28T11:09:54.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd4937fe700 1 -- 172.21.15.156:0/1901798368 learned_addr learned my addr 172.21.15.156:0/1901798368 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:54.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd498e6d700 1 --2- 172.21.15.156:0/1901798368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd494160540 0x7fd4941609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd4937fe700 1 -- 172.21.15.156:0/1901798368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd494160540 msgr2=0x7fd4941609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:54.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd4937fe700 1 --2- 172.21.15.156:0/1901798368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd494160540 0x7fd4941609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd4937fe700 1 -- 172.21.15.156:0/1901798368 >> v1:172.21.15.186:6789/0 conn(0x7fd494071040 legacy=0x7fd494071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd4937fe700 1 -- 172.21.15.156:0/1901798368 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49415d900 con 0x7fd49406dee0 2023-11-28T11:09:54.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd498e6d700 1 --2- 172.21.15.156:0/1901798368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd494160540 0x7fd4941609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:54.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd4937fe700 1 --2- 172.21.15.156:0/1901798368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd49406dee0 0x7fd49406e330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd4880098f0 tx=0x7fd488009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c28c48b00700eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd492ffd700 1 -- 172.21.15.156:0/1901798368 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800b8a0 con 0x7fd49406dee0 2023-11-28T11:09:54.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd492ffd700 1 -- 172.21.15.156:0/1901798368 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48800ba00 con 0x7fd49406dee0 2023-11-28T11:09:54.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.033+0000 7fd492ffd700 1 -- 172.21.15.156:0/1901798368 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800bd90 con 0x7fd49406dee0 2023-11-28T11:09:54.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.034+0000 7fd49a8d0700 1 -- 172.21.15.156:0/1901798368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd49406dee0 msgr2=0x7fd49406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.034+0000 7fd49a8d0700 1 --2- 172.21.15.156:0/1901798368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd49406dee0 0x7fd49406e330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd4880098f0 tx=0x7fd488009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:54.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.034+0000 7fd49a8d0700 1 -- 172.21.15.156:0/1901798368 shutdown_connections 2023-11-28T11:09:54.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.034+0000 7fd49a8d0700 1 --2- 172.21.15.156:0/1901798368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd494160540 0x7fd4941609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.034+0000 7fd49a8d0700 1 --2- 172.21.15.156:0/1901798368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd49406dee0 0x7fd49406e330 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.034+0000 7fd49a8d0700 1 -- 172.21.15.156:0/1901798368 >> 172.21.15.156:0/1901798368 conn(0x7fd4940d0000 msgr2=0x7fd4940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:54.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.034+0000 7fd49a8d0700 1 -- 172.21.15.156:0/1901798368 shutdown_connections 2023-11-28T11:09:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.034+0000 7fd49a8d0700 1 -- 172.21.15.156:0/1901798368 wait complete. 2023-11-28T11:09:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.035+0000 7fd49a8d0700 1 Processor -- start 2023-11-28T11:09:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.035+0000 7fd49a8d0700 1 -- start start 2023-11-28T11:09:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.036+0000 7fd49a8d0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd49406dee0 0x7fd4941582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.036+0000 7fd49a8d0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd494071040 0x7fd4940c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.036+0000 7fd49a8d0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd494160540 0x7fd4940c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.036+0000 7fd49a8d0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4940b3d40 con 0x7fd494071040 2023-11-28T11:09:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.036+0000 7fd49a8d0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4940c5430 con 0x7fd49406dee0 2023-11-28T11:09:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.036+0000 7fd49a8d0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4940c48b0 con 0x7fd494160540 2023-11-28T11:09:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.036+0000 7fd4937fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd494071040 0x7fd4940c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.036+0000 7fd4937fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd494071040 0x7fd4940c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34596/0 (socket says 172.21.15.156:34596) 2023-11-28T11:09:54.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.036+0000 7fd4937fe700 1 -- 172.21.15.156:0/148168724 learned_addr learned my addr 172.21.15.156:0/148168724 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:54.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.036+0000 7fd493fff700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd49406dee0 0x7fd4941582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.036+0000 7fd498e6d700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd494160540 0x7fd4940c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.037+0000 7fd4937fe700 1 -- 172.21.15.156:0/148168724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd494160540 msgr2=0x7fd4940c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.037+0000 7fd4937fe700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd494160540 0x7fd4940c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.037+0000 7fd4937fe700 1 -- 172.21.15.156:0/148168724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd49406dee0 msgr2=0x7fd4941582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.037+0000 7fd4937fe700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd49406dee0 0x7fd4941582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.037+0000 7fd4937fe700 1 -- 172.21.15.156:0/148168724 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd488009550 con 0x7fd494071040 2023-11-28T11:09:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.037+0000 7fd4937fe700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd494071040 0x7fd4940c5f60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd488009c40 tx=0x7fd488004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.037+0000 7fd490ff9700 1 -- 172.21.15.156:0/148168724 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488004300 con 0x7fd494071040 2023-11-28T11:09:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.037+0000 7fd490ff9700 1 -- 172.21.15.156:0/148168724 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48801e070 con 0x7fd494071040 2023-11-28T11:09:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.037+0000 7fd49a8d0700 1 -- 172.21.15.156:0/148168724 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49415ccd0 con 0x7fd494071040 2023-11-28T11:09:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.037+0000 7fd490ff9700 1 -- 172.21.15.156:0/148168724 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4880036a0 con 0x7fd494071040 2023-11-28T11:09:54.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.037+0000 7fd49a8d0700 1 -- 172.21.15.156:0/148168724 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49406e870 con 0x7fd494071040 2023-11-28T11:09:54.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.038+0000 7fd49a8d0700 1 -- 172.21.15.156:0/148168724 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49406a270 con 0x7fd494071040 2023-11-28T11:09:54.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.038+0000 7fd490ff9700 1 -- 172.21.15.156:0/148168724 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd488029020 con 0x7fd494071040 2023-11-28T11:09:54.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.039+0000 7fd490ff9700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd4800682f0 0x7fd48006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.039+0000 7fd490ff9700 1 -- 172.21.15.156:0/148168724 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fd488086a60 con 0x7fd494071040 2023-11-28T11:09:54.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.039+0000 7fd493fff700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd4800682f0 0x7fd48006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.039+0000 7fd493fff700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd4800682f0 0x7fd48006a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd484007fd0 tx=0x7fd484009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.044+0000 7fd490ff9700 1 -- 172.21.15.156:0/148168724 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd488053320 con 0x7fd494071040 2023-11-28T11:09:54.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.272+0000 7fd49a8d0700 1 -- 172.21.15.156:0/148168724 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd494158830 con 0x7fd494071040 2023-11-28T11:09:54.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.280+0000 7fd490ff9700 1 -- 172.21.15.156:0/148168724 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd488056940 con 0x7fd494071040 2023-11-28T11:09:54.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.285+0000 7fd49a8d0700 1 -- 172.21.15.156:0/148168724 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd4800682f0 msgr2=0x7fd48006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.285+0000 7fd49a8d0700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd4800682f0 0x7fd48006a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd484007fd0 tx=0x7fd484009040 comp rx=0 tx=0).stop 2023-11-28T11:09:54.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.285+0000 7fd49a8d0700 1 -- 172.21.15.156:0/148168724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd494071040 msgr2=0x7fd4940c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.285+0000 7fd49a8d0700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd494071040 0x7fd4940c5f60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd488009c40 tx=0x7fd488004750 comp rx=0 tx=0).stop 2023-11-28T11:09:54.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.285+0000 7fd49a8d0700 1 -- 172.21.15.156:0/148168724 shutdown_connections 2023-11-28T11:09:54.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.285+0000 7fd49a8d0700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd4800682f0 0x7fd48006a7a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.285+0000 7fd49a8d0700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd494160540 0x7fd4940c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.285+0000 7fd49a8d0700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd49406dee0 0x7fd4941582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.285+0000 7fd49a8d0700 1 --2- 172.21.15.156:0/148168724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd494071040 0x7fd4940c5f60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.285+0000 7fd49a8d0700 1 -- 172.21.15.156:0/148168724 >> 172.21.15.156:0/148168724 conn(0x7fd4940d0000 msgr2=0x7fd49415e380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:54.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.286+0000 7fd49a8d0700 1 -- 172.21.15.156:0/148168724 shutdown_connections 2023-11-28T11:09:54.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.286+0000 7fd49a8d0700 1 -- 172.21.15.156:0/148168724 wait complete. 2023-11-28T11:09:54.301 INFO:teuthology.orchestra.run.smithi156.stderr:new fs with metadata pool 7 and data pool 8 2023-11-28T11:09:54.311 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:54.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.431+0000 7ff6acb43700 1 Processor -- start 2023-11-28T11:09:54.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.431+0000 7ff6acb43700 1 -- start start 2023-11-28T11:09:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.432+0000 7ff6acb43700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6a8161340 0x7ff6a80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.432+0000 7ff6acb43700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff6a80d2320 0x7ff6a80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.432+0000 7ff6acb43700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a80d1950 con 0x7ff6a815c9b0 2023-11-28T11:09:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.432+0000 7ff6acb43700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a80b5800 con 0x7ff6a80d2320 2023-11-28T11:09:54.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.432+0000 7ff6acb43700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a80b5300 con 0x7ff6a8161340 2023-11-28T11:09:54.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.432+0000 7ff6a6d9d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff6a80d2320 0x7ff6a80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.433+0000 7ff6a659c700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7ff6a815c9b0 0x7ff6a80b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:59840/0 (socket says 172.21.15.156:59840) 2023-11-28T11:09:54.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.433+0000 7ff6a659c700 1 -- 172.21.15.156:0/2154476573 learned_addr learned my addr 172.21.15.156:0/2154476573 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:54.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.433+0000 7ff6a5d9b700 1 --2- 172.21.15.156:0/2154476573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6a8161340 0x7ff6a80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.433+0000 7ff6a6d9d700 1 -- 172.21.15.156:0/2154476573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6a8161340 msgr2=0x7ff6a80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:54.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.433+0000 7ff6a6d9d700 1 --2- 172.21.15.156:0/2154476573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6a8161340 0x7ff6a80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.433+0000 7ff6a6d9d700 1 -- 172.21.15.156:0/2154476573 >> v1:172.21.15.186:6789/0 conn(0x7ff6a815c9b0 legacy=0x7ff6a80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.433+0000 7ff6a559a700 1 -- 172.21.15.156:0/2154476573 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232930821 0 0) 0x7ff6a80d1950 con 0x7ff6a815c9b0 2023-11-28T11:09:54.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.433+0000 7ff6a6d9d700 1 -- 172.21.15.156:0/2154476573 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a80b5520 con 0x7ff6a80d2320 2023-11-28T11:09:54.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.433+0000 7ff6a6d9d700 1 --2- 172.21.15.156:0/2154476573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6a80d2320 0x7ff6a80cd7c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009dc0 tx=0x7ff6a000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c097e7dce7923147 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.433+0000 7ff6a559a700 1 -- 172.21.15.156:0/2154476573 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a000e070 con 0x7ff6a80d2320 2023-11-28T11:09:54.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.433+0000 7ff6a559a700 1 -- 172.21.15.156:0/2154476573 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a000bae0 con 0x7ff6a80d2320 2023-11-28T11:09:54.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.433+0000 7ff6a559a700 1 -- 172.21.15.156:0/2154476573 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a000be90 con 0x7ff6a80d2320 2023-11-28T11:09:54.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.434+0000 7ff6acb43700 1 -- 172.21.15.156:0/2154476573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6a80d2320 msgr2=0x7ff6a80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.434+0000 7ff6acb43700 1 --2- 172.21.15.156:0/2154476573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6a80d2320 0x7ff6a80cd7c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009dc0 tx=0x7ff6a000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:54.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.434+0000 7ff6acb43700 1 -- 172.21.15.156:0/2154476573 shutdown_connections 2023-11-28T11:09:54.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.434+0000 7ff6acb43700 1 --2- 172.21.15.156:0/2154476573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6a80d2320 0x7ff6a80cd7c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.434+0000 7ff6acb43700 1 --2- 172.21.15.156:0/2154476573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6a8161340 0x7ff6a80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.434+0000 7ff6acb43700 1 -- 172.21.15.156:0/2154476573 >> 172.21.15.156:0/2154476573 conn(0x7ff6a8072600 msgr2=0x7ff6a80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:54.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.434+0000 7ff6acb43700 1 -- 172.21.15.156:0/2154476573 shutdown_connections 2023-11-28T11:09:54.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.434+0000 7ff6acb43700 1 -- 172.21.15.156:0/2154476573 wait complete. 2023-11-28T11:09:54.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.435+0000 7ff6acb43700 1 Processor -- start 2023-11-28T11:09:54.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.435+0000 7ff6acb43700 1 -- start start 2023-11-28T11:09:54.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6acb43700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6a80d2320 0x7ff6a81469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6acb43700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a815c9b0 0x7ff6a81594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6acb43700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6a8161340 0x7ff6a8159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6acb43700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a8154350 con 0x7ff6a815c9b0 2023-11-28T11:09:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6acb43700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a8146ef0 con 0x7ff6a80d2320 2023-11-28T11:09:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6acb43700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a8152130 con 0x7ff6a8161340 2023-11-28T11:09:54.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6a5d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a815c9b0 0x7ff6a81594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6a5d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a815c9b0 0x7ff6a81594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34610/0 (socket says 172.21.15.156:34610) 2023-11-28T11:09:54.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6a5d9b700 1 -- 172.21.15.156:0/3370870925 learned_addr learned my addr 172.21.15.156:0/3370870925 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:54.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6a659c700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6a80d2320 0x7ff6a81469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6a6d9d700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6a8161340 0x7ff6a8159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6a5d9b700 1 -- 172.21.15.156:0/3370870925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6a8161340 msgr2=0x7ff6a8159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6a5d9b700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6a8161340 0x7ff6a8159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6a5d9b700 1 -- 172.21.15.156:0/3370870925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6a80d2320 msgr2=0x7ff6a81469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6a5d9b700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6a80d2320 0x7ff6a81469b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.436+0000 7ff6a5d9b700 1 -- 172.21.15.156:0/3370870925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0009a70 con 0x7ff6a815c9b0 2023-11-28T11:09:54.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.437+0000 7ff6a5d9b700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a815c9b0 0x7ff6a81594d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff69800e870 tx=0x7ff69800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.437+0000 7ff696ffd700 1 -- 172.21.15.156:0/3370870925 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698009590 con 0x7ff6a815c9b0 2023-11-28T11:09:54.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.437+0000 7ff696ffd700 1 -- 172.21.15.156:0/3370870925 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff698013680 con 0x7ff6a815c9b0 2023-11-28T11:09:54.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.437+0000 7ff6acb43700 1 -- 172.21.15.156:0/3370870925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a80d15e0 con 0x7ff6a815c9b0 2023-11-28T11:09:54.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.437+0000 7ff696ffd700 1 -- 172.21.15.156:0/3370870925 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6980139f0 con 0x7ff6a815c9b0 2023-11-28T11:09:54.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.437+0000 7ff6acb43700 1 -- 172.21.15.156:0/3370870925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a80d1300 con 0x7ff6a815c9b0 2023-11-28T11:09:54.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.438+0000 7ff6acb43700 1 -- 172.21.15.156:0/3370870925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a815cd80 con 0x7ff6a815c9b0 2023-11-28T11:09:54.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.438+0000 7ff696ffd700 1 -- 172.21.15.156:0/3370870925 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff698019030 con 0x7ff6a815c9b0 2023-11-28T11:09:54.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.438+0000 7ff696ffd700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6880682f0 0x7ff68806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.439+0000 7ff696ffd700 1 -- 172.21.15.156:0/3370870925 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff698085950 con 0x7ff6a815c9b0 2023-11-28T11:09:54.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.439+0000 7ff6a659c700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6880682f0 0x7ff68806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.439+0000 7ff6a659c700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6880682f0 0x7ff68806a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff69c000f60 tx=0x7ff69c007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.443+0000 7ff696ffd700 1 -- 172.21.15.156:0/3370870925 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff6980521b0 con 0x7ff6a815c9b0 2023-11-28T11:09:54.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.627+0000 7ff6acb43700 1 -- 172.21.15.156:0/3370870925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6a80745c0 con 0x7ff6a815c9b0 2023-11-28T11:09:54.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.629+0000 7ff696ffd700 1 -- 172.21.15.156:0/3370870925 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+18040 (secure 0 0 0) 0x7ff6980557d0 con 0x7ff6a815c9b0 2023-11-28T11:09:54.630 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:54.630 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":48,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:54.273921+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:52.331853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:53.232328+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:54.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.631+0000 7ff6acb43700 1 -- 172.21.15.156:0/3370870925 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6880682f0 msgr2=0x7ff68806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.631+0000 7ff6acb43700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6880682f0 0x7ff68806a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff69c000f60 tx=0x7ff69c007e70 comp rx=0 tx=0).stop 2023-11-28T11:09:54.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.631+0000 7ff6acb43700 1 -- 172.21.15.156:0/3370870925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a815c9b0 msgr2=0x7ff6a81594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.631+0000 7ff6acb43700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a815c9b0 0x7ff6a81594d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff69800e870 tx=0x7ff69800eb80 comp rx=0 tx=0).stop 2023-11-28T11:09:54.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.631+0000 7ff6acb43700 1 -- 172.21.15.156:0/3370870925 shutdown_connections 2023-11-28T11:09:54.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.631+0000 7ff6acb43700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6880682f0 0x7ff68806a7a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.631+0000 7ff6acb43700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6a8161340 0x7ff6a8159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.631+0000 7ff6acb43700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6a80d2320 0x7ff6a81469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.631+0000 7ff6acb43700 1 --2- 172.21.15.156:0/3370870925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6a815c9b0 0x7ff6a81594d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.631+0000 7ff6acb43700 1 -- 172.21.15.156:0/3370870925 >> 172.21.15.156:0/3370870925 conn(0x7ff6a8072600 msgr2=0x7ff6a806b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:54.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.631+0000 7ff6acb43700 1 -- 172.21.15.156:0/3370870925 shutdown_connections 2023-11-28T11:09:54.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.631+0000 7ff6acb43700 1 -- 172.21.15.156:0/3370870925 wait complete. 2023-11-28T11:09:54.645 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:54.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.764+0000 7fbb72827700 1 Processor -- start 2023-11-28T11:09:54.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.764+0000 7fbb72827700 1 -- start start 2023-11-28T11:09:54.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.765+0000 7fbb72827700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb6c15cef0 0x7fbb6c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.765+0000 7fbb72827700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6c0cc5f0 con 0x7fbb6c06ba90 2023-11-28T11:09:54.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.765+0000 7fbb72827700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6c15d470 con 0x7fbb6c0cbdf0 2023-11-28T11:09:54.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.765+0000 7fbb72827700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c15d750 con 0x7fbb6c15cef0 2023-11-28T11:09:54.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.765+0000 7fbb70dc4700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fbb6c06ba90 0x7fbb6c0ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49498/0 (socket says 172.21.15.156:49498) 2023-11-28T11:09:54.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.765+0000 7fbb70dc4700 1 -- 172.21.15.156:0/4057233595 learned_addr learned my addr 172.21.15.156:0/4057233595 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:54.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.766+0000 7fbb6bfff700 1 --2- 172.21.15.156:0/4057233595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb6c15cef0 0x7fbb6c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.766+0000 7fbb6bfff700 1 -- 172.21.15.156:0/4057233595 >> v1:172.21.15.186:6789/0 conn(0x7fbb6c0cbdf0 legacy=0x7fbb6c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.766+0000 7fbb6bfff700 1 -- 172.21.15.156:0/4057233595 >> v1:172.21.15.156:6789/0 conn(0x7fbb6c06ba90 legacy=0x7fbb6c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.766+0000 7fbb6bfff700 1 -- 172.21.15.156:0/4057233595 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c15da30 con 0x7fbb6c15cef0 2023-11-28T11:09:54.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.766+0000 7fbb6bfff700 1 --2- 172.21.15.156:0/4057233595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb6c15cef0 0x7fbb6c0af400 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009870 tx=0x7fbb5c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a01230eb96c785 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.768+0000 7fbb6affd700 1 -- 172.21.15.156:0/4057233595 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c011830 con 0x7fbb6c15cef0 2023-11-28T11:09:54.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.768+0000 7fbb6affd700 1 -- 172.21.15.156:0/4057233595 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c011990 con 0x7fbb6c15cef0 2023-11-28T11:09:54.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.768+0000 7fbb6affd700 1 -- 172.21.15.156:0/4057233595 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c011d20 con 0x7fbb6c15cef0 2023-11-28T11:09:54.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.768+0000 7fbb72827700 1 -- 172.21.15.156:0/4057233595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb6c15cef0 msgr2=0x7fbb6c0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.768+0000 7fbb72827700 1 --2- 172.21.15.156:0/4057233595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb6c15cef0 0x7fbb6c0af400 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009870 tx=0x7fbb5c009b80 comp rx=0 tx=0).stop 2023-11-28T11:09:54.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.768+0000 7fbb72827700 1 -- 172.21.15.156:0/4057233595 shutdown_connections 2023-11-28T11:09:54.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.768+0000 7fbb72827700 1 --2- 172.21.15.156:0/4057233595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb6c15cef0 0x7fbb6c0af400 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.768+0000 7fbb72827700 1 -- 172.21.15.156:0/4057233595 >> 172.21.15.156:0/4057233595 conn(0x7fbb6c0d0000 msgr2=0x7fbb6c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:54.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.768+0000 7fbb72827700 1 -- 172.21.15.156:0/4057233595 shutdown_connections 2023-11-28T11:09:54.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.768+0000 7fbb72827700 1 -- 172.21.15.156:0/4057233595 wait complete. 2023-11-28T11:09:54.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.770+0000 7fbb72827700 1 Processor -- start 2023-11-28T11:09:54.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.770+0000 7fbb72827700 1 -- start start 2023-11-28T11:09:54.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb72827700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb6c06ba90 0x7fbb6c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb72827700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb6c0cbdf0 0x7fbb6c159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb72827700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb6c15cef0 0x7fbb6c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb72827700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0c4880 con 0x7fbb6c15cef0 2023-11-28T11:09:54.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb72827700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0b57b0 con 0x7fbb6c06ba90 2023-11-28T11:09:54.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb72827700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0b5a90 con 0x7fbb6c0cbdf0 2023-11-28T11:09:54.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb70dc4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb6c15cef0 0x7fbb6c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb70dc4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb6c15cef0 0x7fbb6c0b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34626/0 (socket says 172.21.15.156:34626) 2023-11-28T11:09:54.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb70dc4700 1 -- 172.21.15.156:0/3657645881 learned_addr learned my addr 172.21.15.156:0/3657645881 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:54.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb6b7fe700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb6c0cbdf0 0x7fbb6c159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb6bfff700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb6c06ba90 0x7fbb6c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb70dc4700 1 -- 172.21.15.156:0/3657645881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb6c0cbdf0 msgr2=0x7fbb6c159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb70dc4700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb6c0cbdf0 0x7fbb6c159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb70dc4700 1 -- 172.21.15.156:0/3657645881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb6c06ba90 msgr2=0x7fbb6c0b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb70dc4700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb6c06ba90 0x7fbb6c0b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.771+0000 7fbb70dc4700 1 -- 172.21.15.156:0/3657645881 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c0094a0 con 0x7fbb6c15cef0 2023-11-28T11:09:54.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.772+0000 7fbb70dc4700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb6c15cef0 0x7fbb6c0b2e30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400ee60 tx=0x7fbb6400c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.772+0000 7fbb68ff9700 1 -- 172.21.15.156:0/3657645881 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64011470 con 0x7fbb6c15cef0 2023-11-28T11:09:54.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.772+0000 7fbb68ff9700 1 -- 172.21.15.156:0/3657645881 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb640115d0 con 0x7fbb6c15cef0 2023-11-28T11:09:54.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.772+0000 7fbb72827700 1 -- 172.21.15.156:0/3657645881 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c15dff0 con 0x7fbb6c15cef0 2023-11-28T11:09:54.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.772+0000 7fbb68ff9700 1 -- 172.21.15.156:0/3657645881 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64011940 con 0x7fbb6c15cef0 2023-11-28T11:09:54.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.772+0000 7fbb72827700 1 -- 172.21.15.156:0/3657645881 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c15dd10 con 0x7fbb6c15cef0 2023-11-28T11:09:54.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.773+0000 7fbb72827700 1 -- 172.21.15.156:0/3657645881 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6c0cad60 con 0x7fbb6c15cef0 2023-11-28T11:09:54.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.773+0000 7fbb68ff9700 1 -- 172.21.15.156:0/3657645881 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb64021070 con 0x7fbb6c15cef0 2023-11-28T11:09:54.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.773+0000 7fbb68ff9700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbb580682f0 0x7fbb5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.774+0000 7fbb68ff9700 1 -- 172.21.15.156:0/3657645881 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbb64088690 con 0x7fbb6c15cef0 2023-11-28T11:09:54.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.774+0000 7fbb6bfff700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbb580682f0 0x7fbb5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.774+0000 7fbb6bfff700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbb580682f0 0x7fbb5806a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0090b0 tx=0x7fbb5c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.778+0000 7fbb68ff9700 1 -- 172.21.15.156:0/3657645881 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbb64054ef0 con 0x7fbb6c15cef0 2023-11-28T11:09:54.865 INFO:tasks.ceph.mds.d.smithi186.stderr:2023-11-28T11:09:54.863+0000 7f78f8657700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:09:54.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.964+0000 7fbb72827700 1 -- 172.21.15.156:0/3657645881 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb6c0cf7f0 con 0x7fbb6c15cef0 2023-11-28T11:09:54.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.965+0000 7fbb68ff9700 1 -- 172.21.15.156:0/3657645881 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+18040 (secure 0 0 0) 0x7fbb64058510 con 0x7fbb6c15cef0 2023-11-28T11:09:54.966 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:54.966 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":48,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:54.273921+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:52.331853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:53.232328+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:54.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.967+0000 7fbb72827700 1 -- 172.21.15.156:0/3657645881 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbb580682f0 msgr2=0x7fbb5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.967+0000 7fbb72827700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbb580682f0 0x7fbb5806a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0090b0 tx=0x7fbb5c008030 comp rx=0 tx=0).stop 2023-11-28T11:09:54.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.967+0000 7fbb72827700 1 -- 172.21.15.156:0/3657645881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb6c15cef0 msgr2=0x7fbb6c0b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.967+0000 7fbb72827700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb6c15cef0 0x7fbb6c0b2e30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400ee60 tx=0x7fbb6400c550 comp rx=0 tx=0).stop 2023-11-28T11:09:54.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.967+0000 7fbb72827700 1 -- 172.21.15.156:0/3657645881 shutdown_connections 2023-11-28T11:09:54.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.967+0000 7fbb72827700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb6c15cef0 0x7fbb6c0b2e30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.967+0000 7fbb72827700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb6c06ba90 0x7fbb6c0b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.967+0000 7fbb72827700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbb580682f0 0x7fbb5806a7a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.967+0000 7fbb72827700 1 --2- 172.21.15.156:0/3657645881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb6c0cbdf0 0x7fbb6c159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.967+0000 7fbb72827700 1 -- 172.21.15.156:0/3657645881 >> 172.21.15.156:0/3657645881 conn(0x7fbb6c0d0000 msgr2=0x7fbb6c06b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:54.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.967+0000 7fbb72827700 1 -- 172.21.15.156:0/3657645881 shutdown_connections 2023-11-28T11:09:54.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:54.967+0000 7fbb72827700 1 -- 172.21.15.156:0/3657645881 wait complete. 2023-11-28T11:09:54.980 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:09:55.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.102+0000 7f0c8f66b700 1 Processor -- start 2023-11-28T11:09:55.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.103+0000 7f0c8f66b700 1 -- start start 2023-11-28T11:09:55.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c8f66b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c8806dee0 0x7f0c8806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c8f66b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8815d060 con 0x7f0c88071040 2023-11-28T11:09:55.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c8f66b700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8815d340 con 0x7f0c88160540 2023-11-28T11:09:55.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c8f66b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8815d620 con 0x7f0c8806dee0 2023-11-28T11:09:55.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c8d407700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f0c88071040 0x7f0c88071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49508/0 (socket says 172.21.15.156:49508) 2023-11-28T11:09:55.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c8d407700 1 -- 172.21.15.156:0/2310365790 learned_addr learned my addr 172.21.15.156:0/2310365790 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:55.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c8cc06700 1 --2- 172.21.15.156:0/2310365790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c8806dee0 0x7f0c8806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846783959 0 0) 0x7f0c8815d060 con 0x7f0c88071040 2023-11-28T11:09:55.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c740035b0 con 0x7f0c88071040 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1565230179 0 0) 0x7f0c740035b0 con 0x7f0c88071040 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c8806dee0 msgr2=0x7f0c8806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 --2- 172.21.15.156:0/2310365790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c8806dee0 0x7f0c8806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 >> v1:172.21.15.186:6789/0 conn(0x7f0c88160540 legacy=0x7f0c881609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8815d900 con 0x7f0c88071040 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f0c78001670 con 0x7f0c88071040 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 >> v1:172.21.15.156:6789/0 conn(0x7f0c88071040 legacy=0x7f0c88071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 --2- 172.21.15.156:0/2310365790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c8806dee0 0x7f0c74003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 --2- 172.21.15.156:0/2310365790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c74003f70 0x7f0c74004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 --2- 172.21.15.156:0/2310365790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c74004d90 0x7f0c74005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8815d620 con 0x7f0c74004d90 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7400bba0 con 0x7f0c74003f70 2023-11-28T11:09:55.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.104+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7400bd60 con 0x7f0c8806dee0 2023-11-28T11:09:55.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.105+0000 7f0c8dc08700 1 --2- 172.21.15.156:0/2310365790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c74004d90 0x7f0c74005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.105+0000 7f0c8d407700 1 --2- 172.21.15.156:0/2310365790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c8806dee0 0x7f0c74003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.105+0000 7f0c8cc06700 1 --2- 172.21.15.156:0/2310365790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c74003f70 0x7f0c74004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.105+0000 7f0c8dc08700 1 -- 172.21.15.156:0/2310365790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c8806dee0 msgr2=0x7f0c74003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:55.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.105+0000 7f0c8dc08700 1 --2- 172.21.15.156:0/2310365790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c8806dee0 0x7f0c74003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.105+0000 7f0c8dc08700 1 -- 172.21.15.156:0/2310365790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c74003f70 msgr2=0x7f0c74004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:55.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.105+0000 7f0c8dc08700 1 --2- 172.21.15.156:0/2310365790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c74003f70 0x7f0c74004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.105+0000 7f0c8dc08700 1 -- 172.21.15.156:0/2310365790 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c740035b0 con 0x7f0c74004d90 2023-11-28T11:09:55.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.105+0000 7f0c8dc08700 1 --2- 172.21.15.156:0/2310365790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c74004d90 0x7f0c74005200 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000c290 tx=0x7f0c74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0ec52ca12ccc7b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:55.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.105+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c80001e70 con 0x7f0c74004d90 2023-11-28T11:09:55.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.105+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8000ce50 con 0x7f0c74004d90 2023-11-28T11:09:55.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.105+0000 7f0c7ffff700 1 -- 172.21.15.156:0/2310365790 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c80012660 con 0x7f0c74004d90 2023-11-28T11:09:55.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.106+0000 7f0c8f66b700 1 -- 172.21.15.156:0/2310365790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c74004d90 msgr2=0x7f0c74005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.106+0000 7f0c8f66b700 1 --2- 172.21.15.156:0/2310365790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c74004d90 0x7f0c74005200 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000c290 tx=0x7f0c74005880 comp rx=0 tx=0).stop 2023-11-28T11:09:55.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.106+0000 7f0c8dc08700 1 -- 172.21.15.156:0/2310365790 reap_dead start 2023-11-28T11:09:55.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.106+0000 7f0c8f66b700 1 -- 172.21.15.156:0/2310365790 shutdown_connections 2023-11-28T11:09:55.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.106+0000 7f0c8f66b700 1 -- 172.21.15.156:0/2310365790 >> 172.21.15.156:0/2310365790 conn(0x7f0c880d0000 msgr2=0x7f0c880d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:55.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.106+0000 7f0c8f66b700 1 -- 172.21.15.156:0/2310365790 shutdown_connections 2023-11-28T11:09:55.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.106+0000 7f0c8f66b700 1 -- 172.21.15.156:0/2310365790 wait complete. 2023-11-28T11:09:55.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.107+0000 7f0c8f66b700 1 Processor -- start 2023-11-28T11:09:55.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.107+0000 7f0c8f66b700 1 -- start start 2023-11-28T11:09:55.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8f66b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c88156890 0x7f0c881f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8f66b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c881f2960 0x7f0c880afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8f66b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c880b3a70 0x7f0c880aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8f66b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c881f33f0 con 0x7f0c881f2960 2023-11-28T11:09:55.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8f66b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c881f2e30 con 0x7f0c880b3a70 2023-11-28T11:09:55.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8f66b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c881f3110 con 0x7f0c88156890 2023-11-28T11:09:55.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8cc06700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c881f2960 0x7f0c880afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8cc06700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c881f2960 0x7f0c880afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34644/0 (socket says 172.21.15.156:34644) 2023-11-28T11:09:55.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8cc06700 1 -- 172.21.15.156:0/3881045504 learned_addr learned my addr 172.21.15.156:0/3881045504 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:55.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8dc08700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c880b3a70 0x7f0c880aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8d407700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c88156890 0x7f0c881f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8cc06700 1 -- 172.21.15.156:0/3881045504 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c88156890 msgr2=0x7f0c881f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8cc06700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c88156890 0x7f0c881f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8cc06700 1 -- 172.21.15.156:0/3881045504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c880b3a70 msgr2=0x7f0c880aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8cc06700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c880b3a70 0x7f0c880aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8cc06700 1 -- 172.21.15.156:0/3881045504 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8806a270 con 0x7f0c881f2960 2023-11-28T11:09:55.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8cc06700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c881f2960 0x7f0c880afa00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0c84011aa0 tx=0x7f0c84011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:55.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c7dffb700 1 -- 172.21.15.156:0/3881045504 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84011ed0 con 0x7f0c881f2960 2023-11-28T11:09:55.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c7dffb700 1 -- 172.21.15.156:0/3881045504 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8400f940 con 0x7f0c881f2960 2023-11-28T11:09:55.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c8f66b700 1 -- 172.21.15.156:0/3881045504 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8815ccd0 con 0x7f0c881f2960 2023-11-28T11:09:55.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.108+0000 7f0c7dffb700 1 -- 172.21.15.156:0/3881045504 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8400fcf0 con 0x7f0c881f2960 2023-11-28T11:09:55.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.109+0000 7f0c8f66b700 1 -- 172.21.15.156:0/3881045504 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c880b4460 con 0x7f0c881f2960 2023-11-28T11:09:55.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.110+0000 7f0c8f66b700 1 -- 172.21.15.156:0/3881045504 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8806a400 con 0x7f0c881f2960 2023-11-28T11:09:55.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.110+0000 7f0c7dffb700 1 -- 172.21.15.156:0/3881045504 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c84014440 con 0x7f0c881f2960 2023-11-28T11:09:55.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.110+0000 7f0c7dffb700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c6c0682f0 0x7f0c6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.110+0000 7f0c7dffb700 1 -- 172.21.15.156:0/3881045504 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0c840864d0 con 0x7f0c881f2960 2023-11-28T11:09:55.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.110+0000 7f0c8d407700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c6c0682f0 0x7f0c6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.111+0000 7f0c8d407700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c6c0682f0 0x7f0c6c06a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0c780010e0 tx=0x7f0c78000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:55.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.115+0000 7f0c7dffb700 1 -- 172.21.15.156:0/3881045504 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0c8806a400 con 0x7f0c881f2960 2023-11-28T11:09:55.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:55.341+0000 7f0c8f66b700 1 -- 172.21.15.156:0/3881045504 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0c8806a400 con 0x7f0c881f2960 2023-11-28T11:09:56.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.286+0000 7f0c7dffb700 1 -- 172.21.15.156:0/3881045504 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0c8806a400 con 0x7f0c881f2960 2023-11-28T11:09:56.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.289+0000 7f0c8f66b700 1 -- 172.21.15.156:0/3881045504 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c6c0682f0 msgr2=0x7f0c6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.289+0000 7f0c8f66b700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c6c0682f0 0x7f0c6c06a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0c780010e0 tx=0x7f0c78000ed0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.289+0000 7f0c8f66b700 1 -- 172.21.15.156:0/3881045504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c881f2960 msgr2=0x7f0c880afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.289+0000 7f0c8f66b700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c881f2960 0x7f0c880afa00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0c84011aa0 tx=0x7f0c84011db0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.289+0000 7f0c8f66b700 1 -- 172.21.15.156:0/3881045504 shutdown_connections 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.289+0000 7f0c8f66b700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c880b3a70 0x7f0c880aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.289+0000 7f0c8f66b700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0c6c0682f0 0x7f0c6c06a7a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.289+0000 7f0c8f66b700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c88156890 0x7f0c881f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.289+0000 7f0c8f66b700 1 --2- 172.21.15.156:0/3881045504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c881f2960 0x7f0c880afa00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.289+0000 7f0c8f66b700 1 -- 172.21.15.156:0/3881045504 >> 172.21.15.156:0/3881045504 conn(0x7f0c880d0000 msgr2=0x7f0c880cd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.289+0000 7f0c8f66b700 1 -- 172.21.15.156:0/3881045504 shutdown_connections 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.289+0000 7f0c8f66b700 1 -- 172.21.15.156:0/3881045504 wait complete. 2023-11-28T11:09:56.305 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:09:56.305 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:09:56.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.434+0000 7ff6fb74e700 1 Processor -- start 2023-11-28T11:09:56.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.434+0000 7ff6fb74e700 1 -- start start 2023-11-28T11:09:56.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6fb74e700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff6f415c9b0 0x7ff6f40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6fb74e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff6f4161340 0x7ff6f40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6fb74e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f40d1950 con 0x7ff6f40d2320 2023-11-28T11:09:56.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6fb74e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f40b5800 con 0x7ff6f4161340 2023-11-28T11:09:56.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6fb74e700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f40b5520 con 0x7ff6f415c9b0 2023-11-28T11:09:56.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6f8ce9700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff6f4161340 0x7ff6f40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6f9ceb700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff6f40d2320 0x7ff6f40cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49532/0 (socket says 172.21.15.156:49532) 2023-11-28T11:09:56.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6f9ceb700 1 -- 172.21.15.156:0/489689221 learned_addr learned my addr 172.21.15.156:0/489689221 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:56.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6f94ea700 1 --2- 172.21.15.156:0/489689221 >> v2:172.21.15.186:3300/0 conn(0x7ff6f415c9b0 0x7ff6f40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6f8ce9700 1 -- 172.21.15.156:0/489689221 >> v2:172.21.15.186:3300/0 conn(0x7ff6f415c9b0 msgr2=0x7ff6f40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:56.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6f8ce9700 1 --2- 172.21.15.156:0/489689221 >> v2:172.21.15.186:3300/0 conn(0x7ff6f415c9b0 0x7ff6f40b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6f8ce9700 1 -- 172.21.15.156:0/489689221 >> v1:172.21.15.156:6789/0 conn(0x7ff6f40d2320 legacy=0x7ff6f40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6ebfff700 1 -- 172.21.15.156:0/489689221 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013923248 0 0) 0x7ff6f40d1950 con 0x7ff6f40d2320 2023-11-28T11:09:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6f8ce9700 1 -- 172.21.15.156:0/489689221 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f40d1300 con 0x7ff6f4161340 2023-11-28T11:09:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.435+0000 7ff6f8ce9700 1 --2- 172.21.15.156:0/489689221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6f4161340 0x7ff6f40d1de0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4067460 tx=0x7ff6e4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d73ad5eb54f569 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.436+0000 7ff6ebfff700 1 -- 172.21.15.156:0/489689221 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4011830 con 0x7ff6f4161340 2023-11-28T11:09:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.436+0000 7ff6ebfff700 1 -- 172.21.15.156:0/489689221 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e4011990 con 0x7ff6f4161340 2023-11-28T11:09:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.436+0000 7ff6ebfff700 1 -- 172.21.15.156:0/489689221 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4011d20 con 0x7ff6f4161340 2023-11-28T11:09:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.436+0000 7ff6fb74e700 1 -- 172.21.15.156:0/489689221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6f4161340 msgr2=0x7ff6f40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.436+0000 7ff6fb74e700 1 --2- 172.21.15.156:0/489689221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6f4161340 0x7ff6f40d1de0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4067460 tx=0x7ff6e4009b20 comp rx=0 tx=0).stop 2023-11-28T11:09:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.436+0000 7ff6fb74e700 1 -- 172.21.15.156:0/489689221 shutdown_connections 2023-11-28T11:09:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.436+0000 7ff6fb74e700 1 --2- 172.21.15.156:0/489689221 >> v2:172.21.15.186:3300/0 conn(0x7ff6f415c9b0 0x7ff6f40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.436+0000 7ff6fb74e700 1 --2- 172.21.15.156:0/489689221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6f4161340 0x7ff6f40d1de0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.436+0000 7ff6fb74e700 1 -- 172.21.15.156:0/489689221 >> 172.21.15.156:0/489689221 conn(0x7ff6f4072600 msgr2=0x7ff6f40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:56.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.436+0000 7ff6fb74e700 1 -- 172.21.15.156:0/489689221 shutdown_connections 2023-11-28T11:09:56.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.436+0000 7ff6fb74e700 1 -- 172.21.15.156:0/489689221 wait complete. 2023-11-28T11:09:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.437+0000 7ff6fb74e700 1 Processor -- start 2023-11-28T11:09:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.437+0000 7ff6fb74e700 1 -- start start 2023-11-28T11:09:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6fb74e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6f40d2320 0x7ff6f40c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6fb74e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f415c9b0 0x7ff6f40b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6fb74e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6f4161340 0x7ff6f40c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6fb74e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f4150bb0 con 0x7ff6f40d2320 2023-11-28T11:09:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6fb74e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40c7d60 con 0x7ff6f415c9b0 2023-11-28T11:09:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6fb74e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40c9240 con 0x7ff6f4161340 2023-11-28T11:09:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6f94ea700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6f40d2320 0x7ff6f40c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6f94ea700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6f40d2320 0x7ff6f40c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34656/0 (socket says 172.21.15.156:34656) 2023-11-28T11:09:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6f94ea700 1 -- 172.21.15.156:0/852879000 learned_addr learned my addr 172.21.15.156:0/852879000 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6f9ceb700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6f4161340 0x7ff6f40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6f8ce9700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f415c9b0 0x7ff6f40b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6f94ea700 1 -- 172.21.15.156:0/852879000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6f4161340 msgr2=0x7ff6f40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6f94ea700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6f4161340 0x7ff6f40c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6f94ea700 1 -- 172.21.15.156:0/852879000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f415c9b0 msgr2=0x7ff6f40b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6f94ea700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f415c9b0 0x7ff6f40b76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6f94ea700 1 -- 172.21.15.156:0/852879000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e40094a0 con 0x7ff6f40d2320 2023-11-28T11:09:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.438+0000 7ff6f8ce9700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f415c9b0 0x7ff6f40b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:56.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.439+0000 7ff6f94ea700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6f40d2320 0x7ff6f40c7820 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000e810 tx=0x7ff6f000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.439+0000 7ff6e9ffb700 1 -- 172.21.15.156:0/852879000 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0013610 con 0x7ff6f40d2320 2023-11-28T11:09:56.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.439+0000 7ff6e9ffb700 1 -- 172.21.15.156:0/852879000 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f0013770 con 0x7ff6f40d2320 2023-11-28T11:09:56.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.439+0000 7ff6fb74e700 1 -- 172.21.15.156:0/852879000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f40b5130 con 0x7ff6f40d2320 2023-11-28T11:09:56.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.439+0000 7ff6e9ffb700 1 -- 172.21.15.156:0/852879000 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0013b20 con 0x7ff6f40d2320 2023-11-28T11:09:56.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.439+0000 7ff6fb74e700 1 -- 172.21.15.156:0/852879000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f40d15e0 con 0x7ff6f40d2320 2023-11-28T11:09:56.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.440+0000 7ff6fb74e700 1 -- 172.21.15.156:0/852879000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f40b52c0 con 0x7ff6f40d2320 2023-11-28T11:09:56.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.440+0000 7ff6e9ffb700 1 -- 172.21.15.156:0/852879000 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff6f0023070 con 0x7ff6f40d2320 2023-11-28T11:09:56.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.441+0000 7ff6e9ffb700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6d80682f0 0x7ff6d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.441+0000 7ff6e9ffb700 1 -- 172.21.15.156:0/852879000 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff6f00859a0 con 0x7ff6f40d2320 2023-11-28T11:09:56.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.441+0000 7ff6f8ce9700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6d80682f0 0x7ff6d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.441+0000 7ff6f8ce9700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6d80682f0 0x7ff6d806a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff6f41f0950 tx=0x7ff6e4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.445+0000 7ff6e9ffb700 1 -- 172.21.15.156:0/852879000 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff6f0052180 con 0x7ff6f40d2320 2023-11-28T11:09:56.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:56.676+0000 7ff6fb74e700 1 -- 172.21.15.156:0/852879000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff6f40c7670 con 0x7ff6f40d2320 2023-11-28T11:09:57.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.290+0000 7ff6e9ffb700 1 -- 172.21.15.156:0/852879000 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff6f00557a0 con 0x7ff6f40d2320 2023-11-28T11:09:57.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.292+0000 7ff6fb74e700 1 -- 172.21.15.156:0/852879000 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6d80682f0 msgr2=0x7ff6d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.293+0000 7ff6fb74e700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6d80682f0 0x7ff6d806a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff6f41f0950 tx=0x7ff6e4008030 comp rx=0 tx=0).stop 2023-11-28T11:09:57.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.293+0000 7ff6fb74e700 1 -- 172.21.15.156:0/852879000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6f40d2320 msgr2=0x7ff6f40c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.293+0000 7ff6fb74e700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6f40d2320 0x7ff6f40c7820 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000e810 tx=0x7ff6f000ebd0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.293+0000 7ff6fb74e700 1 -- 172.21.15.156:0/852879000 shutdown_connections 2023-11-28T11:09:57.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.293+0000 7ff6fb74e700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6d80682f0 0x7ff6d806a7a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.293+0000 7ff6fb74e700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6f4161340 0x7ff6f40c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.293+0000 7ff6fb74e700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6f40d2320 0x7ff6f40c7820 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.293+0000 7ff6fb74e700 1 --2- 172.21.15.156:0/852879000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f415c9b0 0x7ff6f40b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.293+0000 7ff6fb74e700 1 -- 172.21.15.156:0/852879000 >> 172.21.15.156:0/852879000 conn(0x7ff6f4072600 msgr2=0x7ff6f415a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.293+0000 7ff6fb74e700 1 -- 172.21.15.156:0/852879000 shutdown_connections 2023-11-28T11:09:57.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.293+0000 7ff6fb74e700 1 -- 172.21.15.156:0/852879000 wait complete. 2023-11-28T11:09:57.308 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:09:57.309 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:57.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.433+0000 7fd7b15bb700 1 Processor -- start 2023-11-28T11:09:57.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.433+0000 7fd7b15bb700 1 -- start start 2023-11-28T11:09:57.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.433+0000 7fd7b15bb700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac15ca50 0x7fd7ac06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.433+0000 7fd7b15bb700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ac0cf5e0 con 0x7fd7ac0cce10 2023-11-28T11:09:57.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.433+0000 7fd7b15bb700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ac0cecb0 con 0x7fd7ac06b8c0 2023-11-28T11:09:57.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.433+0000 7fd7b15bb700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac0cd290 con 0x7fd7ac15ca50 2023-11-28T11:09:57.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7ab7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd7ac0cce10 0x7fd7ac0cfa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49544/0 (socket says 172.21.15.156:49544) 2023-11-28T11:09:57.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7ab7fe700 1 -- 172.21.15.156:0/1850723526 learned_addr learned my addr 172.21.15.156:0/1850723526 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:57.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7aaffd700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac15ca50 0x7fd7ac06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403892701 0 0) 0x7fd7ac0cf5e0 con 0x7fd7ac0cce10 2023-11-28T11:09:57.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7940035b0 con 0x7fd7ac0cce10 2023-11-28T11:09:57.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397210593 0 0) 0x7fd7ac0cecb0 con 0x7fd7ac06b8c0 2023-11-28T11:09:57.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7ac0cf5e0 con 0x7fd7ac06b8c0 2023-11-28T11:09:57.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 339106538 0 0) 0x7fd7940035b0 con 0x7fd7ac0cce10 2023-11-28T11:09:57.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac15ca50 msgr2=0x7fd7ac06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac15ca50 0x7fd7ac06b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 >> v1:172.21.15.186:6789/0 conn(0x7fd7ac06b8c0 legacy=0x7fd7ac0cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac0cef90 con 0x7fd7ac0cce10 2023-11-28T11:09:57.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fd7a40019f0 con 0x7fd7ac0cce10 2023-11-28T11:09:57.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 >> v1:172.21.15.156:6789/0 conn(0x7fd7ac0cce10 legacy=0x7fd7ac0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd794003bc0 0x7fd794003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac15ca50 0x7fd794004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794004d90 0x7fd794005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac0cd290 con 0x7fd794003bc0 2023-11-28T11:09:57.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79400bac0 con 0x7fd794004d90 2023-11-28T11:09:57.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79400bc30 con 0x7fd7ac15ca50 2023-11-28T11:09:57.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7aaffd700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd794003bc0 0x7fd794003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7aa7fc700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac15ca50 0x7fd794004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.434+0000 7fd7ab7fe700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794004d90 0x7fd794005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7aaffd700 1 -- 172.21.15.156:0/1850723526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac15ca50 msgr2=0x7fd794004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7aaffd700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac15ca50 0x7fd794004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7aaffd700 1 -- 172.21.15.156:0/1850723526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794004d90 msgr2=0x7fd794005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7aaffd700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794004d90 0x7fd794005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7aaffd700 1 -- 172.21.15.156:0/1850723526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7940035b0 con 0x7fd794003bc0 2023-11-28T11:09:57.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7aa7fc700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac15ca50 0x7fd794004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:57.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7aaffd700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd794003bc0 0x7fd794003fb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009b60 tx=0x7fd794005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1d1a6e42f1458b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00e660 con 0x7fd794003bc0 2023-11-28T11:09:57.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c00e7c0 con 0x7fd794003bc0 2023-11-28T11:09:57.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7a9ffb700 1 -- 172.21.15.156:0/1850723526 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00eb30 con 0x7fd794003bc0 2023-11-28T11:09:57.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7b15bb700 1 -- 172.21.15.156:0/1850723526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd794003bc0 msgr2=0x7fd794003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7b15bb700 1 --2- 172.21.15.156:0/1850723526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd794003bc0 0x7fd794003fb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009b60 tx=0x7fd794005880 comp rx=0 tx=0).stop 2023-11-28T11:09:57.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7ab7fe700 1 -- 172.21.15.156:0/1850723526 reap_dead start 2023-11-28T11:09:57.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7b15bb700 1 -- 172.21.15.156:0/1850723526 shutdown_connections 2023-11-28T11:09:57.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.435+0000 7fd7b15bb700 1 -- 172.21.15.156:0/1850723526 >> 172.21.15.156:0/1850723526 conn(0x7fd7ac09a230 msgr2=0x7fd7ac08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.436+0000 7fd7b15bb700 1 -- 172.21.15.156:0/1850723526 shutdown_connections 2023-11-28T11:09:57.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.436+0000 7fd7b15bb700 1 -- 172.21.15.156:0/1850723526 wait complete. 2023-11-28T11:09:57.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.437+0000 7fd7b15bb700 1 Processor -- start 2023-11-28T11:09:57.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.437+0000 7fd7b15bb700 1 -- start start 2023-11-28T11:09:57.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7b15bb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794003bc0 0x7fd7ac158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7b15bb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7ac0b6610 0x7fd7ac0b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7b15bb700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac0b2870 0x7fd7ac0b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7b15bb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac158ab0 con 0x7fd7ac0b6610 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7b15bb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac0b2ce0 con 0x7fd794003bc0 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7b15bb700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac0b6180 con 0x7fd7ac0b2870 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7aa7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7ac0b6610 0x7fd7ac0b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7aa7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7ac0b6610 0x7fd7ac0b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34678/0 (socket says 172.21.15.156:34678) 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7aa7fc700 1 -- 172.21.15.156:0/2961662012 learned_addr learned my addr 172.21.15.156:0/2961662012 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7aa7fc700 1 -- 172.21.15.156:0/2961662012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac0b2870 msgr2=0x7fd7ac0b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7aa7fc700 1 --2- 172.21.15.156:0/2961662012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac0b2870 0x7fd7ac0b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7aa7fc700 1 -- 172.21.15.156:0/2961662012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794003bc0 msgr2=0x7fd7ac158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7aa7fc700 1 --2- 172.21.15.156:0/2961662012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794003bc0 0x7fd7ac158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7aa7fc700 1 -- 172.21.15.156:0/2961662012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac15ce20 con 0x7fd7ac0b6610 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7aa7fc700 1 --2- 172.21.15.156:0/2961662012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7ac0b6610 0x7fd7ac0b2330 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000e7e0 tx=0x7fd7a000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7937fe700 1 -- 172.21.15.156:0/2961662012 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0017070 con 0x7fd7ac0b6610 2023-11-28T11:09:57.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7937fe700 1 -- 172.21.15.156:0/2961662012 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a00052e0 con 0x7fd7ac0b6610 2023-11-28T11:09:57.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7b15bb700 1 -- 172.21.15.156:0/2961662012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ac0cf270 con 0x7fd7ac0b6610 2023-11-28T11:09:57.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7937fe700 1 -- 172.21.15.156:0/2961662012 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0005470 con 0x7fd7ac0b6610 2023-11-28T11:09:57.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.438+0000 7fd7b15bb700 1 -- 172.21.15.156:0/2961662012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac15cfb0 con 0x7fd7ac0b6610 2023-11-28T11:09:57.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.439+0000 7fd7b15bb700 1 -- 172.21.15.156:0/2961662012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ac0745c0 con 0x7fd7ac0b6610 2023-11-28T11:09:57.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.440+0000 7fd7937fe700 1 -- 172.21.15.156:0/2961662012 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd7a0006860 con 0x7fd7ac0b6610 2023-11-28T11:09:57.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.440+0000 7fd7937fe700 1 --2- 172.21.15.156:0/2961662012 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7980682f0 0x7fd79806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.440+0000 7fd7937fe700 1 -- 172.21.15.156:0/2961662012 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd7a0085550 con 0x7fd7ac0b6610 2023-11-28T11:09:57.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.441+0000 7fd7aaffd700 1 --2- 172.21.15.156:0/2961662012 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7980682f0 0x7fd79806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.441+0000 7fd7aaffd700 1 --2- 172.21.15.156:0/2961662012 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7980682f0 0x7fd79806a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0090b0 tx=0x7fd79c013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.445+0000 7fd7937fe700 1 -- 172.21.15.156:0/2961662012 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd7a0051cc0 con 0x7fd7ac0b6610 2023-11-28T11:09:57.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.669+0000 7fd7b15bb700 1 -- 172.21.15.156:0/2961662012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7ac0b62c0 con 0x7fd7ac0b6610 2023-11-28T11:09:57.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.670+0000 7fd7937fe700 1 -- 172.21.15.156:0/2961662012 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fd7ac0b62c0 con 0x7fd7ac0b6610 2023-11-28T11:09:57.671 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:57.671 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/365552946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":365552946},{"type":"v1","addr":"172.21.15.186:6834","nonce":365552946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1918325985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":1918325985},{"type":"v1","addr":"172.21.15.156:6837","nonce":1918325985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/2889641252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2889641252},{"type":"v1","addr":"172.21.15.156:6836","nonce":2889641252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:54.274258+0000","modified":"2023-11-28T11:09:57.285496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3832587641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3832587641},{"type":"v1","addr":"172.21.15.186:6835","nonce":3832587641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:09:57.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.672+0000 7fd7b15bb700 1 -- 172.21.15.156:0/2961662012 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7980682f0 msgr2=0x7fd79806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.672+0000 7fd7b15bb700 1 --2- 172.21.15.156:0/2961662012 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7980682f0 0x7fd79806a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0090b0 tx=0x7fd79c013000 comp rx=0 tx=0).stop 2023-11-28T11:09:57.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.672+0000 7fd7b15bb700 1 -- 172.21.15.156:0/2961662012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7ac0b6610 msgr2=0x7fd7ac0b2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.672+0000 7fd7b15bb700 1 --2- 172.21.15.156:0/2961662012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7ac0b6610 0x7fd7ac0b2330 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000e7e0 tx=0x7fd7a000eba0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.672+0000 7fd7b15bb700 1 -- 172.21.15.156:0/2961662012 shutdown_connections 2023-11-28T11:09:57.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.672+0000 7fd7b15bb700 1 --2- 172.21.15.156:0/2961662012 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7980682f0 0x7fd79806a7a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.672+0000 7fd7b15bb700 1 --2- 172.21.15.156:0/2961662012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7ac0b2870 0x7fd7ac0b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.672+0000 7fd7b15bb700 1 --2- 172.21.15.156:0/2961662012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794003bc0 0x7fd7ac158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.672+0000 7fd7b15bb700 1 --2- 172.21.15.156:0/2961662012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7ac0b6610 0x7fd7ac0b2330 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.672+0000 7fd7b15bb700 1 -- 172.21.15.156:0/2961662012 >> 172.21.15.156:0/2961662012 conn(0x7fd7ac09a230 msgr2=0x7fd7ac0d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.672+0000 7fd7b15bb700 1 -- 172.21.15.156:0/2961662012 shutdown_connections 2023-11-28T11:09:57.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.672+0000 7fd7b15bb700 1 -- 172.21.15.156:0/2961662012 wait complete. 2023-11-28T11:09:57.675 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T11:09:57.685 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.804+0000 7f601918d700 1 Processor -- start 2023-11-28T11:09:57.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.804+0000 7f601918d700 1 -- start start 2023-11-28T11:09:57.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.805+0000 7f601918d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f60141658c0 0x7f6014167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.805+0000 7f601918d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601415d060 con 0x7f601406dee0 2023-11-28T11:09:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.805+0000 7f601918d700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601415d340 con 0x7f6014071040 2023-11-28T11:09:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.805+0000 7f601918d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6014161550 con 0x7f60141658c0 2023-11-28T11:09:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.805+0000 7f601359e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f60141658c0 0x7f6014167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.805+0000 7f601259c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f601406dee0 0x7f601406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49564/0 (socket says 172.21.15.156:49564) 2023-11-28T11:09:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.805+0000 7f601259c700 1 -- 172.21.15.156:0/3216519607 learned_addr learned my addr 172.21.15.156:0/3216519607 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f601359e700 1 -- 172.21.15.156:0/3216519607 >> v1:172.21.15.186:6789/0 conn(0x7f6014071040 legacy=0x7f6014071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f601359e700 1 -- 172.21.15.156:0/3216519607 >> v1:172.21.15.156:6789/0 conn(0x7f601406dee0 legacy=0x7f601406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f601359e700 1 -- 172.21.15.156:0/3216519607 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601415d620 con 0x7f60141658c0 2023-11-28T11:09:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f601359e700 1 --2- 172.21.15.156:0/3216519607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60141658c0 0x7f6014167ca0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f600c009d90 tx=0x7f600c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ade3276d8249af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f6011d9b700 1 -- 172.21.15.156:0/3216519607 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c00e070 con 0x7f60141658c0 2023-11-28T11:09:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f6011d9b700 1 -- 172.21.15.156:0/3216519607 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c00bbd0 con 0x7f60141658c0 2023-11-28T11:09:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f6011d9b700 1 -- 172.21.15.156:0/3216519607 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c012430 con 0x7f60141658c0 2023-11-28T11:09:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f601918d700 1 -- 172.21.15.156:0/3216519607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60141658c0 msgr2=0x7f6014167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f601918d700 1 --2- 172.21.15.156:0/3216519607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60141658c0 0x7f6014167ca0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f600c009d90 tx=0x7f600c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f601918d700 1 -- 172.21.15.156:0/3216519607 shutdown_connections 2023-11-28T11:09:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f601918d700 1 --2- 172.21.15.156:0/3216519607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60141658c0 0x7f6014167ca0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f601918d700 1 -- 172.21.15.156:0/3216519607 >> 172.21.15.156:0/3216519607 conn(0x7f60140d0000 msgr2=0x7f60140d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f601918d700 1 -- 172.21.15.156:0/3216519607 shutdown_connections 2023-11-28T11:09:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.806+0000 7f601918d700 1 -- 172.21.15.156:0/3216519607 wait complete. 2023-11-28T11:09:57.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.808+0000 7f601918d700 1 Processor -- start 2023-11-28T11:09:57.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.808+0000 7f601918d700 1 -- start start 2023-11-28T11:09:57.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601918d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f601406dee0 0x7f60140b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601918d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6014071040 0x7f60140b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601918d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60141658c0 0x7f60140b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601918d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60140c4760 con 0x7f60141658c0 2023-11-28T11:09:57.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601918d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60140c4a00 con 0x7f601406dee0 2023-11-28T11:09:57.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601918d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60140b5e20 con 0x7f6014071040 2023-11-28T11:09:57.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601359e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60141658c0 0x7f60140b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601359e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60141658c0 0x7f60140b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34694/0 (socket says 172.21.15.156:34694) 2023-11-28T11:09:57.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601359e700 1 -- 172.21.15.156:0/2827271516 learned_addr learned my addr 172.21.15.156:0/2827271516 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:57.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601259c700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6014071040 0x7f60140b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f6012d9d700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f601406dee0 0x7f60140b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601359e700 1 -- 172.21.15.156:0/2827271516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6014071040 msgr2=0x7f60140b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601359e700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6014071040 0x7f60140b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601359e700 1 -- 172.21.15.156:0/2827271516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f601406dee0 msgr2=0x7f60140b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601359e700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f601406dee0 0x7f60140b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601359e700 1 -- 172.21.15.156:0/2827271516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c009a70 con 0x7f60141658c0 2023-11-28T11:09:57.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601359e700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60141658c0 0x7f60140b58e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f600c014040 tx=0x7f600c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f5ffb7fe700 1 -- 172.21.15.156:0/2827271516 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c0098f0 con 0x7f60141658c0 2023-11-28T11:09:57.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f5ffb7fe700 1 -- 172.21.15.156:0/2827271516 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c020070 con 0x7f60141658c0 2023-11-28T11:09:57.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601918d700 1 -- 172.21.15.156:0/2827271516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601415ccd0 con 0x7f60141658c0 2023-11-28T11:09:57.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f5ffb7fe700 1 -- 172.21.15.156:0/2827271516 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c004280 con 0x7f60141658c0 2023-11-28T11:09:57.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.809+0000 7f601918d700 1 -- 172.21.15.156:0/2827271516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601415d900 con 0x7f60141658c0 2023-11-28T11:09:57.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.810+0000 7f601918d700 1 -- 172.21.15.156:0/2827271516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601406a270 con 0x7f60141658c0 2023-11-28T11:09:57.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.811+0000 7f5ffb7fe700 1 -- 172.21.15.156:0/2827271516 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f600c0043e0 con 0x7f60141658c0 2023-11-28T11:09:57.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.811+0000 7f5ffb7fe700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60000682f0 0x7f600006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.811+0000 7f6012d9d700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60000682f0 0x7f600006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.811+0000 7f5ffb7fe700 1 -- 172.21.15.156:0/2827271516 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f600c022070 con 0x7f60141658c0 2023-11-28T11:09:57.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.812+0000 7f6012d9d700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60000682f0 0x7f600006a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6008000f60 tx=0x7f6008009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:57.816+0000 7f5ffb7fe700 1 -- 172.21.15.156:0/2827271516 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f600c054ab0 con 0x7f60141658c0 2023-11-28T11:09:58.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.000+0000 7f601918d700 1 -- 172.21.15.156:0/2827271516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60140c4b40 con 0x7f60141658c0 2023-11-28T11:09:58.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.001+0000 7f5ffb7fe700 1 -- 172.21.15.156:0/2827271516 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f600c0580d0 con 0x7f60141658c0 2023-11-28T11:09:58.003 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:58.003 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":49,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:09:55.275682+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:52.331853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:53.232328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:58.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.003+0000 7f601918d700 1 -- 172.21.15.156:0/2827271516 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60000682f0 msgr2=0x7f600006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.003+0000 7f601918d700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60000682f0 0x7f600006a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6008000f60 tx=0x7f6008009040 comp rx=0 tx=0).stop 2023-11-28T11:09:58.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.003+0000 7f601918d700 1 -- 172.21.15.156:0/2827271516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60141658c0 msgr2=0x7f60140b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.003+0000 7f601918d700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60141658c0 0x7f60140b58e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f600c014040 tx=0x7f600c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.003+0000 7f601918d700 1 -- 172.21.15.156:0/2827271516 shutdown_connections 2023-11-28T11:09:58.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.003+0000 7f601918d700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60141658c0 0x7f60140b58e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.003+0000 7f601918d700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f601406dee0 0x7f60140b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.003+0000 7f601918d700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60000682f0 0x7f600006a7a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.003+0000 7f601918d700 1 --2- 172.21.15.156:0/2827271516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6014071040 0x7f60140b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.003+0000 7f601918d700 1 -- 172.21.15.156:0/2827271516 >> 172.21.15.156:0/2827271516 conn(0x7f60140d0000 msgr2=0x7f601415e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:58.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.003+0000 7f601918d700 1 -- 172.21.15.156:0/2827271516 shutdown_connections 2023-11-28T11:09:58.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.003+0000 7f601918d700 1 -- 172.21.15.156:0/2827271516 wait complete. 2023-11-28T11:09:58.016 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:09:58.017 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:09:58.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.136+0000 7fd4adb48700 1 Processor -- start 2023-11-28T11:09:58.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.137+0000 7fd4adb48700 1 -- start start 2023-11-28T11:09:58.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.137+0000 7fd4adb48700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4a8071040 0x7fd4a8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.137+0000 7fd4adb48700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd4a806dee0 0x7fd4a806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.137+0000 7fd4adb48700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a8071950 con 0x7fd4a806cdc0 2023-11-28T11:09:58.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.137+0000 7fd4adb48700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a8161550 con 0x7fd4a806dee0 2023-11-28T11:09:58.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.137+0000 7fd4adb48700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a815d060 con 0x7fd4a8071040 2023-11-28T11:09:58.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.137+0000 7fd4a7fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd4a806cdc0 0x7fd4a806d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49580/0 (socket says 172.21.15.156:49580) 2023-11-28T11:09:58.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.137+0000 7fd4a7fff700 1 -- 172.21.15.156:0/3589082044 learned_addr learned my addr 172.21.15.156:0/3589082044 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:58.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a77fe700 1 --2- 172.21.15.156:0/3589082044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4a8071040 0x7fd4a8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a6ffd700 1 --2- 172.21.15.156:0/3589082044 >> v2:172.21.15.186:3300/0 conn(0x7fd4a806dee0 0x7fd4a806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391733065 0 0) 0x7fd4a8071950 con 0x7fd4a806cdc0 2023-11-28T11:09:58.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4940035b0 con 0x7fd4a806cdc0 2023-11-28T11:09:58.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1304812497 0 0) 0x7fd4940035b0 con 0x7fd4a806cdc0 2023-11-28T11:09:58.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4a8071040 msgr2=0x7fd4a8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:58.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 --2- 172.21.15.156:0/3589082044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4a8071040 0x7fd4a8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 >> v2:172.21.15.186:3300/0 conn(0x7fd4a806dee0 msgr2=0x7fd4a806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:58.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 --2- 172.21.15.156:0/3589082044 >> v2:172.21.15.186:3300/0 conn(0x7fd4a806dee0 0x7fd4a806e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a815d340 con 0x7fd4a806cdc0 2023-11-28T11:09:58.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a6ffd700 1 --2- 172.21.15.156:0/3589082044 >> v2:172.21.15.186:3300/0 conn(0x7fd4a806dee0 0x7fd4a806e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:58.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fd4a00019f0 con 0x7fd4a806cdc0 2023-11-28T11:09:58.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 >> v1:172.21.15.156:6789/0 conn(0x7fd4a806cdc0 legacy=0x7fd4a806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 --2- 172.21.15.156:0/3589082044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd494003a00 0x7fd494003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 --2- 172.21.15.156:0/3589082044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4a8071040 0x7fd494004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 --2- 172.21.15.156:0/3589082044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd494004d90 0x7fd494005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a8161550 con 0x7fd494004d90 2023-11-28T11:09:58.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a815d060 con 0x7fd494003a00 2023-11-28T11:09:58.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49400ba90 con 0x7fd4a8071040 2023-11-28T11:09:58.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a7fff700 1 --2- 172.21.15.156:0/3589082044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd494004d90 0x7fd494005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a77fe700 1 --2- 172.21.15.156:0/3589082044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd494003a00 0x7fd494003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a6ffd700 1 --2- 172.21.15.156:0/3589082044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4a8071040 0x7fd494004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a7fff700 1 -- 172.21.15.156:0/3589082044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4a8071040 msgr2=0x7fd494004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:58.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a7fff700 1 --2- 172.21.15.156:0/3589082044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4a8071040 0x7fd494004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a7fff700 1 -- 172.21.15.156:0/3589082044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd494003a00 msgr2=0x7fd494003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:58.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a7fff700 1 --2- 172.21.15.156:0/3589082044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd494003a00 0x7fd494003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.138+0000 7fd4a7fff700 1 -- 172.21.15.156:0/3589082044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4940035b0 con 0x7fd494004d90 2023-11-28T11:09:58.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.139+0000 7fd4a7fff700 1 --2- 172.21.15.156:0/3589082044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd494004d90 0x7fd494005200 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000a8d0 tx=0x7fd494005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5096bf78724dbf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.139+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a000a340 con 0x7fd494004d90 2023-11-28T11:09:58.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.139+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a0012600 con 0x7fd494004d90 2023-11-28T11:09:58.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.139+0000 7fd4a67fc700 1 -- 172.21.15.156:0/3589082044 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00129b0 con 0x7fd494004d90 2023-11-28T11:09:58.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.139+0000 7fd4adb48700 1 -- 172.21.15.156:0/3589082044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd494004d90 msgr2=0x7fd494005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.139+0000 7fd4adb48700 1 --2- 172.21.15.156:0/3589082044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd494004d90 0x7fd494005200 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000a8d0 tx=0x7fd494005880 comp rx=0 tx=0).stop 2023-11-28T11:09:58.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.139+0000 7fd4a7fff700 1 -- 172.21.15.156:0/3589082044 reap_dead start 2023-11-28T11:09:58.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.139+0000 7fd4adb48700 1 -- 172.21.15.156:0/3589082044 shutdown_connections 2023-11-28T11:09:58.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.139+0000 7fd4adb48700 1 -- 172.21.15.156:0/3589082044 >> 172.21.15.156:0/3589082044 conn(0x7fd4a80d0000 msgr2=0x7fd4a80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:58.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.139+0000 7fd4adb48700 1 -- 172.21.15.156:0/3589082044 shutdown_connections 2023-11-28T11:09:58.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.139+0000 7fd4adb48700 1 -- 172.21.15.156:0/3589082044 wait complete. 2023-11-28T11:09:58.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.141+0000 7fd4adb48700 1 Processor -- start 2023-11-28T11:09:58.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.141+0000 7fd4adb48700 1 -- start start 2023-11-28T11:09:58.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.141+0000 7fd4adb48700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a80c57e0 0x7fd4a81497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.141+0000 7fd4adb48700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4a814a020 0x7fd4a80c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4adb48700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a815f160 0x7fd4a815f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4adb48700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a80b2f50 con 0x7fd4a80c57e0 2023-11-28T11:09:58.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4adb48700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a815fb10 con 0x7fd4a815f160 2023-11-28T11:09:58.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4adb48700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a814b160 con 0x7fd4a814a020 2023-11-28T11:09:58.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4a77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a80c57e0 0x7fd4a81497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4a77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a80c57e0 0x7fd4a81497d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34710/0 (socket says 172.21.15.156:34710) 2023-11-28T11:09:58.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4a77fe700 1 -- 172.21.15.156:0/4279872837 learned_addr learned my addr 172.21.15.156:0/4279872837 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:58.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4a77fe700 1 -- 172.21.15.156:0/4279872837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4a814a020 msgr2=0x7fd4a80c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4a77fe700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4a814a020 0x7fd4a80c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4a7fff700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a815f160 0x7fd4a815f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4a77fe700 1 -- 172.21.15.156:0/4279872837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a815f160 msgr2=0x7fd4a815f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4a77fe700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a815f160 0x7fd4a815f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4a77fe700 1 -- 172.21.15.156:0/4279872837 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a815d900 con 0x7fd4a80c57e0 2023-11-28T11:09:58.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4a77fe700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a80c57e0 0x7fd4a81497d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd498011a00 tx=0x7fd498011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd49ffff700 1 -- 172.21.15.156:0/4279872837 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49800f8e0 con 0x7fd4a80c57e0 2023-11-28T11:09:58.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd49ffff700 1 -- 172.21.15.156:0/4279872837 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49800fa40 con 0x7fd4a80c57e0 2023-11-28T11:09:58.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd49ffff700 1 -- 172.21.15.156:0/4279872837 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49800fdd0 con 0x7fd4a80c57e0 2023-11-28T11:09:58.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4adb48700 1 -- 172.21.15.156:0/4279872837 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a815d620 con 0x7fd4a80c57e0 2023-11-28T11:09:58.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.142+0000 7fd4adb48700 1 -- 172.21.15.156:0/4279872837 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a806e890 con 0x7fd4a80c57e0 2023-11-28T11:09:58.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.143+0000 7fd4adb48700 1 -- 172.21.15.156:0/4279872837 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a814b440 con 0x7fd4a80c57e0 2023-11-28T11:09:58.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.144+0000 7fd49ffff700 1 -- 172.21.15.156:0/4279872837 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd498014440 con 0x7fd4a80c57e0 2023-11-28T11:09:58.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.144+0000 7fd49ffff700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd4880682f0 0x7fd48806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.144+0000 7fd49ffff700 1 -- 172.21.15.156:0/4279872837 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd498086580 con 0x7fd4a80c57e0 2023-11-28T11:09:58.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.144+0000 7fd4a6ffd700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd4880682f0 0x7fd48806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.145+0000 7fd4a6ffd700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd4880682f0 0x7fd48806a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd4a81f5c10 tx=0x7fd490009820 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.148+0000 7fd49ffff700 1 -- 172.21.15.156:0/4279872837 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd498052d40 con 0x7fd4a80c57e0 2023-11-28T11:09:58.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.328+0000 7fd4adb48700 1 -- 172.21.15.156:0/4279872837 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4a814b2a0 con 0x7fd4880682f0 2023-11-28T11:09:58.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.336+0000 7fd49ffff700 1 -- 172.21.15.156:0/4279872837 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302313 (secure 0 0 0) 0x7fd4900099b0 con 0x7fd4880682f0 2023-11-28T11:09:58.345 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:58.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.339+0000 7fd4adb48700 1 -- 172.21.15.156:0/4279872837 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd4880682f0 msgr2=0x7fd48806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.339+0000 7fd4adb48700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd4880682f0 0x7fd48806a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd4a81f5c10 tx=0x7fd490009820 comp rx=0 tx=0).stop 2023-11-28T11:09:58.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.339+0000 7fd4adb48700 1 -- 172.21.15.156:0/4279872837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a80c57e0 msgr2=0x7fd4a81497d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.339+0000 7fd4adb48700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a80c57e0 0x7fd4a81497d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd498011a00 tx=0x7fd498011dc0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.340+0000 7fd4adb48700 1 -- 172.21.15.156:0/4279872837 shutdown_connections 2023-11-28T11:09:58.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.340+0000 7fd4adb48700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a815f160 0x7fd4a815f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.340+0000 7fd4adb48700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a80c57e0 0x7fd4a81497d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.340+0000 7fd4adb48700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd4880682f0 0x7fd48806a7a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.340+0000 7fd4adb48700 1 --2- 172.21.15.156:0/4279872837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4a814a020 0x7fd4a80c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.340+0000 7fd4adb48700 1 -- 172.21.15.156:0/4279872837 >> 172.21.15.156:0/4279872837 conn(0x7fd4a80d0000 msgr2=0x7fd4a806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:58.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.340+0000 7fd4adb48700 1 -- 172.21.15.156:0/4279872837 shutdown_connections 2023-11-28T11:09:58.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.340+0000 7fd4adb48700 1 -- 172.21.15.156:0/4279872837 wait complete. 2023-11-28T11:09:58.349 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:09:58.353 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-11-28T11:09:56.546294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591972,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":181,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":94,"acting":94,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":73096,"kb_used_data":5384,"kb_used_omap":0,"kb_used_meta":67648,"kb_avail":754901624,"statfs":{"total":773094113280,"available":773019262976,"internally_reserved":0,"allocated":5513216,"data_stored":2997040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":69271552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1604,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000641"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278973+0000","last_change":"2023-11-28T11:09:54.292339+0000","last_active":"2023-11-28T11:09:55.278973+0000","last_peered":"2023-11-28T11:09:55.278973+0000","last_clean":"2023-11-28T11:09:55.278973+0000","last_became_active":"2023-11-28T11:09:54.292122+0000","last_became_peered":"2023-11-28T11:09:54.292122+0000","last_unstale":"2023-11-28T11:09:55.278973+0000","last_undegraded":"2023-11-28T11:09:55.278973+0000","last_fullsized":"2023-11-28T11:09:55.278973+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:27.615241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278709+0000","last_change":"2023-11-28T11:09:54.294027+0000","last_active":"2023-11-28T11:09:55.278709+0000","last_peered":"2023-11-28T11:09:55.278709+0000","last_clean":"2023-11-28T11:09:55.278709+0000","last_became_active":"2023-11-28T11:09:54.293389+0000","last_became_peered":"2023-11-28T11:09:54.293389+0000","last_unstale":"2023-11-28T11:09:55.278709+0000","last_undegraded":"2023-11-28T11:09:55.278709+0000","last_fullsized":"2023-11-28T11:09:55.278709+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:34.271243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864548+0000","last_change":"2023-11-28T11:09:54.292815+0000","last_active":"2023-11-28T11:09:55.864548+0000","last_peered":"2023-11-28T11:09:55.864548+0000","last_clean":"2023-11-28T11:09:55.864548+0000","last_became_active":"2023-11-28T11:09:54.292575+0000","last_became_peered":"2023-11-28T11:09:54.292575+0000","last_unstale":"2023-11-28T11:09:55.864548+0000","last_undegraded":"2023-11-28T11:09:55.864548+0000","last_fullsized":"2023-11-28T11:09:55.864548+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:34.271243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865009+0000","last_change":"2023-11-28T11:09:54.294161+0000","last_active":"2023-11-28T11:09:55.865009+0000","last_peered":"2023-11-28T11:09:55.865009+0000","last_clean":"2023-11-28T11:09:55.865009+0000","last_became_active":"2023-11-28T11:09:54.293771+0000","last_became_peered":"2023-11-28T11:09:54.293771+0000","last_unstale":"2023-11-28T11:09:55.865009+0000","last_undegraded":"2023-11-28T11:09:55.865009+0000","last_fullsized":"2023-11-28T11:09:55.865009+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:53:35.361188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864870+0000","last_change":"2023-11-28T11:09:54.297459+0000","last_active":"2023-11-28T11:09:55.864870+0000","last_peered":"2023-11-28T11:09:55.864870+0000","last_clean":"2023-11-28T11:09:55.864870+0000","last_became_active":"2023-11-28T11:09:54.297178+0000","last_became_peered":"2023-11-28T11:09:54.297178+0000","last_unstale":"2023-11-28T11:09:55.864870+0000","last_undegraded":"2023-11-28T11:09:55.864870+0000","last_fullsized":"2023-11-28T11:09:55.864870+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:27.615241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279060+0000","last_change":"2023-11-28T11:09:54.294297+0000","last_active":"2023-11-28T11:09:55.279060+0000","last_peered":"2023-11-28T11:09:55.279060+0000","last_clean":"2023-11-28T11:09:55.279060+0000","last_became_active":"2023-11-28T11:09:54.294059+0000","last_became_peered":"2023-11-28T11:09:54.294059+0000","last_unstale":"2023-11-28T11:09:55.279060+0000","last_undegraded":"2023-11-28T11:09:55.279060+0000","last_fullsized":"2023-11-28T11:09:55.279060+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:33.227523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279314+0000","last_change":"2023-11-28T11:09:54.299470+0000","last_active":"2023-11-28T11:09:55.279314+0000","last_peered":"2023-11-28T11:09:55.279314+0000","last_clean":"2023-11-28T11:09:55.279314+0000","last_became_active":"2023-11-28T11:09:54.299205+0000","last_became_peered":"2023-11-28T11:09:54.299205+0000","last_unstale":"2023-11-28T11:09:55.279314+0000","last_undegraded":"2023-11-28T11:09:55.279314+0000","last_fullsized":"2023-11-28T11:09:55.279314+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:35.037648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279060+0000","last_change":"2023-11-28T11:09:54.301917+0000","last_active":"2023-11-28T11:09:55.279060+0000","last_peered":"2023-11-28T11:09:55.279060+0000","last_clean":"2023-11-28T11:09:55.279060+0000","last_became_active":"2023-11-28T11:09:54.301657+0000","last_became_peered":"2023-11-28T11:09:54.301657+0000","last_unstale":"2023-11-28T11:09:55.279060+0000","last_undegraded":"2023-11-28T11:09:55.279060+0000","last_fullsized":"2023-11-28T11:09:55.279060+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:56.200657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864664+0000","last_change":"2023-11-28T11:09:54.295880+0000","last_active":"2023-11-28T11:09:55.864664+0000","last_peered":"2023-11-28T11:09:55.864664+0000","last_clean":"2023-11-28T11:09:55.864664+0000","last_became_active":"2023-11-28T11:09:54.295677+0000","last_became_peered":"2023-11-28T11:09:54.295677+0000","last_unstale":"2023-11-28T11:09:55.864664+0000","last_undegraded":"2023-11-28T11:09:55.864664+0000","last_fullsized":"2023-11-28T11:09:55.864664+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:56.200657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278660+0000","last_change":"2023-11-28T11:09:54.294262+0000","last_active":"2023-11-28T11:09:55.278660+0000","last_peered":"2023-11-28T11:09:55.278660+0000","last_clean":"2023-11-28T11:09:55.278660+0000","last_became_active":"2023-11-28T11:09:54.293416+0000","last_became_peered":"2023-11-28T11:09:54.293416+0000","last_unstale":"2023-11-28T11:09:55.278660+0000","last_undegraded":"2023-11-28T11:09:55.278660+0000","last_fullsized":"2023-11-28T11:09:55.278660+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:00.934162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279330+0000","last_change":"2023-11-28T11:09:54.295861+0000","last_active":"2023-11-28T11:09:55.279330+0000","last_peered":"2023-11-28T11:09:55.279330+0000","last_clean":"2023-11-28T11:09:55.279330+0000","last_became_active":"2023-11-28T11:09:54.295542+0000","last_became_peered":"2023-11-28T11:09:54.295542+0000","last_unstale":"2023-11-28T11:09:55.279330+0000","last_undegraded":"2023-11-28T11:09:55.279330+0000","last_fullsized":"2023-11-28T11:09:55.279330+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:51:33.313288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864877+0000","last_change":"2023-11-28T11:09:54.287429+0000","last_active":"2023-11-28T11:09:55.864877+0000","last_peered":"2023-11-28T11:09:55.864877+0000","last_clean":"2023-11-28T11:09:55.864877+0000","last_became_active":"2023-11-28T11:09:54.287039+0000","last_became_peered":"2023-11-28T11:09:54.287039+0000","last_unstale":"2023-11-28T11:09:55.864877+0000","last_undegraded":"2023-11-28T11:09:55.864877+0000","last_fullsized":"2023-11-28T11:09:55.864877+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:05:28.432364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278747+0000","last_change":"2023-11-28T11:09:54.295532+0000","last_active":"2023-11-28T11:09:55.278747+0000","last_peered":"2023-11-28T11:09:55.278747+0000","last_clean":"2023-11-28T11:09:55.278747+0000","last_became_active":"2023-11-28T11:09:54.295277+0000","last_became_peered":"2023-11-28T11:09:54.295277+0000","last_unstale":"2023-11-28T11:09:55.278747+0000","last_undegraded":"2023-11-28T11:09:55.278747+0000","last_fullsized":"2023-11-28T11:09:55.278747+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:05:28.432364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278808+0000","last_change":"2023-11-28T11:09:54.294192+0000","last_active":"2023-11-28T11:09:55.278808+0000","last_peered":"2023-11-28T11:09:55.278808+0000","last_clean":"2023-11-28T11:09:55.278808+0000","last_became_active":"2023-11-28T11:09:54.293689+0000","last_became_peered":"2023-11-28T11:09:54.293689+0000","last_unstale":"2023-11-28T11:09:55.278808+0000","last_undegraded":"2023-11-28T11:09:55.278808+0000","last_fullsized":"2023-11-28T11:09:55.278808+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:24:24.473745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864741+0000","last_change":"2023-11-28T11:09:54.286450+0000","last_active":"2023-11-28T11:09:55.864741+0000","last_peered":"2023-11-28T11:09:55.864741+0000","last_clean":"2023-11-28T11:09:55.864741+0000","last_became_active":"2023-11-28T11:09:54.286188+0000","last_became_peered":"2023-11-28T11:09:54.286188+0000","last_unstale":"2023-11-28T11:09:55.864741+0000","last_undegraded":"2023-11-28T11:09:55.864741+0000","last_fullsized":"2023-11-28T11:09:55.864741+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:43:19.089769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278828+0000","last_change":"2023-11-28T11:09:53.893375+0000","last_active":"2023-11-28T11:09:55.278828+0000","last_peered":"2023-11-28T11:09:55.278828+0000","last_clean":"2023-11-28T11:09:55.278828+0000","last_became_active":"2023-11-28T11:09:53.893090+0000","last_became_peered":"2023-11-28T11:09:53.893090+0000","last_unstale":"2023-11-28T11:09:55.278828+0000","last_undegraded":"2023-11-28T11:09:55.278828+0000","last_fullsized":"2023-11-28T11:09:55.278828+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:51.766776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864977+0000","last_change":"2023-11-28T11:09:54.287276+0000","last_active":"2023-11-28T11:09:55.864977+0000","last_peered":"2023-11-28T11:09:55.864977+0000","last_clean":"2023-11-28T11:09:55.864977+0000","last_became_active":"2023-11-28T11:09:54.286712+0000","last_became_peered":"2023-11-28T11:09:54.286712+0000","last_unstale":"2023-11-28T11:09:55.864977+0000","last_undegraded":"2023-11-28T11:09:55.864977+0000","last_fullsized":"2023-11-28T11:09:55.864977+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:36:35.247278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279126+0000","last_change":"2023-11-28T11:09:53.882587+0000","last_active":"2023-11-28T11:09:55.279126+0000","last_peered":"2023-11-28T11:09:55.279126+0000","last_clean":"2023-11-28T11:09:55.279126+0000","last_became_active":"2023-11-28T11:09:53.881772+0000","last_became_peered":"2023-11-28T11:09:53.881772+0000","last_unstale":"2023-11-28T11:09:55.279126+0000","last_undegraded":"2023-11-28T11:09:55.279126+0000","last_fullsized":"2023-11-28T11:09:55.279126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:17:58.083155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279208+0000","last_change":"2023-11-28T11:09:54.297393+0000","last_active":"2023-11-28T11:09:55.279208+0000","last_peered":"2023-11-28T11:09:55.279208+0000","last_clean":"2023-11-28T11:09:55.279208+0000","last_became_active":"2023-11-28T11:09:54.296960+0000","last_became_peered":"2023-11-28T11:09:54.296960+0000","last_unstale":"2023-11-28T11:09:55.279208+0000","last_undegraded":"2023-11-28T11:09:55.279208+0000","last_fullsized":"2023-11-28T11:09:55.279208+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:03:51.215320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864964+0000","last_change":"2023-11-28T11:09:53.887028+0000","last_active":"2023-11-28T11:09:55.864964+0000","last_peered":"2023-11-28T11:09:55.864964+0000","last_clean":"2023-11-28T11:09:55.864964+0000","last_became_active":"2023-11-28T11:09:53.886756+0000","last_became_peered":"2023-11-28T11:09:53.886756+0000","last_unstale":"2023-11-28T11:09:55.864964+0000","last_undegraded":"2023-11-28T11:09:55.864964+0000","last_fullsized":"2023-11-28T11:09:55.864964+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:38.043246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278914+0000","last_change":"2023-11-28T11:09:53.892691+0000","last_active":"2023-11-28T11:09:55.278914+0000","last_peered":"2023-11-28T11:09:55.278914+0000","last_clean":"2023-11-28T11:09:55.278914+0000","last_became_active":"2023-11-28T11:09:53.892409+0000","last_became_peered":"2023-11-28T11:09:53.892409+0000","last_unstale":"2023-11-28T11:09:55.278914+0000","last_undegraded":"2023-11-28T11:09:55.278914+0000","last_fullsized":"2023-11-28T11:09:55.278914+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:03.817799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279159+0000","last_change":"2023-11-28T11:09:54.290831+0000","last_active":"2023-11-28T11:09:55.279159+0000","last_peered":"2023-11-28T11:09:55.279159+0000","last_clean":"2023-11-28T11:09:55.279159+0000","last_became_active":"2023-11-28T11:09:54.290270+0000","last_became_peered":"2023-11-28T11:09:54.290270+0000","last_unstale":"2023-11-28T11:09:55.279159+0000","last_undegraded":"2023-11-28T11:09:55.279159+0000","last_fullsized":"2023-11-28T11:09:55.279159+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:08:40.328821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864903+0000","last_change":"2023-11-28T11:09:54.296132+0000","last_active":"2023-11-28T11:09:55.864903+0000","last_peered":"2023-11-28T11:09:55.864903+0000","last_clean":"2023-11-28T11:09:55.864903+0000","last_became_active":"2023-11-28T11:09:54.295853+0000","last_became_peered":"2023-11-28T11:09:54.295853+0000","last_unstale":"2023-11-28T11:09:55.864903+0000","last_undegraded":"2023-11-28T11:09:55.864903+0000","last_fullsized":"2023-11-28T11:09:55.864903+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:35.037648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279265+0000","last_change":"2023-11-28T11:09:54.291597+0000","last_active":"2023-11-28T11:09:55.279265+0000","last_peered":"2023-11-28T11:09:55.279265+0000","last_clean":"2023-11-28T11:09:55.279265+0000","last_became_active":"2023-11-28T11:09:54.291136+0000","last_became_peered":"2023-11-28T11:09:54.291136+0000","last_unstale":"2023-11-28T11:09:55.279265+0000","last_undegraded":"2023-11-28T11:09:55.279265+0000","last_fullsized":"2023-11-28T11:09:55.279265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:53:35.361188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":19,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864751+0000","last_change":"2023-11-28T11:09:53.886321+0000","last_active":"2023-11-28T11:09:55.864751+0000","last_peered":"2023-11-28T11:09:55.864751+0000","last_clean":"2023-11-28T11:09:55.864751+0000","last_became_active":"2023-11-28T11:09:53.886065+0000","last_became_peered":"2023-11-28T11:09:53.886065+0000","last_unstale":"2023-11-28T11:09:55.864751+0000","last_undegraded":"2023-11-28T11:09:55.864751+0000","last_fullsized":"2023-11-28T11:09:55.864751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:59.962919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864604+0000","last_change":"2023-11-28T11:09:54.293936+0000","last_active":"2023-11-28T11:09:55.864604+0000","last_peered":"2023-11-28T11:09:55.864604+0000","last_clean":"2023-11-28T11:09:55.864604+0000","last_became_active":"2023-11-28T11:09:54.293615+0000","last_became_peered":"2023-11-28T11:09:54.293615+0000","last_unstale":"2023-11-28T11:09:55.864604+0000","last_undegraded":"2023-11-28T11:09:55.864604+0000","last_fullsized":"2023-11-28T11:09:55.864604+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:38:19.762032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279214+0000","last_change":"2023-11-28T11:09:53.892731+0000","last_active":"2023-11-28T11:09:55.279214+0000","last_peered":"2023-11-28T11:09:55.279214+0000","last_clean":"2023-11-28T11:09:55.279214+0000","last_became_active":"2023-11-28T11:09:53.892494+0000","last_became_peered":"2023-11-28T11:09:53.892494+0000","last_unstale":"2023-11-28T11:09:55.279214+0000","last_undegraded":"2023-11-28T11:09:55.279214+0000","last_fullsized":"2023-11-28T11:09:55.279214+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:45.115674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864633+0000","last_change":"2023-11-28T11:09:53.887902+0000","last_active":"2023-11-28T11:09:55.864633+0000","last_peered":"2023-11-28T11:09:55.864633+0000","last_clean":"2023-11-28T11:09:55.864633+0000","last_became_active":"2023-11-28T11:09:53.887657+0000","last_became_peered":"2023-11-28T11:09:53.887657+0000","last_unstale":"2023-11-28T11:09:55.864633+0000","last_undegraded":"2023-11-28T11:09:55.864633+0000","last_fullsized":"2023-11-28T11:09:55.864633+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:09.088415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279058+0000","last_change":"2023-11-28T11:09:54.296684+0000","last_active":"2023-11-28T11:09:55.279058+0000","last_peered":"2023-11-28T11:09:55.279058+0000","last_clean":"2023-11-28T11:09:55.279058+0000","last_became_active":"2023-11-28T11:09:54.296456+0000","last_became_peered":"2023-11-28T11:09:54.296456+0000","last_unstale":"2023-11-28T11:09:55.279058+0000","last_undegraded":"2023-11-28T11:09:55.279058+0000","last_fullsized":"2023-11-28T11:09:55.279058+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:43:19.089769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278722+0000","last_change":"2023-11-28T11:09:53.893080+0000","last_active":"2023-11-28T11:09:55.278722+0000","last_peered":"2023-11-28T11:09:55.278722+0000","last_clean":"2023-11-28T11:09:55.278722+0000","last_became_active":"2023-11-28T11:09:53.892644+0000","last_became_peered":"2023-11-28T11:09:53.892644+0000","last_unstale":"2023-11-28T11:09:55.278722+0000","last_undegraded":"2023-11-28T11:09:55.278722+0000","last_fullsized":"2023-11-28T11:09:55.278722+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:25:14.791056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278965+0000","last_change":"2023-11-28T11:09:54.294678+0000","last_active":"2023-11-28T11:09:55.278965+0000","last_peered":"2023-11-28T11:09:55.278965+0000","last_clean":"2023-11-28T11:09:55.278965+0000","last_became_active":"2023-11-28T11:09:54.294419+0000","last_became_peered":"2023-11-28T11:09:54.294419+0000","last_unstale":"2023-11-28T11:09:55.278965+0000","last_undegraded":"2023-11-28T11:09:55.278965+0000","last_fullsized":"2023-11-28T11:09:55.278965+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:18:40.259162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:09:49.011719+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:09:49.011719+0000","last_peered":"2023-11-28T11:09:49.011719+0000","last_clean":"2023-11-28T11:09:49.011719+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:09:49.011719+0000","last_undegraded":"2023-11-28T11:09:49.011719+0000","last_fullsized":"2023-11-28T11:09:49.011719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:09:50.063294+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:09:50.063294+0000","last_peered":"2023-11-28T11:09:50.063294+0000","last_clean":"2023-11-28T11:09:50.063294+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:09:50.063294+0000","last_undegraded":"2023-11-28T11:09:50.063294+0000","last_fullsized":"2023-11-28T11:09:50.063294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864707+0000","last_change":"2023-11-28T11:09:54.300740+0000","last_active":"2023-11-28T11:09:55.864707+0000","last_peered":"2023-11-28T11:09:55.864707+0000","last_clean":"2023-11-28T11:09:55.864707+0000","last_became_active":"2023-11-28T11:09:54.300478+0000","last_became_peered":"2023-11-28T11:09:54.300478+0000","last_unstale":"2023-11-28T11:09:55.864707+0000","last_undegraded":"2023-11-28T11:09:55.864707+0000","last_fullsized":"2023-11-28T11:09:55.864707+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:18:40.259162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279048+0000","last_change":"2023-11-28T11:09:53.888334+0000","last_active":"2023-11-28T11:09:55.279048+0000","last_peered":"2023-11-28T11:09:55.279048+0000","last_clean":"2023-11-28T11:09:55.279048+0000","last_became_active":"2023-11-28T11:09:53.887962+0000","last_became_peered":"2023-11-28T11:09:53.887962+0000","last_unstale":"2023-11-28T11:09:55.279048+0000","last_undegraded":"2023-11-28T11:09:55.279048+0000","last_fullsized":"2023-11-28T11:09:55.279048+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:16:31.069137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:09:49.073683+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:09:49.073683+0000","last_peered":"2023-11-28T11:09:49.073683+0000","last_clean":"2023-11-28T11:09:49.073683+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:09:49.073683+0000","last_undegraded":"2023-11-28T11:09:49.073683+0000","last_fullsized":"2023-11-28T11:09:49.073683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279132+0000","last_change":"2023-11-28T11:09:53.888284+0000","last_active":"2023-11-28T11:09:55.279132+0000","last_peered":"2023-11-28T11:09:55.279132+0000","last_clean":"2023-11-28T11:09:55.279132+0000","last_became_active":"2023-11-28T11:09:53.887999+0000","last_became_peered":"2023-11-28T11:09:53.887999+0000","last_unstale":"2023-11-28T11:09:55.279132+0000","last_undegraded":"2023-11-28T11:09:55.279132+0000","last_fullsized":"2023-11-28T11:09:55.279132+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:58.473943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:09:50.063294+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:09:50.063294+0000","last_peered":"2023-11-28T11:09:50.063294+0000","last_clean":"2023-11-28T11:09:50.063294+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:09:50.063294+0000","last_undegraded":"2023-11-28T11:09:50.063294+0000","last_fullsized":"2023-11-28T11:09:50.063294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":186,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:09:50.719750+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:09:50.719750+0000","last_peered":"2023-11-28T11:09:50.719750+0000","last_clean":"2023-11-28T11:09:50.719750+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:09:50.719750+0000","last_undegraded":"2023-11-28T11:09:50.719750+0000","last_fullsized":"2023-11-28T11:09:50.719750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:09:50.719804+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:09:50.719804+0000","last_peered":"2023-11-28T11:09:50.719804+0000","last_clean":"2023-11-28T11:09:50.719804+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:09:50.719804+0000","last_undegraded":"2023-11-28T11:09:50.719804+0000","last_fullsized":"2023-11-28T11:09:50.719804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865047+0000","last_change":"2023-11-28T11:09:54.283364+0000","last_active":"2023-11-28T11:09:55.865047+0000","last_peered":"2023-11-28T11:09:55.865047+0000","last_clean":"2023-11-28T11:09:55.865047+0000","last_became_active":"2023-11-28T11:09:54.283070+0000","last_became_peered":"2023-11-28T11:09:54.283070+0000","last_unstale":"2023-11-28T11:09:55.865047+0000","last_undegraded":"2023-11-28T11:09:55.865047+0000","last_fullsized":"2023-11-28T11:09:55.865047+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:56.494742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:52.861641+0000","last_change":"2023-11-28T11:09:52.861641+0000","last_active":"2023-11-28T11:09:52.861641+0000","last_peered":"2023-11-28T11:09:52.861641+0000","last_clean":"2023-11-28T11:09:52.861641+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:52.861641+0000","last_undegraded":"2023-11-28T11:09:52.861641+0000","last_fullsized":"2023-11-28T11:09:52.861641+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:09:50.719750+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:09:50.719750+0000","last_peered":"2023-11-28T11:09:50.719750+0000","last_clean":"2023-11-28T11:09:50.719750+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:09:50.719750+0000","last_undegraded":"2023-11-28T11:09:50.719750+0000","last_fullsized":"2023-11-28T11:09:50.719750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278906+0000","last_change":"2023-11-28T11:09:54.295175+0000","last_active":"2023-11-28T11:09:55.278906+0000","last_peered":"2023-11-28T11:09:55.278906+0000","last_clean":"2023-11-28T11:09:55.278906+0000","last_became_active":"2023-11-28T11:09:54.294883+0000","last_became_peered":"2023-11-28T11:09:54.294883+0000","last_unstale":"2023-11-28T11:09:55.278906+0000","last_undegraded":"2023-11-28T11:09:55.278906+0000","last_fullsized":"2023-11-28T11:09:55.278906+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:55.105792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864794+0000","last_change":"2023-11-28T11:09:53.887922+0000","last_active":"2023-11-28T11:09:55.864794+0000","last_peered":"2023-11-28T11:09:55.864794+0000","last_clean":"2023-11-28T11:09:55.864794+0000","last_became_active":"2023-11-28T11:09:53.887679+0000","last_became_peered":"2023-11-28T11:09:53.887679+0000","last_unstale":"2023-11-28T11:09:55.864794+0000","last_undegraded":"2023-11-28T11:09:55.864794+0000","last_fullsized":"2023-11-28T11:09:55.864794+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:56.044847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:09:49.011723+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:09:49.011723+0000","last_peered":"2023-11-28T11:09:49.011723+0000","last_clean":"2023-11-28T11:09:49.011723+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:09:49.011723+0000","last_undegraded":"2023-11-28T11:09:49.011723+0000","last_fullsized":"2023-11-28T11:09:49.011723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:09:53.865953+0000","last_change":"2023-11-28T11:09:53.865953+0000","last_active":"2023-11-28T11:09:53.865953+0000","last_peered":"2023-11-28T11:09:53.865953+0000","last_clean":"2023-11-28T11:09:53.865953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:09:53.865953+0000","last_undegraded":"2023-11-28T11:09:53.865953+0000","last_fullsized":"2023-11-28T11:09:53.865953+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864781+0000","last_change":"2023-11-28T11:09:53.883492+0000","last_active":"2023-11-28T11:09:55.864781+0000","last_peered":"2023-11-28T11:09:55.864781+0000","last_clean":"2023-11-28T11:09:55.864781+0000","last_became_active":"2023-11-28T11:09:53.883215+0000","last_became_peered":"2023-11-28T11:09:53.883215+0000","last_unstale":"2023-11-28T11:09:55.864781+0000","last_undegraded":"2023-11-28T11:09:55.864781+0000","last_fullsized":"2023-11-28T11:09:55.864781+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:16:31.069137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:09:49.017711+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:09:49.017711+0000","last_peered":"2023-11-28T11:09:49.017711+0000","last_clean":"2023-11-28T11:09:49.017711+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:09:49.017711+0000","last_undegraded":"2023-11-28T11:09:49.017711+0000","last_fullsized":"2023-11-28T11:09:49.017711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1604,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":24,"acting":24,"num_store_stats":2},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":52,"acting":52,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10000,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94361840,"statfs":{"total":96636764160,"available":96626524160,"internally_reserved":0,"allocated":1122304,"data_stored":811928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.55300000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.86199999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66300000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.97199999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.751},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.1319999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.95099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8652,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363188,"statfs":{"total":96636764160,"available":96627904512,"internally_reserved":0,"allocated":528384,"data_stored":221560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":5.1239999999999997,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.0410000000000004,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.76800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.0250000000000004,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":5.1509999999999998,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.79800000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":6.0620000000000003,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.9409999999999998,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.68100000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.64100000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.72599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8780,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363060,"statfs":{"total":96636764160,"available":96627773440,"internally_reserved":0,"allocated":528384,"data_stored":221560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.0519999999999996,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.376},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.248}]},{"osd":2,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":7.0890000000000004,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.35799999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.33000000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":18.024999999999999,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59999999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":17.966999999999999,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.54900000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":18.056999999999999,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.60799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10056,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94361784,"statfs":{"total":96636764160,"available":96626466816,"internally_reserved":0,"allocated":1179648,"data_stored":838563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.379},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.27700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.35499999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.40799999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.51500000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52200000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57599999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8716,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363124,"statfs":{"total":96636764160,"available":96627838976,"internally_reserved":0,"allocated":528384,"data_stored":221560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.54300000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.47999999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66300000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.58399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9164,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362676,"statfs":{"total":96636764160,"available":96627380224,"internally_reserved":0,"allocated":593920,"data_stored":249709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.31}]},{"osd":1,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.36499999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.72799999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69299999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.745}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":8896,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362944,"statfs":{"total":96636764160,"available":96627654656,"internally_reserved":0,"allocated":516096,"data_stored":216080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.65000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.623},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":12.263,"5min":12.263,"15min":12.263},"last":0.55300000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.66900000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.72699999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.443},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.53200000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60599999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.77700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8832,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94363008,"statfs":{"total":96636764160,"available":96627720192,"internally_reserved":0,"allocated":516096,"data_stored":216080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.60299999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.498}]},{"osd":2,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.67300000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.622}]},{"osd":4,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42999999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.41099999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:09:58.358 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:09:58.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.511+0000 7f877f337700 1 Processor -- start 2023-11-28T11:09:58.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.512+0000 7f877f337700 1 -- start start 2023-11-28T11:09:58.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.512+0000 7f877f337700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f87780cd8c0 0x7f8778140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.512+0000 7f877f337700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f877806f850 0x7f877806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.512+0000 7f877f337700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877813ff00 con 0x7f87780cdc90 2023-11-28T11:09:58.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.512+0000 7f877f337700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87781404c0 con 0x7f877806f850 2023-11-28T11:09:58.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.512+0000 7f877f337700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877815aa80 con 0x7f87780cd8c0 2023-11-28T11:09:58.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.512+0000 7f877d8d4700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f877806f850 0x7f877806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.512+0000 7f877c8d2700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f87780cdc90 0x7f87780b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49596/0 (socket says 172.21.15.156:49596) 2023-11-28T11:09:58.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.513+0000 7f877c8d2700 1 -- 172.21.15.156:0/963245640 learned_addr learned my addr 172.21.15.156:0/963245640 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:58.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.513+0000 7f877d8d4700 1 -- 172.21.15.156:0/963245640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f87780cd8c0 msgr2=0x7f8778140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:58.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.513+0000 7f877d0d3700 1 --2- 172.21.15.156:0/963245640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f87780cd8c0 0x7f8778140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.513+0000 7f876ffff700 1 -- 172.21.15.156:0/963245640 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605848444 0 0) 0x7f877813ff00 con 0x7f87780cdc90 2023-11-28T11:09:58.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.513+0000 7f877d8d4700 1 --2- 172.21.15.156:0/963245640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f87780cd8c0 0x7f8778140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.513+0000 7f877d8d4700 1 -- 172.21.15.156:0/963245640 >> v1:172.21.15.156:6789/0 conn(0x7f87780cdc90 legacy=0x7f87780b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.513+0000 7f877d8d4700 1 -- 172.21.15.156:0/963245640 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87780b3040 con 0x7f877806f850 2023-11-28T11:09:58.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.513+0000 7f877d8d4700 1 --2- 172.21.15.156:0/963245640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f877806f850 0x7f877806fcc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8770009d90 tx=0x7f877000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8653e41041f671f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.513+0000 7f876ffff700 1 -- 172.21.15.156:0/963245640 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877000e070 con 0x7f877806f850 2023-11-28T11:09:58.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.513+0000 7f876ffff700 1 -- 172.21.15.156:0/963245640 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877000bbd0 con 0x7f877806f850 2023-11-28T11:09:58.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.513+0000 7f876ffff700 1 -- 172.21.15.156:0/963245640 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770012430 con 0x7f877806f850 2023-11-28T11:09:58.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.514+0000 7f877f337700 1 -- 172.21.15.156:0/963245640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f877806f850 msgr2=0x7f877806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.514+0000 7f877f337700 1 --2- 172.21.15.156:0/963245640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f877806f850 0x7f877806fcc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8770009d90 tx=0x7f877000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:58.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.514+0000 7f877f337700 1 -- 172.21.15.156:0/963245640 shutdown_connections 2023-11-28T11:09:58.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.514+0000 7f877f337700 1 --2- 172.21.15.156:0/963245640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f877806f850 0x7f877806fcc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.514+0000 7f877f337700 1 --2- 172.21.15.156:0/963245640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f87780cd8c0 0x7f8778140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.514+0000 7f877f337700 1 -- 172.21.15.156:0/963245640 >> 172.21.15.156:0/963245640 conn(0x7f877806aed0 msgr2=0x7f877806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:58.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.514+0000 7f877f337700 1 -- 172.21.15.156:0/963245640 shutdown_connections 2023-11-28T11:09:58.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.514+0000 7f877f337700 1 -- 172.21.15.156:0/963245640 wait complete. 2023-11-28T11:09:58.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.515+0000 7f877f337700 1 Processor -- start 2023-11-28T11:09:58.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.515+0000 7f877f337700 1 -- start start 2023-11-28T11:09:58.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877f337700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f877806f850 0x7f8778156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877f337700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87780cd8c0 0x7f87780b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877f337700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87780cdc90 0x7f87780b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877f337700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87780b22c0 con 0x7f87780cdc90 2023-11-28T11:09:58.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877f337700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87780b1fe0 con 0x7f87780cd8c0 2023-11-28T11:09:58.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877f337700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87780c8c50 con 0x7f877806f850 2023-11-28T11:09:58.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877d8d4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87780cdc90 0x7f87780b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877d8d4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87780cdc90 0x7f87780b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34726/0 (socket says 172.21.15.156:34726) 2023-11-28T11:09:58.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877d8d4700 1 -- 172.21.15.156:0/1308048968 learned_addr learned my addr 172.21.15.156:0/1308048968 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:58.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877d8d4700 1 -- 172.21.15.156:0/1308048968 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f877806f850 msgr2=0x7f8778156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877d8d4700 1 --2- 172.21.15.156:0/1308048968 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f877806f850 0x7f8778156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877d8d4700 1 -- 172.21.15.156:0/1308048968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87780cd8c0 msgr2=0x7f87780b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877d8d4700 1 --2- 172.21.15.156:0/1308048968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87780cd8c0 0x7f87780b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877d8d4700 1 -- 172.21.15.156:0/1308048968 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8770009a70 con 0x7f87780cdc90 2023-11-28T11:09:58.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f877d8d4700 1 --2- 172.21.15.156:0/1308048968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87780cdc90 0x7f87780b4940 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f87780b3aa0 tx=0x7f8770004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f876dffb700 1 -- 172.21.15.156:0/1308048968 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770003ea0 con 0x7f87780cdc90 2023-11-28T11:09:58.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.516+0000 7f876dffb700 1 -- 172.21.15.156:0/1308048968 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8770020070 con 0x7f87780cdc90 2023-11-28T11:09:58.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.517+0000 7f877f337700 1 -- 172.21.15.156:0/1308048968 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87780b35f0 con 0x7f87780cdc90 2023-11-28T11:09:58.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.517+0000 7f876dffb700 1 -- 172.21.15.156:0/1308048968 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770004280 con 0x7f87780cdc90 2023-11-28T11:09:58.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.517+0000 7f877f337700 1 -- 172.21.15.156:0/1308048968 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87780b3310 con 0x7f87780cdc90 2023-11-28T11:09:58.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.518+0000 7f877f337700 1 -- 172.21.15.156:0/1308048968 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87780c9260 con 0x7f87780cdc90 2023-11-28T11:09:58.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.518+0000 7f876dffb700 1 -- 172.21.15.156:0/1308048968 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f87700043e0 con 0x7f87780cdc90 2023-11-28T11:09:58.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.519+0000 7f876dffb700 1 --2- 172.21.15.156:0/1308048968 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f875c0682f0 0x7f875c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.519+0000 7f876dffb700 1 -- 172.21.15.156:0/1308048968 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8770022070 con 0x7f87780cdc90 2023-11-28T11:09:58.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.519+0000 7f877d0d3700 1 --2- 172.21.15.156:0/1308048968 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f875c0682f0 0x7f875c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.519+0000 7f877d0d3700 1 --2- 172.21.15.156:0/1308048968 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f875c0682f0 0x7f875c06a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8768006fd0 tx=0x7f8768008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.523+0000 7f876dffb700 1 -- 172.21.15.156:0/1308048968 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8770054ad0 con 0x7f87780cdc90 2023-11-28T11:09:58.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.701+0000 7f877f337700 1 -- 172.21.15.156:0/1308048968 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8778068690 con 0x7f875c0682f0 2023-11-28T11:09:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.709+0000 7f876dffb700 1 -- 172.21.15.156:0/1308048968 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311692 (secure 0 0 0) 0x7f8778068690 con 0x7f875c0682f0 2023-11-28T11:09:58.714 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.713+0000 7f877f337700 1 -- 172.21.15.156:0/1308048968 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f875c0682f0 msgr2=0x7f875c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.713+0000 7f877f337700 1 --2- 172.21.15.156:0/1308048968 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f875c0682f0 0x7f875c06a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8768006fd0 tx=0x7f8768008040 comp rx=0 tx=0).stop 2023-11-28T11:09:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.713+0000 7f877f337700 1 -- 172.21.15.156:0/1308048968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87780cdc90 msgr2=0x7f87780b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.713+0000 7f877f337700 1 --2- 172.21.15.156:0/1308048968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87780cdc90 0x7f87780b4940 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f87780b3aa0 tx=0x7f8770004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.713+0000 7f877f337700 1 -- 172.21.15.156:0/1308048968 shutdown_connections 2023-11-28T11:09:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.713+0000 7f877f337700 1 --2- 172.21.15.156:0/1308048968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87780cdc90 0x7f87780b4940 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.713+0000 7f877f337700 1 --2- 172.21.15.156:0/1308048968 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f875c0682f0 0x7f875c06a7a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.713+0000 7f877f337700 1 --2- 172.21.15.156:0/1308048968 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f877806f850 0x7f8778156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.713+0000 7f877f337700 1 --2- 172.21.15.156:0/1308048968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87780cd8c0 0x7f87780b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.713+0000 7f877f337700 1 -- 172.21.15.156:0/1308048968 >> 172.21.15.156:0/1308048968 conn(0x7f877806aed0 msgr2=0x7f87780cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:58.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.713+0000 7f877f337700 1 -- 172.21.15.156:0/1308048968 shutdown_connections 2023-11-28T11:09:58.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.713+0000 7f877f337700 1 -- 172.21.15.156:0/1308048968 wait complete. 2023-11-28T11:09:58.718 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:09:58.726 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-11-28T11:09:58.546600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":74268,"kb_used_data":5724,"kb_used_omap":0,"kb_used_meta":68480,"kb_avail":754900452,"statfs":{"total":773094113280,"available":773018062848,"internally_reserved":0,"allocated":5861376,"data_stored":3163855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70123520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.680334"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.690046+0000","last_change":"2023-11-28T11:09:54.289376+0000","last_active":"2023-11-28T11:09:55.690046+0000","last_peered":"2023-11-28T11:09:55.690046+0000","last_clean":"2023-11-28T11:09:55.690046+0000","last_became_active":"2023-11-28T11:09:54.289035+0000","last_became_peered":"2023-11-28T11:09:54.289035+0000","last_unstale":"2023-11-28T11:09:55.690046+0000","last_undegraded":"2023-11-28T11:09:55.690046+0000","last_fullsized":"2023-11-28T11:09:55.690046+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:38:19.762032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.866526+0000","last_change":"2023-11-28T11:09:54.295480+0000","last_active":"2023-11-28T11:09:55.866526+0000","last_peered":"2023-11-28T11:09:55.866526+0000","last_clean":"2023-11-28T11:09:55.866526+0000","last_became_active":"2023-11-28T11:09:54.295017+0000","last_became_peered":"2023-11-28T11:09:54.295017+0000","last_unstale":"2023-11-28T11:09:55.866526+0000","last_undegraded":"2023-11-28T11:09:55.866526+0000","last_fullsized":"2023-11-28T11:09:55.866526+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:34.535871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278973+0000","last_change":"2023-11-28T11:09:54.292339+0000","last_active":"2023-11-28T11:09:55.278973+0000","last_peered":"2023-11-28T11:09:55.278973+0000","last_clean":"2023-11-28T11:09:55.278973+0000","last_became_active":"2023-11-28T11:09:54.292122+0000","last_became_peered":"2023-11-28T11:09:54.292122+0000","last_unstale":"2023-11-28T11:09:55.278973+0000","last_undegraded":"2023-11-28T11:09:55.278973+0000","last_fullsized":"2023-11-28T11:09:55.278973+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:27.615241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865695+0000","last_change":"2023-11-28T11:09:54.299193+0000","last_active":"2023-11-28T11:09:55.865695+0000","last_peered":"2023-11-28T11:09:55.865695+0000","last_clean":"2023-11-28T11:09:55.865695+0000","last_became_active":"2023-11-28T11:09:54.298867+0000","last_became_peered":"2023-11-28T11:09:54.298867+0000","last_unstale":"2023-11-28T11:09:55.865695+0000","last_undegraded":"2023-11-28T11:09:55.865695+0000","last_fullsized":"2023-11-28T11:09:55.865695+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:34:25.541797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865971+0000","last_change":"2023-11-28T11:09:54.297341+0000","last_active":"2023-11-28T11:09:55.865971+0000","last_peered":"2023-11-28T11:09:55.865971+0000","last_clean":"2023-11-28T11:09:55.865971+0000","last_became_active":"2023-11-28T11:09:54.297062+0000","last_became_peered":"2023-11-28T11:09:54.297062+0000","last_unstale":"2023-11-28T11:09:55.865971+0000","last_undegraded":"2023-11-28T11:09:55.865971+0000","last_fullsized":"2023-11-28T11:09:55.865971+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:36:59.771116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.689846+0000","last_change":"2023-11-28T11:09:54.301440+0000","last_active":"2023-11-28T11:09:55.689846+0000","last_peered":"2023-11-28T11:09:55.689846+0000","last_clean":"2023-11-28T11:09:55.689846+0000","last_became_active":"2023-11-28T11:09:54.300129+0000","last_became_peered":"2023-11-28T11:09:54.300129+0000","last_unstale":"2023-11-28T11:09:55.689846+0000","last_undegraded":"2023-11-28T11:09:55.689846+0000","last_fullsized":"2023-11-28T11:09:55.689846+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:47:01.823875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278709+0000","last_change":"2023-11-28T11:09:54.294027+0000","last_active":"2023-11-28T11:09:55.278709+0000","last_peered":"2023-11-28T11:09:55.278709+0000","last_clean":"2023-11-28T11:09:55.278709+0000","last_became_active":"2023-11-28T11:09:54.293389+0000","last_became_peered":"2023-11-28T11:09:54.293389+0000","last_unstale":"2023-11-28T11:09:55.278709+0000","last_undegraded":"2023-11-28T11:09:55.278709+0000","last_fullsized":"2023-11-28T11:09:55.278709+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:34.271243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864548+0000","last_change":"2023-11-28T11:09:54.292815+0000","last_active":"2023-11-28T11:09:55.864548+0000","last_peered":"2023-11-28T11:09:55.864548+0000","last_clean":"2023-11-28T11:09:55.864548+0000","last_became_active":"2023-11-28T11:09:54.292575+0000","last_became_peered":"2023-11-28T11:09:54.292575+0000","last_unstale":"2023-11-28T11:09:55.864548+0000","last_undegraded":"2023-11-28T11:09:55.864548+0000","last_fullsized":"2023-11-28T11:09:55.864548+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:34.271243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865009+0000","last_change":"2023-11-28T11:09:54.294161+0000","last_active":"2023-11-28T11:09:55.865009+0000","last_peered":"2023-11-28T11:09:55.865009+0000","last_clean":"2023-11-28T11:09:55.865009+0000","last_became_active":"2023-11-28T11:09:54.293771+0000","last_became_peered":"2023-11-28T11:09:54.293771+0000","last_unstale":"2023-11-28T11:09:55.865009+0000","last_undegraded":"2023-11-28T11:09:55.865009+0000","last_fullsized":"2023-11-28T11:09:55.865009+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:53:35.361188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.690024+0000","last_change":"2023-11-28T11:09:54.298623+0000","last_active":"2023-11-28T11:09:55.690024+0000","last_peered":"2023-11-28T11:09:55.690024+0000","last_clean":"2023-11-28T11:09:55.690024+0000","last_became_active":"2023-11-28T11:09:54.298369+0000","last_became_peered":"2023-11-28T11:09:54.298369+0000","last_unstale":"2023-11-28T11:09:55.690024+0000","last_undegraded":"2023-11-28T11:09:55.690024+0000","last_fullsized":"2023-11-28T11:09:55.690024+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:04.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.282035+0000","last_change":"2023-11-28T11:09:54.298948+0000","last_active":"2023-11-28T11:09:55.282035+0000","last_peered":"2023-11-28T11:09:55.282035+0000","last_clean":"2023-11-28T11:09:55.282035+0000","last_became_active":"2023-11-28T11:09:54.298688+0000","last_became_peered":"2023-11-28T11:09:54.298688+0000","last_unstale":"2023-11-28T11:09:55.282035+0000","last_undegraded":"2023-11-28T11:09:55.282035+0000","last_fullsized":"2023-11-28T11:09:55.282035+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:43.941414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864870+0000","last_change":"2023-11-28T11:09:54.297459+0000","last_active":"2023-11-28T11:09:55.864870+0000","last_peered":"2023-11-28T11:09:55.864870+0000","last_clean":"2023-11-28T11:09:55.864870+0000","last_became_active":"2023-11-28T11:09:54.297178+0000","last_became_peered":"2023-11-28T11:09:54.297178+0000","last_unstale":"2023-11-28T11:09:55.864870+0000","last_undegraded":"2023-11-28T11:09:55.864870+0000","last_fullsized":"2023-11-28T11:09:55.864870+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:27.615241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.689901+0000","last_change":"2023-11-28T11:09:54.301769+0000","last_active":"2023-11-28T11:09:55.689901+0000","last_peered":"2023-11-28T11:09:55.689901+0000","last_clean":"2023-11-28T11:09:55.689901+0000","last_became_active":"2023-11-28T11:09:54.300101+0000","last_became_peered":"2023-11-28T11:09:54.300101+0000","last_unstale":"2023-11-28T11:09:55.689901+0000","last_undegraded":"2023-11-28T11:09:55.689901+0000","last_fullsized":"2023-11-28T11:09:55.689901+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:36:59.771116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279060+0000","last_change":"2023-11-28T11:09:54.294297+0000","last_active":"2023-11-28T11:09:55.279060+0000","last_peered":"2023-11-28T11:09:55.279060+0000","last_clean":"2023-11-28T11:09:55.279060+0000","last_became_active":"2023-11-28T11:09:54.294059+0000","last_became_peered":"2023-11-28T11:09:54.294059+0000","last_unstale":"2023-11-28T11:09:55.279060+0000","last_undegraded":"2023-11-28T11:09:55.279060+0000","last_fullsized":"2023-11-28T11:09:55.279060+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:33.227523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865700+0000","last_change":"2023-11-28T11:09:54.296229+0000","last_active":"2023-11-28T11:09:55.865700+0000","last_peered":"2023-11-28T11:09:55.865700+0000","last_clean":"2023-11-28T11:09:55.865700+0000","last_became_active":"2023-11-28T11:09:54.295939+0000","last_became_peered":"2023-11-28T11:09:54.295939+0000","last_unstale":"2023-11-28T11:09:55.865700+0000","last_undegraded":"2023-11-28T11:09:55.865700+0000","last_fullsized":"2023-11-28T11:09:55.865700+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:50:40.722651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.689800+0000","last_change":"2023-11-28T11:09:54.299095+0000","last_active":"2023-11-28T11:09:55.689800+0000","last_peered":"2023-11-28T11:09:55.689800+0000","last_clean":"2023-11-28T11:09:55.689800+0000","last_became_active":"2023-11-28T11:09:54.298857+0000","last_became_peered":"2023-11-28T11:09:54.298857+0000","last_unstale":"2023-11-28T11:09:55.689800+0000","last_undegraded":"2023-11-28T11:09:55.689800+0000","last_fullsized":"2023-11-28T11:09:55.689800+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:57.049159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279314+0000","last_change":"2023-11-28T11:09:54.299470+0000","last_active":"2023-11-28T11:09:55.279314+0000","last_peered":"2023-11-28T11:09:55.279314+0000","last_clean":"2023-11-28T11:09:55.279314+0000","last_became_active":"2023-11-28T11:09:54.299205+0000","last_became_peered":"2023-11-28T11:09:54.299205+0000","last_unstale":"2023-11-28T11:09:55.279314+0000","last_undegraded":"2023-11-28T11:09:55.279314+0000","last_fullsized":"2023-11-28T11:09:55.279314+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:35.037648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.866610+0000","last_change":"2023-11-28T11:09:54.295654+0000","last_active":"2023-11-28T11:09:55.866610+0000","last_peered":"2023-11-28T11:09:55.866610+0000","last_clean":"2023-11-28T11:09:55.866610+0000","last_became_active":"2023-11-28T11:09:54.295344+0000","last_became_peered":"2023-11-28T11:09:54.295344+0000","last_unstale":"2023-11-28T11:09:55.866610+0000","last_undegraded":"2023-11-28T11:09:55.866610+0000","last_fullsized":"2023-11-28T11:09:55.866610+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:37.779494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279060+0000","last_change":"2023-11-28T11:09:54.301917+0000","last_active":"2023-11-28T11:09:55.279060+0000","last_peered":"2023-11-28T11:09:55.279060+0000","last_clean":"2023-11-28T11:09:55.279060+0000","last_became_active":"2023-11-28T11:09:54.301657+0000","last_became_peered":"2023-11-28T11:09:54.301657+0000","last_unstale":"2023-11-28T11:09:55.279060+0000","last_undegraded":"2023-11-28T11:09:55.279060+0000","last_fullsized":"2023-11-28T11:09:55.279060+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:56.200657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864719+0000","last_change":"2023-11-28T11:09:54.301507+0000","last_active":"2023-11-28T11:09:55.864719+0000","last_peered":"2023-11-28T11:09:55.864719+0000","last_clean":"2023-11-28T11:09:55.864719+0000","last_became_active":"2023-11-28T11:09:54.301165+0000","last_became_peered":"2023-11-28T11:09:54.301165+0000","last_unstale":"2023-11-28T11:09:55.864719+0000","last_undegraded":"2023-11-28T11:09:55.864719+0000","last_fullsized":"2023-11-28T11:09:55.864719+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:00.471237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864998+0000","last_change":"2023-11-28T11:09:54.300705+0000","last_active":"2023-11-28T11:09:55.864998+0000","last_peered":"2023-11-28T11:09:55.864998+0000","last_clean":"2023-11-28T11:09:55.864998+0000","last_became_active":"2023-11-28T11:09:54.298027+0000","last_became_peered":"2023-11-28T11:09:54.298027+0000","last_unstale":"2023-11-28T11:09:55.864998+0000","last_undegraded":"2023-11-28T11:09:55.864998+0000","last_fullsized":"2023-11-28T11:09:55.864998+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:25:15.795367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864664+0000","last_change":"2023-11-28T11:09:54.295880+0000","last_active":"2023-11-28T11:09:55.864664+0000","last_peered":"2023-11-28T11:09:55.864664+0000","last_clean":"2023-11-28T11:09:55.864664+0000","last_became_active":"2023-11-28T11:09:54.295677+0000","last_became_peered":"2023-11-28T11:09:54.295677+0000","last_unstale":"2023-11-28T11:09:55.864664+0000","last_undegraded":"2023-11-28T11:09:55.864664+0000","last_fullsized":"2023-11-28T11:09:55.864664+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:56.200657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.281834+0000","last_change":"2023-11-28T11:09:54.298930+0000","last_active":"2023-11-28T11:09:55.281834+0000","last_peered":"2023-11-28T11:09:55.281834+0000","last_clean":"2023-11-28T11:09:55.281834+0000","last_became_active":"2023-11-28T11:09:54.298659+0000","last_became_peered":"2023-11-28T11:09:54.298659+0000","last_unstale":"2023-11-28T11:09:55.281834+0000","last_undegraded":"2023-11-28T11:09:55.281834+0000","last_fullsized":"2023-11-28T11:09:55.281834+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:35.038084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278660+0000","last_change":"2023-11-28T11:09:54.294262+0000","last_active":"2023-11-28T11:09:55.278660+0000","last_peered":"2023-11-28T11:09:55.278660+0000","last_clean":"2023-11-28T11:09:55.278660+0000","last_became_active":"2023-11-28T11:09:54.293416+0000","last_became_peered":"2023-11-28T11:09:54.293416+0000","last_unstale":"2023-11-28T11:09:55.278660+0000","last_undegraded":"2023-11-28T11:09:55.278660+0000","last_fullsized":"2023-11-28T11:09:55.278660+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:00.934162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279330+0000","last_change":"2023-11-28T11:09:54.295861+0000","last_active":"2023-11-28T11:09:55.279330+0000","last_peered":"2023-11-28T11:09:55.279330+0000","last_clean":"2023-11-28T11:09:55.279330+0000","last_became_active":"2023-11-28T11:09:54.295542+0000","last_became_peered":"2023-11-28T11:09:54.295542+0000","last_unstale":"2023-11-28T11:09:55.279330+0000","last_undegraded":"2023-11-28T11:09:55.279330+0000","last_fullsized":"2023-11-28T11:09:55.279330+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:51:33.313288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864877+0000","last_change":"2023-11-28T11:09:54.287429+0000","last_active":"2023-11-28T11:09:55.864877+0000","last_peered":"2023-11-28T11:09:55.864877+0000","last_clean":"2023-11-28T11:09:55.864877+0000","last_became_active":"2023-11-28T11:09:54.287039+0000","last_became_peered":"2023-11-28T11:09:54.287039+0000","last_unstale":"2023-11-28T11:09:55.864877+0000","last_undegraded":"2023-11-28T11:09:55.864877+0000","last_fullsized":"2023-11-28T11:09:55.864877+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:05:28.432364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.690158+0000","last_change":"2023-11-28T11:09:54.302549+0000","last_active":"2023-11-28T11:09:55.690158+0000","last_peered":"2023-11-28T11:09:55.690158+0000","last_clean":"2023-11-28T11:09:55.690158+0000","last_became_active":"2023-11-28T11:09:54.302305+0000","last_became_peered":"2023-11-28T11:09:54.302305+0000","last_unstale":"2023-11-28T11:09:55.690158+0000","last_undegraded":"2023-11-28T11:09:55.690158+0000","last_fullsized":"2023-11-28T11:09:55.690158+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:37.779494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864957+0000","last_change":"2023-11-28T11:09:54.293922+0000","last_active":"2023-11-28T11:09:55.864957+0000","last_peered":"2023-11-28T11:09:55.864957+0000","last_clean":"2023-11-28T11:09:55.864957+0000","last_became_active":"2023-11-28T11:09:54.293650+0000","last_became_peered":"2023-11-28T11:09:54.293650+0000","last_unstale":"2023-11-28T11:09:55.864957+0000","last_undegraded":"2023-11-28T11:09:55.864957+0000","last_fullsized":"2023-11-28T11:09:55.864957+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:17:59.087466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278747+0000","last_change":"2023-11-28T11:09:54.295532+0000","last_active":"2023-11-28T11:09:55.278747+0000","last_peered":"2023-11-28T11:09:55.278747+0000","last_clean":"2023-11-28T11:09:55.278747+0000","last_became_active":"2023-11-28T11:09:54.295277+0000","last_became_peered":"2023-11-28T11:09:54.295277+0000","last_unstale":"2023-11-28T11:09:55.278747+0000","last_undegraded":"2023-11-28T11:09:55.278747+0000","last_fullsized":"2023-11-28T11:09:55.278747+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:05:28.432364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865527+0000","last_change":"2023-11-28T11:09:54.304324+0000","last_active":"2023-11-28T11:09:55.865527+0000","last_peered":"2023-11-28T11:09:55.865527+0000","last_clean":"2023-11-28T11:09:55.865527+0000","last_became_active":"2023-11-28T11:09:54.303964+0000","last_became_peered":"2023-11-28T11:09:54.303964+0000","last_unstale":"2023-11-28T11:09:55.865527+0000","last_undegraded":"2023-11-28T11:09:55.865527+0000","last_fullsized":"2023-11-28T11:09:55.865527+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:36:59.771116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278808+0000","last_change":"2023-11-28T11:09:54.294192+0000","last_active":"2023-11-28T11:09:55.278808+0000","last_peered":"2023-11-28T11:09:55.278808+0000","last_clean":"2023-11-28T11:09:55.278808+0000","last_became_active":"2023-11-28T11:09:54.293689+0000","last_became_peered":"2023-11-28T11:09:54.293689+0000","last_unstale":"2023-11-28T11:09:55.278808+0000","last_undegraded":"2023-11-28T11:09:55.278808+0000","last_fullsized":"2023-11-28T11:09:55.278808+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:24:24.473745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.282032+0000","last_change":"2023-11-28T11:09:54.290664+0000","last_active":"2023-11-28T11:09:55.282032+0000","last_peered":"2023-11-28T11:09:55.282032+0000","last_clean":"2023-11-28T11:09:55.282032+0000","last_became_active":"2023-11-28T11:09:54.290300+0000","last_became_peered":"2023-11-28T11:09:54.290300+0000","last_unstale":"2023-11-28T11:09:55.282032+0000","last_undegraded":"2023-11-28T11:09:55.282032+0000","last_fullsized":"2023-11-28T11:09:55.282032+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:37.779494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864597+0000","last_change":"2023-11-28T11:09:54.303013+0000","last_active":"2023-11-28T11:09:55.864597+0000","last_peered":"2023-11-28T11:09:55.864597+0000","last_clean":"2023-11-28T11:09:55.864597+0000","last_became_active":"2023-11-28T11:09:54.299985+0000","last_became_peered":"2023-11-28T11:09:54.299985+0000","last_unstale":"2023-11-28T11:09:55.864597+0000","last_undegraded":"2023-11-28T11:09:55.864597+0000","last_fullsized":"2023-11-28T11:09:55.864597+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:10.092726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864836+0000","last_change":"2023-11-28T11:09:53.883655+0000","last_active":"2023-11-28T11:09:55.864836+0000","last_peered":"2023-11-28T11:09:55.864836+0000","last_clean":"2023-11-28T11:09:55.864836+0000","last_became_active":"2023-11-28T11:09:53.882889+0000","last_became_peered":"2023-11-28T11:09:53.882889+0000","last_unstale":"2023-11-28T11:09:55.864836+0000","last_undegraded":"2023-11-28T11:09:55.864836+0000","last_fullsized":"2023-11-28T11:09:55.864836+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:34.033772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.282022+0000","last_change":"2023-11-28T11:09:54.297642+0000","last_active":"2023-11-28T11:09:55.282022+0000","last_peered":"2023-11-28T11:09:55.282022+0000","last_clean":"2023-11-28T11:09:55.282022+0000","last_became_active":"2023-11-28T11:09:54.296996+0000","last_became_peered":"2023-11-28T11:09:54.296996+0000","last_unstale":"2023-11-28T11:09:55.282022+0000","last_undegraded":"2023-11-28T11:09:55.282022+0000","last_fullsized":"2023-11-28T11:09:55.282022+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:34.535871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.690043+0000","last_change":"2023-11-28T11:09:53.886383+0000","last_active":"2023-11-28T11:09:55.690043+0000","last_peered":"2023-11-28T11:09:55.690043+0000","last_clean":"2023-11-28T11:09:55.690043+0000","last_became_active":"2023-11-28T11:09:53.886139+0000","last_became_peered":"2023-11-28T11:09:53.886139+0000","last_unstale":"2023-11-28T11:09:55.690043+0000","last_undegraded":"2023-11-28T11:09:55.690043+0000","last_fullsized":"2023-11-28T11:09:55.690043+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:53:10.946985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865020+0000","last_change":"2023-11-28T11:09:54.298492+0000","last_active":"2023-11-28T11:09:55.865020+0000","last_peered":"2023-11-28T11:09:55.865020+0000","last_clean":"2023-11-28T11:09:55.865020+0000","last_became_active":"2023-11-28T11:09:54.298145+0000","last_became_peered":"2023-11-28T11:09:54.298145+0000","last_unstale":"2023-11-28T11:09:55.865020+0000","last_undegraded":"2023-11-28T11:09:55.865020+0000","last_fullsized":"2023-11-28T11:09:55.865020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:20:10.137372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.690191+0000","last_change":"2023-11-28T11:09:53.890336+0000","last_active":"2023-11-28T11:09:55.690191+0000","last_peered":"2023-11-28T11:09:55.690191+0000","last_clean":"2023-11-28T11:09:55.690191+0000","last_became_active":"2023-11-28T11:09:53.890018+0000","last_became_peered":"2023-11-28T11:09:53.890018+0000","last_unstale":"2023-11-28T11:09:55.690191+0000","last_undegraded":"2023-11-28T11:09:55.690191+0000","last_fullsized":"2023-11-28T11:09:55.690191+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:46.046727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864741+0000","last_change":"2023-11-28T11:09:54.286450+0000","last_active":"2023-11-28T11:09:55.864741+0000","last_peered":"2023-11-28T11:09:55.864741+0000","last_clean":"2023-11-28T11:09:55.864741+0000","last_became_active":"2023-11-28T11:09:54.286188+0000","last_became_peered":"2023-11-28T11:09:54.286188+0000","last_unstale":"2023-11-28T11:09:55.864741+0000","last_undegraded":"2023-11-28T11:09:55.864741+0000","last_fullsized":"2023-11-28T11:09:55.864741+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:43:19.089769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.282470+0000","last_change":"2023-11-28T11:09:53.874138+0000","last_active":"2023-11-28T11:09:55.282470+0000","last_peered":"2023-11-28T11:09:55.282470+0000","last_clean":"2023-11-28T11:09:55.282470+0000","last_became_active":"2023-11-28T11:09:53.873744+0000","last_became_peered":"2023-11-28T11:09:53.873744+0000","last_unstale":"2023-11-28T11:09:55.282470+0000","last_undegraded":"2023-11-28T11:09:55.282470+0000","last_fullsized":"2023-11-28T11:09:55.282470+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:25:14.791056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.866336+0000","last_change":"2023-11-28T11:09:54.305862+0000","last_active":"2023-11-28T11:09:55.866336+0000","last_peered":"2023-11-28T11:09:55.866336+0000","last_clean":"2023-11-28T11:09:55.866336+0000","last_became_active":"2023-11-28T11:09:54.305588+0000","last_became_peered":"2023-11-28T11:09:54.305588+0000","last_unstale":"2023-11-28T11:09:55.866336+0000","last_undegraded":"2023-11-28T11:09:55.866336+0000","last_fullsized":"2023-11-28T11:09:55.866336+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:49.846597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278828+0000","last_change":"2023-11-28T11:09:53.893375+0000","last_active":"2023-11-28T11:09:55.278828+0000","last_peered":"2023-11-28T11:09:55.278828+0000","last_clean":"2023-11-28T11:09:55.278828+0000","last_became_active":"2023-11-28T11:09:53.893090+0000","last_became_peered":"2023-11-28T11:09:53.893090+0000","last_unstale":"2023-11-28T11:09:55.278828+0000","last_undegraded":"2023-11-28T11:09:55.278828+0000","last_fullsized":"2023-11-28T11:09:55.278828+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:51.766776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.866006+0000","last_change":"2023-11-28T11:09:54.304611+0000","last_active":"2023-11-28T11:09:55.866006+0000","last_peered":"2023-11-28T11:09:55.866006+0000","last_clean":"2023-11-28T11:09:55.866006+0000","last_became_active":"2023-11-28T11:09:54.304307+0000","last_became_peered":"2023-11-28T11:09:54.304307+0000","last_unstale":"2023-11-28T11:09:55.866006+0000","last_undegraded":"2023-11-28T11:09:55.866006+0000","last_fullsized":"2023-11-28T11:09:55.866006+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:38:19.762032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.866075+0000","last_change":"2023-11-28T11:09:53.883510+0000","last_active":"2023-11-28T11:09:55.866075+0000","last_peered":"2023-11-28T11:09:55.866075+0000","last_clean":"2023-11-28T11:09:55.866075+0000","last_became_active":"2023-11-28T11:09:53.883230+0000","last_became_peered":"2023-11-28T11:09:53.883230+0000","last_unstale":"2023-11-28T11:09:55.866075+0000","last_undegraded":"2023-11-28T11:09:55.866075+0000","last_fullsized":"2023-11-28T11:09:55.866075+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:38.043246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864977+0000","last_change":"2023-11-28T11:09:54.287276+0000","last_active":"2023-11-28T11:09:55.864977+0000","last_peered":"2023-11-28T11:09:55.864977+0000","last_clean":"2023-11-28T11:09:55.864977+0000","last_became_active":"2023-11-28T11:09:54.286712+0000","last_became_peered":"2023-11-28T11:09:54.286712+0000","last_unstale":"2023-11-28T11:09:55.864977+0000","last_undegraded":"2023-11-28T11:09:55.864977+0000","last_fullsized":"2023-11-28T11:09:55.864977+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:36:35.247278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865066+0000","last_change":"2023-11-28T11:09:53.883108+0000","last_active":"2023-11-28T11:09:55.865066+0000","last_peered":"2023-11-28T11:09:55.865066+0000","last_clean":"2023-11-28T11:09:55.865066+0000","last_became_active":"2023-11-28T11:09:53.882714+0000","last_became_peered":"2023-11-28T11:09:53.882714+0000","last_unstale":"2023-11-28T11:09:55.865066+0000","last_undegraded":"2023-11-28T11:09:55.865066+0000","last_fullsized":"2023-11-28T11:09:55.865066+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:10.799818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.866590+0000","last_change":"2023-11-28T11:09:54.302197+0000","last_active":"2023-11-28T11:09:55.866590+0000","last_peered":"2023-11-28T11:09:55.866590+0000","last_clean":"2023-11-28T11:09:55.866590+0000","last_became_active":"2023-11-28T11:09:54.301906+0000","last_became_peered":"2023-11-28T11:09:54.301906+0000","last_unstale":"2023-11-28T11:09:55.866590+0000","last_undegraded":"2023-11-28T11:09:55.866590+0000","last_fullsized":"2023-11-28T11:09:55.866590+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:47:01.823875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279126+0000","last_change":"2023-11-28T11:09:53.882587+0000","last_active":"2023-11-28T11:09:55.279126+0000","last_peered":"2023-11-28T11:09:55.279126+0000","last_clean":"2023-11-28T11:09:55.279126+0000","last_became_active":"2023-11-28T11:09:53.881772+0000","last_became_peered":"2023-11-28T11:09:53.881772+0000","last_unstale":"2023-11-28T11:09:55.279126+0000","last_undegraded":"2023-11-28T11:09:55.279126+0000","last_fullsized":"2023-11-28T11:09:55.279126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:17:58.083155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865725+0000","last_change":"2023-11-28T11:09:54.303544+0000","last_active":"2023-11-28T11:09:55.865725+0000","last_peered":"2023-11-28T11:09:55.865725+0000","last_clean":"2023-11-28T11:09:55.865725+0000","last_became_active":"2023-11-28T11:09:54.302571+0000","last_became_peered":"2023-11-28T11:09:54.302571+0000","last_unstale":"2023-11-28T11:09:55.865725+0000","last_undegraded":"2023-11-28T11:09:55.865725+0000","last_fullsized":"2023-11-28T11:09:55.865725+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:24:24.473745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.866575+0000","last_change":"2023-11-28T11:09:53.881139+0000","last_active":"2023-11-28T11:09:55.866575+0000","last_peered":"2023-11-28T11:09:55.866575+0000","last_clean":"2023-11-28T11:09:55.866575+0000","last_became_active":"2023-11-28T11:09:53.880811+0000","last_became_peered":"2023-11-28T11:09:53.880811+0000","last_unstale":"2023-11-28T11:09:55.866575+0000","last_undegraded":"2023-11-28T11:09:55.866575+0000","last_fullsized":"2023-11-28T11:09:55.866575+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:20:09.133061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.281967+0000","last_change":"2023-11-28T11:09:54.297839+0000","last_active":"2023-11-28T11:09:55.281967+0000","last_peered":"2023-11-28T11:09:55.281967+0000","last_clean":"2023-11-28T11:09:55.281967+0000","last_became_active":"2023-11-28T11:09:54.297368+0000","last_became_peered":"2023-11-28T11:09:54.297368+0000","last_unstale":"2023-11-28T11:09:55.281967+0000","last_undegraded":"2023-11-28T11:09:55.281967+0000","last_fullsized":"2023-11-28T11:09:55.281967+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:50:40.722651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.866520+0000","last_change":"2023-11-28T11:09:53.893618+0000","last_active":"2023-11-28T11:09:55.866520+0000","last_peered":"2023-11-28T11:09:55.866520+0000","last_clean":"2023-11-28T11:09:55.866520+0000","last_became_active":"2023-11-28T11:09:53.893350+0000","last_became_peered":"2023-11-28T11:09:53.893350+0000","last_unstale":"2023-11-28T11:09:55.866520+0000","last_undegraded":"2023-11-28T11:09:55.866520+0000","last_fullsized":"2023-11-28T11:09:55.866520+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:56.044847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279208+0000","last_change":"2023-11-28T11:09:54.297393+0000","last_active":"2023-11-28T11:09:55.279208+0000","last_peered":"2023-11-28T11:09:55.279208+0000","last_clean":"2023-11-28T11:09:55.279208+0000","last_became_active":"2023-11-28T11:09:54.296960+0000","last_became_peered":"2023-11-28T11:09:54.296960+0000","last_unstale":"2023-11-28T11:09:55.279208+0000","last_undegraded":"2023-11-28T11:09:55.279208+0000","last_fullsized":"2023-11-28T11:09:55.279208+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:03:51.215320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865898+0000","last_change":"2023-11-28T11:09:53.891065+0000","last_active":"2023-11-28T11:09:55.865898+0000","last_peered":"2023-11-28T11:09:55.865898+0000","last_clean":"2023-11-28T11:09:55.865898+0000","last_became_active":"2023-11-28T11:09:53.890720+0000","last_became_peered":"2023-11-28T11:09:53.890720+0000","last_unstale":"2023-11-28T11:09:55.865898+0000","last_undegraded":"2023-11-28T11:09:55.865898+0000","last_fullsized":"2023-11-28T11:09:55.865898+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:59.466925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.866029+0000","last_change":"2023-11-28T11:09:54.295150+0000","last_active":"2023-11-28T11:09:55.866029+0000","last_peered":"2023-11-28T11:09:55.866029+0000","last_clean":"2023-11-28T11:09:55.866029+0000","last_became_active":"2023-11-28T11:09:54.294866+0000","last_became_peered":"2023-11-28T11:09:54.294866+0000","last_unstale":"2023-11-28T11:09:55.866029+0000","last_undegraded":"2023-11-28T11:09:55.866029+0000","last_fullsized":"2023-11-28T11:09:55.866029+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:00.934162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864964+0000","last_change":"2023-11-28T11:09:53.887028+0000","last_active":"2023-11-28T11:09:55.864964+0000","last_peered":"2023-11-28T11:09:55.864964+0000","last_clean":"2023-11-28T11:09:55.864964+0000","last_became_active":"2023-11-28T11:09:53.886756+0000","last_became_peered":"2023-11-28T11:09:53.886756+0000","last_unstale":"2023-11-28T11:09:55.864964+0000","last_undegraded":"2023-11-28T11:09:55.864964+0000","last_fullsized":"2023-11-28T11:09:55.864964+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:38.043246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.282372+0000","last_change":"2023-11-28T11:09:54.296937+0000","last_active":"2023-11-28T11:09:55.282372+0000","last_peered":"2023-11-28T11:09:55.282372+0000","last_clean":"2023-11-28T11:09:55.282372+0000","last_became_active":"2023-11-28T11:09:54.296645+0000","last_became_peered":"2023-11-28T11:09:54.296645+0000","last_unstale":"2023-11-28T11:09:55.282372+0000","last_undegraded":"2023-11-28T11:09:55.282372+0000","last_fullsized":"2023-11-28T11:09:55.282372+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:34:25.541797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278914+0000","last_change":"2023-11-28T11:09:53.892691+0000","last_active":"2023-11-28T11:09:55.278914+0000","last_peered":"2023-11-28T11:09:55.278914+0000","last_clean":"2023-11-28T11:09:55.278914+0000","last_became_active":"2023-11-28T11:09:53.892409+0000","last_became_peered":"2023-11-28T11:09:53.892409+0000","last_unstale":"2023-11-28T11:09:55.278914+0000","last_undegraded":"2023-11-28T11:09:55.278914+0000","last_fullsized":"2023-11-28T11:09:55.278914+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:03.817799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279159+0000","last_change":"2023-11-28T11:09:54.290831+0000","last_active":"2023-11-28T11:09:55.279159+0000","last_peered":"2023-11-28T11:09:55.279159+0000","last_clean":"2023-11-28T11:09:55.279159+0000","last_became_active":"2023-11-28T11:09:54.290270+0000","last_became_peered":"2023-11-28T11:09:54.290270+0000","last_unstale":"2023-11-28T11:09:55.279159+0000","last_undegraded":"2023-11-28T11:09:55.279159+0000","last_fullsized":"2023-11-28T11:09:55.279159+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:08:40.328821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865113+0000","last_change":"2023-11-28T11:09:53.893486+0000","last_active":"2023-11-28T11:09:55.865113+0000","last_peered":"2023-11-28T11:09:55.865113+0000","last_clean":"2023-11-28T11:09:55.865113+0000","last_became_active":"2023-11-28T11:09:53.893176+0000","last_became_peered":"2023-11-28T11:09:53.893176+0000","last_unstale":"2023-11-28T11:09:55.865113+0000","last_undegraded":"2023-11-28T11:09:55.865113+0000","last_fullsized":"2023-11-28T11:09:55.865113+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:40.969610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864903+0000","last_change":"2023-11-28T11:09:54.296132+0000","last_active":"2023-11-28T11:09:55.864903+0000","last_peered":"2023-11-28T11:09:55.864903+0000","last_clean":"2023-11-28T11:09:55.864903+0000","last_became_active":"2023-11-28T11:09:54.295853+0000","last_became_peered":"2023-11-28T11:09:54.295853+0000","last_unstale":"2023-11-28T11:09:55.864903+0000","last_undegraded":"2023-11-28T11:09:55.864903+0000","last_fullsized":"2023-11-28T11:09:55.864903+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:35.037648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865627+0000","last_change":"2023-11-28T11:09:53.881556+0000","last_active":"2023-11-28T11:09:55.865627+0000","last_peered":"2023-11-28T11:09:55.865627+0000","last_clean":"2023-11-28T11:09:55.865627+0000","last_became_active":"2023-11-28T11:09:53.881235+0000","last_became_peered":"2023-11-28T11:09:53.881235+0000","last_unstale":"2023-11-28T11:09:55.865627+0000","last_undegraded":"2023-11-28T11:09:55.865627+0000","last_fullsized":"2023-11-28T11:09:55.865627+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:46.046727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279265+0000","last_change":"2023-11-28T11:09:54.291597+0000","last_active":"2023-11-28T11:09:55.279265+0000","last_peered":"2023-11-28T11:09:55.279265+0000","last_clean":"2023-11-28T11:09:55.279265+0000","last_became_active":"2023-11-28T11:09:54.291136+0000","last_became_peered":"2023-11-28T11:09:54.291136+0000","last_unstale":"2023-11-28T11:09:55.279265+0000","last_undegraded":"2023-11-28T11:09:55.279265+0000","last_fullsized":"2023-11-28T11:09:55.279265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:53:35.361188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":19,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864751+0000","last_change":"2023-11-28T11:09:53.886321+0000","last_active":"2023-11-28T11:09:55.864751+0000","last_peered":"2023-11-28T11:09:55.864751+0000","last_clean":"2023-11-28T11:09:55.864751+0000","last_became_active":"2023-11-28T11:09:53.886065+0000","last_became_peered":"2023-11-28T11:09:53.886065+0000","last_unstale":"2023-11-28T11:09:55.864751+0000","last_undegraded":"2023-11-28T11:09:55.864751+0000","last_fullsized":"2023-11-28T11:09:55.864751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:59.962919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864604+0000","last_change":"2023-11-28T11:09:54.293936+0000","last_active":"2023-11-28T11:09:55.864604+0000","last_peered":"2023-11-28T11:09:55.864604+0000","last_clean":"2023-11-28T11:09:55.864604+0000","last_became_active":"2023-11-28T11:09:54.293615+0000","last_became_peered":"2023-11-28T11:09:54.293615+0000","last_unstale":"2023-11-28T11:09:55.864604+0000","last_undegraded":"2023-11-28T11:09:55.864604+0000","last_fullsized":"2023-11-28T11:09:55.864604+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:38:19.762032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279214+0000","last_change":"2023-11-28T11:09:53.892731+0000","last_active":"2023-11-28T11:09:55.279214+0000","last_peered":"2023-11-28T11:09:55.279214+0000","last_clean":"2023-11-28T11:09:55.279214+0000","last_became_active":"2023-11-28T11:09:53.892494+0000","last_became_peered":"2023-11-28T11:09:53.892494+0000","last_unstale":"2023-11-28T11:09:55.279214+0000","last_undegraded":"2023-11-28T11:09:55.279214+0000","last_fullsized":"2023-11-28T11:09:55.279214+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:45.115674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.689815+0000","last_change":"2023-11-28T11:09:54.302213+0000","last_active":"2023-11-28T11:09:55.689815+0000","last_peered":"2023-11-28T11:09:55.689815+0000","last_clean":"2023-11-28T11:09:55.689815+0000","last_became_active":"2023-11-28T11:09:54.301957+0000","last_became_peered":"2023-11-28T11:09:54.301957+0000","last_unstale":"2023-11-28T11:09:55.689815+0000","last_undegraded":"2023-11-28T11:09:55.689815+0000","last_fullsized":"2023-11-28T11:09:55.689815+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:10.092726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.282025+0000","last_change":"2023-11-28T11:09:53.886204+0000","last_active":"2023-11-28T11:09:55.282025+0000","last_peered":"2023-11-28T11:09:55.282025+0000","last_clean":"2023-11-28T11:09:55.282025+0000","last_became_active":"2023-11-28T11:09:53.885935+0000","last_became_peered":"2023-11-28T11:09:53.885935+0000","last_unstale":"2023-11-28T11:09:55.282025+0000","last_undegraded":"2023-11-28T11:09:55.282025+0000","last_fullsized":"2023-11-28T11:09:55.282025+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:53:16.906042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865973+0000","last_change":"2023-11-28T11:09:54.303442+0000","last_active":"2023-11-28T11:09:55.865973+0000","last_peered":"2023-11-28T11:09:55.865973+0000","last_clean":"2023-11-28T11:09:55.865973+0000","last_became_active":"2023-11-28T11:09:54.303155+0000","last_became_peered":"2023-11-28T11:09:54.303155+0000","last_unstale":"2023-11-28T11:09:55.865973+0000","last_undegraded":"2023-11-28T11:09:55.865973+0000","last_fullsized":"2023-11-28T11:09:55.865973+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:08:13.649766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864863+0000","last_change":"2023-11-28T11:09:53.890889+0000","last_active":"2023-11-28T11:09:55.864863+0000","last_peered":"2023-11-28T11:09:55.864863+0000","last_clean":"2023-11-28T11:09:55.864863+0000","last_became_active":"2023-11-28T11:09:53.890632+0000","last_became_peered":"2023-11-28T11:09:53.890632+0000","last_unstale":"2023-11-28T11:09:55.864863+0000","last_undegraded":"2023-11-28T11:09:55.864863+0000","last_fullsized":"2023-11-28T11:09:55.864863+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:36:43.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.690130+0000","last_change":"2023-11-28T11:09:54.289877+0000","last_active":"2023-11-28T11:09:55.690130+0000","last_peered":"2023-11-28T11:09:55.690130+0000","last_clean":"2023-11-28T11:09:55.690130+0000","last_became_active":"2023-11-28T11:09:54.289501+0000","last_became_peered":"2023-11-28T11:09:54.289501+0000","last_unstale":"2023-11-28T11:09:55.690130+0000","last_undegraded":"2023-11-28T11:09:55.690130+0000","last_fullsized":"2023-11-28T11:09:55.690130+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:34:25.541797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864825+0000","last_change":"2023-11-28T11:09:53.889768+0000","last_active":"2023-11-28T11:09:55.864825+0000","last_peered":"2023-11-28T11:09:55.864825+0000","last_clean":"2023-11-28T11:09:55.864825+0000","last_became_active":"2023-11-28T11:09:53.889515+0000","last_became_peered":"2023-11-28T11:09:53.889515+0000","last_unstale":"2023-11-28T11:09:55.864825+0000","last_undegraded":"2023-11-28T11:09:55.864825+0000","last_fullsized":"2023-11-28T11:09:55.864825+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:19:23.491566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.282567+0000","last_change":"2023-11-28T11:09:54.299690+0000","last_active":"2023-11-28T11:09:55.282567+0000","last_peered":"2023-11-28T11:09:55.282567+0000","last_clean":"2023-11-28T11:09:55.282567+0000","last_became_active":"2023-11-28T11:09:54.299432+0000","last_became_peered":"2023-11-28T11:09:54.299432+0000","last_unstale":"2023-11-28T11:09:55.282567+0000","last_undegraded":"2023-11-28T11:09:55.282567+0000","last_fullsized":"2023-11-28T11:09:55.282567+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:36:59.771116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.689923+0000","last_change":"2023-11-28T11:09:53.890493+0000","last_active":"2023-11-28T11:09:55.689923+0000","last_peered":"2023-11-28T11:09:55.689923+0000","last_clean":"2023-11-28T11:09:55.689923+0000","last_became_active":"2023-11-28T11:09:53.890179+0000","last_became_peered":"2023-11-28T11:09:53.890179+0000","last_unstale":"2023-11-28T11:09:55.689923+0000","last_undegraded":"2023-11-28T11:09:55.689923+0000","last_fullsized":"2023-11-28T11:09:55.689923+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.257487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864810+0000","last_change":"2023-11-28T11:09:54.300768+0000","last_active":"2023-11-28T11:09:55.864810+0000","last_peered":"2023-11-28T11:09:55.864810+0000","last_clean":"2023-11-28T11:09:55.864810+0000","last_became_active":"2023-11-28T11:09:54.298417+0000","last_became_peered":"2023-11-28T11:09:54.298417+0000","last_unstale":"2023-11-28T11:09:55.864810+0000","last_undegraded":"2023-11-28T11:09:55.864810+0000","last_fullsized":"2023-11-28T11:09:55.864810+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:46.119986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864633+0000","last_change":"2023-11-28T11:09:53.887902+0000","last_active":"2023-11-28T11:09:55.864633+0000","last_peered":"2023-11-28T11:09:55.864633+0000","last_clean":"2023-11-28T11:09:55.864633+0000","last_became_active":"2023-11-28T11:09:53.887657+0000","last_became_peered":"2023-11-28T11:09:53.887657+0000","last_unstale":"2023-11-28T11:09:55.864633+0000","last_undegraded":"2023-11-28T11:09:55.864633+0000","last_fullsized":"2023-11-28T11:09:55.864633+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:09.088415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279058+0000","last_change":"2023-11-28T11:09:54.296684+0000","last_active":"2023-11-28T11:09:55.279058+0000","last_peered":"2023-11-28T11:09:55.279058+0000","last_clean":"2023-11-28T11:09:55.279058+0000","last_became_active":"2023-11-28T11:09:54.296456+0000","last_became_peered":"2023-11-28T11:09:54.296456+0000","last_unstale":"2023-11-28T11:09:55.279058+0000","last_undegraded":"2023-11-28T11:09:55.279058+0000","last_fullsized":"2023-11-28T11:09:55.279058+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:43:19.089769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278722+0000","last_change":"2023-11-28T11:09:53.893080+0000","last_active":"2023-11-28T11:09:55.278722+0000","last_peered":"2023-11-28T11:09:55.278722+0000","last_clean":"2023-11-28T11:09:55.278722+0000","last_became_active":"2023-11-28T11:09:53.892644+0000","last_became_peered":"2023-11-28T11:09:53.892644+0000","last_unstale":"2023-11-28T11:09:55.278722+0000","last_undegraded":"2023-11-28T11:09:55.278722+0000","last_fullsized":"2023-11-28T11:09:55.278722+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:25:14.791056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.690147+0000","last_change":"2023-11-28T11:09:54.284716+0000","last_active":"2023-11-28T11:09:55.690147+0000","last_peered":"2023-11-28T11:09:55.690147+0000","last_clean":"2023-11-28T11:09:55.690147+0000","last_became_active":"2023-11-28T11:09:54.284419+0000","last_became_peered":"2023-11-28T11:09:54.284419+0000","last_unstale":"2023-11-28T11:09:55.690147+0000","last_undegraded":"2023-11-28T11:09:55.690147+0000","last_fullsized":"2023-11-28T11:09:55.690147+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:50:40.722651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.281703+0000","last_change":"2023-11-28T11:09:53.884533+0000","last_active":"2023-11-28T11:09:55.281703+0000","last_peered":"2023-11-28T11:09:55.281703+0000","last_clean":"2023-11-28T11:09:55.281703+0000","last_became_active":"2023-11-28T11:09:53.884236+0000","last_became_peered":"2023-11-28T11:09:53.884236+0000","last_unstale":"2023-11-28T11:09:55.281703+0000","last_undegraded":"2023-11-28T11:09:55.281703+0000","last_fullsized":"2023-11-28T11:09:55.281703+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:36:43.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278965+0000","last_change":"2023-11-28T11:09:54.294678+0000","last_active":"2023-11-28T11:09:55.278965+0000","last_peered":"2023-11-28T11:09:55.278965+0000","last_clean":"2023-11-28T11:09:55.278965+0000","last_became_active":"2023-11-28T11:09:54.294419+0000","last_became_peered":"2023-11-28T11:09:54.294419+0000","last_unstale":"2023-11-28T11:09:55.278965+0000","last_undegraded":"2023-11-28T11:09:55.278965+0000","last_fullsized":"2023-11-28T11:09:55.278965+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:18:40.259162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865910+0000","last_change":"2023-11-28T11:09:53.883792+0000","last_active":"2023-11-28T11:09:55.865910+0000","last_peered":"2023-11-28T11:09:55.865910+0000","last_clean":"2023-11-28T11:09:55.865910+0000","last_became_active":"2023-11-28T11:09:53.883550+0000","last_became_peered":"2023-11-28T11:09:53.883550+0000","last_unstale":"2023-11-28T11:09:55.865910+0000","last_undegraded":"2023-11-28T11:09:55.865910+0000","last_fullsized":"2023-11-28T11:09:55.865910+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:59.962919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:09:54.281071+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:09:54.281071+0000","last_peered":"2023-11-28T11:09:54.281071+0000","last_clean":"2023-11-28T11:09:54.281071+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:09:54.281071+0000","last_undegraded":"2023-11-28T11:09:54.281071+0000","last_fullsized":"2023-11-28T11:09:54.281071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.281952+0000","last_change":"2023-11-28T11:09:54.299347+0000","last_active":"2023-11-28T11:09:55.281952+0000","last_peered":"2023-11-28T11:09:55.281952+0000","last_clean":"2023-11-28T11:09:55.281952+0000","last_became_active":"2023-11-28T11:09:54.299085+0000","last_became_peered":"2023-11-28T11:09:54.299085+0000","last_unstale":"2023-11-28T11:09:55.281952+0000","last_undegraded":"2023-11-28T11:09:55.281952+0000","last_fullsized":"2023-11-28T11:09:55.281952+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:10.092726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865526+0000","last_change":"2023-11-28T11:09:53.894343+0000","last_active":"2023-11-28T11:09:55.865526+0000","last_peered":"2023-11-28T11:09:55.865526+0000","last_clean":"2023-11-28T11:09:55.865526+0000","last_became_active":"2023-11-28T11:09:53.894074+0000","last_became_peered":"2023-11-28T11:09:53.894074+0000","last_unstale":"2023-11-28T11:09:55.865526+0000","last_undegraded":"2023-11-28T11:09:55.865526+0000","last_fullsized":"2023-11-28T11:09:55.865526+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:42.937102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":37,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:09:54.284078+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:09:54.284078+0000","last_peered":"2023-11-28T11:09:54.284078+0000","last_clean":"2023-11-28T11:09:54.284078+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:09:54.284078+0000","last_undegraded":"2023-11-28T11:09:54.284078+0000","last_fullsized":"2023-11-28T11:09:54.284078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864707+0000","last_change":"2023-11-28T11:09:54.300740+0000","last_active":"2023-11-28T11:09:55.864707+0000","last_peered":"2023-11-28T11:09:55.864707+0000","last_clean":"2023-11-28T11:09:55.864707+0000","last_became_active":"2023-11-28T11:09:54.300478+0000","last_became_peered":"2023-11-28T11:09:54.300478+0000","last_unstale":"2023-11-28T11:09:55.864707+0000","last_undegraded":"2023-11-28T11:09:55.864707+0000","last_fullsized":"2023-11-28T11:09:55.864707+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:18:40.259162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279048+0000","last_change":"2023-11-28T11:09:53.888334+0000","last_active":"2023-11-28T11:09:55.279048+0000","last_peered":"2023-11-28T11:09:55.279048+0000","last_clean":"2023-11-28T11:09:55.279048+0000","last_became_active":"2023-11-28T11:09:53.887962+0000","last_became_peered":"2023-11-28T11:09:53.887962+0000","last_unstale":"2023-11-28T11:09:55.279048+0000","last_undegraded":"2023-11-28T11:09:55.279048+0000","last_fullsized":"2023-11-28T11:09:55.279048+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:16:31.069137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:09:54.279862+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:09:54.279862+0000","last_peered":"2023-11-28T11:09:54.279862+0000","last_clean":"2023-11-28T11:09:54.279862+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:09:54.279862+0000","last_undegraded":"2023-11-28T11:09:54.279862+0000","last_fullsized":"2023-11-28T11:09:54.279862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864843+0000","last_change":"2023-11-28T11:09:54.294351+0000","last_active":"2023-11-28T11:09:55.864843+0000","last_peered":"2023-11-28T11:09:55.864843+0000","last_clean":"2023-11-28T11:09:55.864843+0000","last_became_active":"2023-11-28T11:09:54.294064+0000","last_became_peered":"2023-11-28T11:09:54.294064+0000","last_unstale":"2023-11-28T11:09:55.864843+0000","last_undegraded":"2023-11-28T11:09:55.864843+0000","last_fullsized":"2023-11-28T11:09:55.864843+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:59.261799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.279132+0000","last_change":"2023-11-28T11:09:53.888284+0000","last_active":"2023-11-28T11:09:55.279132+0000","last_peered":"2023-11-28T11:09:55.279132+0000","last_clean":"2023-11-28T11:09:55.279132+0000","last_became_active":"2023-11-28T11:09:53.887999+0000","last_became_peered":"2023-11-28T11:09:53.887999+0000","last_unstale":"2023-11-28T11:09:55.279132+0000","last_undegraded":"2023-11-28T11:09:55.279132+0000","last_fullsized":"2023-11-28T11:09:55.279132+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:58.473943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":37,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:09:54.284924+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:09:54.284924+0000","last_peered":"2023-11-28T11:09:54.284924+0000","last_clean":"2023-11-28T11:09:54.284924+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:09:54.284924+0000","last_undegraded":"2023-11-28T11:09:54.284924+0000","last_fullsized":"2023-11-28T11:09:54.284924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:09:54.286386+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:09:54.286386+0000","last_peered":"2023-11-28T11:09:54.286386+0000","last_clean":"2023-11-28T11:09:54.286386+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:09:54.286386+0000","last_undegraded":"2023-11-28T11:09:54.286386+0000","last_fullsized":"2023-11-28T11:09:54.286386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.689869+0000","last_change":"2023-11-28T11:09:54.286437+0000","last_active":"2023-11-28T11:09:55.689869+0000","last_peered":"2023-11-28T11:09:55.689869+0000","last_clean":"2023-11-28T11:09:55.689869+0000","last_became_active":"2023-11-28T11:09:54.286186+0000","last_became_peered":"2023-11-28T11:09:54.286186+0000","last_unstale":"2023-11-28T11:09:55.689869+0000","last_undegraded":"2023-11-28T11:09:55.689869+0000","last_fullsized":"2023-11-28T11:09:55.689869+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:34.535871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865730+0000","last_change":"2023-11-28T11:09:53.886433+0000","last_active":"2023-11-28T11:09:55.865730+0000","last_peered":"2023-11-28T11:09:55.865730+0000","last_clean":"2023-11-28T11:09:55.865730+0000","last_became_active":"2023-11-28T11:09:53.886052+0000","last_became_peered":"2023-11-28T11:09:53.886052+0000","last_unstale":"2023-11-28T11:09:55.865730+0000","last_undegraded":"2023-11-28T11:09:55.865730+0000","last_fullsized":"2023-11-28T11:09:55.865730+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:40:12.552818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:09:54.286211+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:09:54.286211+0000","last_peered":"2023-11-28T11:09:54.286211+0000","last_clean":"2023-11-28T11:09:54.286211+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:09:54.286211+0000","last_undegraded":"2023-11-28T11:09:54.286211+0000","last_fullsized":"2023-11-28T11:09:54.286211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.865047+0000","last_change":"2023-11-28T11:09:54.283364+0000","last_active":"2023-11-28T11:09:55.865047+0000","last_peered":"2023-11-28T11:09:55.865047+0000","last_clean":"2023-11-28T11:09:55.865047+0000","last_became_active":"2023-11-28T11:09:54.283070+0000","last_became_peered":"2023-11-28T11:09:54.283070+0000","last_unstale":"2023-11-28T11:09:55.865047+0000","last_undegraded":"2023-11-28T11:09:55.865047+0000","last_fullsized":"2023-11-28T11:09:55.865047+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:56.494742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.282069+0000","last_change":"2023-11-28T11:09:53.886108+0000","last_active":"2023-11-28T11:09:55.282069+0000","last_peered":"2023-11-28T11:09:55.282069+0000","last_clean":"2023-11-28T11:09:55.282069+0000","last_became_active":"2023-11-28T11:09:53.885821+0000","last_became_peered":"2023-11-28T11:09:53.885821+0000","last_unstale":"2023-11-28T11:09:55.282069+0000","last_undegraded":"2023-11-28T11:09:55.282069+0000","last_fullsized":"2023-11-28T11:09:55.282069+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:45.115674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:09:54.289375+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:09:54.289375+0000","last_peered":"2023-11-28T11:09:54.289375+0000","last_clean":"2023-11-28T11:09:54.289375+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:09:54.289375+0000","last_undegraded":"2023-11-28T11:09:54.289375+0000","last_fullsized":"2023-11-28T11:09:54.289375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.278906+0000","last_change":"2023-11-28T11:09:54.295175+0000","last_active":"2023-11-28T11:09:55.278906+0000","last_peered":"2023-11-28T11:09:55.278906+0000","last_clean":"2023-11-28T11:09:55.278906+0000","last_became_active":"2023-11-28T11:09:54.294883+0000","last_became_peered":"2023-11-28T11:09:54.294883+0000","last_unstale":"2023-11-28T11:09:55.278906+0000","last_undegraded":"2023-11-28T11:09:55.278906+0000","last_fullsized":"2023-11-28T11:09:55.278906+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:55.105792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864794+0000","last_change":"2023-11-28T11:09:53.887922+0000","last_active":"2023-11-28T11:09:55.864794+0000","last_peered":"2023-11-28T11:09:55.864794+0000","last_clean":"2023-11-28T11:09:55.864794+0000","last_became_active":"2023-11-28T11:09:53.887679+0000","last_became_peered":"2023-11-28T11:09:53.887679+0000","last_unstale":"2023-11-28T11:09:55.864794+0000","last_undegraded":"2023-11-28T11:09:55.864794+0000","last_fullsized":"2023-11-28T11:09:55.864794+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:56.044847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:09:54.281279+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:09:54.281279+0000","last_peered":"2023-11-28T11:09:54.281279+0000","last_clean":"2023-11-28T11:09:54.281279+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:09:54.281279+0000","last_undegraded":"2023-11-28T11:09:54.281279+0000","last_fullsized":"2023-11-28T11:09:54.281279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.282237+0000","last_change":"2023-11-28T11:09:54.296713+0000","last_active":"2023-11-28T11:09:55.282237+0000","last_peered":"2023-11-28T11:09:55.282237+0000","last_clean":"2023-11-28T11:09:55.282237+0000","last_became_active":"2023-11-28T11:09:54.296367+0000","last_became_peered":"2023-11-28T11:09:54.296367+0000","last_unstale":"2023-11-28T11:09:55.282237+0000","last_undegraded":"2023-11-28T11:09:55.282237+0000","last_fullsized":"2023-11-28T11:09:55.282237+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:53.865953+0000","last_clean_scrub_stamp":"2023-11-28T11:09:53.865953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:47:01.823875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:09:55.864781+0000","last_change":"2023-11-28T11:09:53.883492+0000","last_active":"2023-11-28T11:09:55.864781+0000","last_peered":"2023-11-28T11:09:55.864781+0000","last_clean":"2023-11-28T11:09:55.864781+0000","last_became_active":"2023-11-28T11:09:53.883215+0000","last_became_peered":"2023-11-28T11:09:53.883215+0000","last_unstale":"2023-11-28T11:09:55.864781+0000","last_undegraded":"2023-11-28T11:09:55.864781+0000","last_fullsized":"2023-11-28T11:09:55.864781+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:09:52.861641+0000","last_clean_scrub_stamp":"2023-11-28T11:09:52.861641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:16:31.069137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:09:54.286872+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:09:54.286872+0000","last_peered":"2023-11-28T11:09:54.286872+0000","last_clean":"2023-11-28T11:09:54.286872+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:09:54.286872+0000","last_undegraded":"2023-11-28T11:09:54.286872+0000","last_fullsized":"2023-11-28T11:09:54.286872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10236,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361604,"statfs":{"total":96636764160,"available":96626282496,"internally_reserved":0,"allocated":1167360,"data_stored":831978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.438},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57499999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.625}]},{"osd":3,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.753},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67100000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79800000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.77400000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.84399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8836,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363004,"statfs":{"total":96636764160,"available":96627716096,"internally_reserved":0,"allocated":585728,"data_stored":248775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":5.1239999999999997,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.0410000000000004,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.443}]},{"osd":1,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.0250000000000004,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":5.1509999999999998,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.57599999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":6.0620000000000003,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.9409999999999998,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.40799999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.51300000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.61099999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.623}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8964,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362876,"statfs":{"total":96636764160,"available":96627585024,"internally_reserved":0,"allocated":585728,"data_stored":248299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.0519999999999996,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.46100000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":7.0890000000000004,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.41199999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.375},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38400000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":18.024999999999999,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57399999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":17.966999999999999,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":18.056999999999999,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.68300000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10056,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94361784,"statfs":{"total":96636764160,"available":96626466816,"internally_reserved":0,"allocated":1179648,"data_stored":838563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.379},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.27700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.35499999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.40799999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.51500000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52200000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57599999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8904,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362936,"statfs":{"total":96636764160,"available":96627646464,"internally_reserved":0,"allocated":589824,"data_stored":249687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.32900000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.36799999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.379}]},{"osd":4,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69499999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67600000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66700000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68799999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9164,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362676,"statfs":{"total":96636764160,"available":96627380224,"internally_reserved":0,"allocated":593920,"data_stored":249709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.31}]},{"osd":1,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.36499999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.72799999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69299999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.745}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9088,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362752,"statfs":{"total":96636764160,"available":96627458048,"internally_reserved":0,"allocated":581632,"data_stored":248705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.53600000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":12.263,"5min":12.263,"15min":12.263},"last":0.60699999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.71599999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.625}]},{"osd":5,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.59199999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.75},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64600000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.92500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9020,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362820,"statfs":{"total":96636764160,"available":96627527680,"internally_reserved":0,"allocated":577536,"data_stored":248139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.56100000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.437},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.72099999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.495}]},{"osd":4,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.59499999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.68400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:09:58.731 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:09:58.732 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:09:58.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.893+0000 7f1698a94700 1 Processor -- start 2023-11-28T11:09:58.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.893+0000 7f1698a94700 1 -- start start 2023-11-28T11:09:58.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.894+0000 7f1698a94700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1694154e70 0x7f16940cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.894+0000 7f1698a94700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1694071750 0x7f169415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.894+0000 7f1698a94700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1694155240 con 0x7f16940cf360 2023-11-28T11:09:58.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.894+0000 7f1698a94700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169415b100 con 0x7f1694154e70 2023-11-28T11:09:58.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.894+0000 7f1698a94700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169415b3e0 con 0x7f1694071750 2023-11-28T11:09:58.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.894+0000 7f169259c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1694154e70 0x7f16940cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.894+0000 7f1691d9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f16940cf360 0x7f1694071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49608/0 (socket says 172.21.15.156:49608) 2023-11-28T11:09:58.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.894+0000 7f1691d9b700 1 -- 172.21.15.156:0/2735951710 learned_addr learned my addr 172.21.15.156:0/2735951710 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:09:58.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.894+0000 7f1692d9d700 1 --2- 172.21.15.156:0/2735951710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1694071750 0x7f169415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.894+0000 7f169259c700 1 -- 172.21.15.156:0/2735951710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1694071750 msgr2=0x7f169415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:58.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f169259c700 1 --2- 172.21.15.156:0/2735951710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1694071750 0x7f169415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f169259c700 1 -- 172.21.15.156:0/2735951710 >> v1:172.21.15.156:6789/0 conn(0x7f16940cf360 legacy=0x7f1694071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f169259c700 1 -- 172.21.15.156:0/2735951710 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169415b6c0 con 0x7f1694154e70 2023-11-28T11:09:58.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f169159a700 1 -- 172.21.15.156:0/2735951710 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530361235 0 0) 0x7f1694155240 con 0x7f16940cf360 2023-11-28T11:09:58.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f169259c700 1 --2- 172.21.15.156:0/2735951710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1694154e70 0x7f16940cdbd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1688009840 tx=0x7f1688009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0137882dc8ba90 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f169159a700 1 -- 172.21.15.156:0/2735951710 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688011830 con 0x7f1694154e70 2023-11-28T11:09:58.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f169159a700 1 -- 172.21.15.156:0/2735951710 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1688011990 con 0x7f1694154e70 2023-11-28T11:09:58.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f169159a700 1 -- 172.21.15.156:0/2735951710 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688011d20 con 0x7f1694154e70 2023-11-28T11:09:58.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f1698a94700 1 -- 172.21.15.156:0/2735951710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1694154e70 msgr2=0x7f16940cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f1698a94700 1 --2- 172.21.15.156:0/2735951710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1694154e70 0x7f16940cdbd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1688009840 tx=0x7f1688009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:58.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f1698a94700 1 -- 172.21.15.156:0/2735951710 shutdown_connections 2023-11-28T11:09:58.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f1698a94700 1 --2- 172.21.15.156:0/2735951710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1694071750 0x7f169415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f1698a94700 1 --2- 172.21.15.156:0/2735951710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1694154e70 0x7f16940cdbd0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f1698a94700 1 -- 172.21.15.156:0/2735951710 >> 172.21.15.156:0/2735951710 conn(0x7f16940d0000 msgr2=0x7f16940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:58.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.895+0000 7f1698a94700 1 -- 172.21.15.156:0/2735951710 shutdown_connections 2023-11-28T11:09:58.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.896+0000 7f1698a94700 1 -- 172.21.15.156:0/2735951710 wait complete. 2023-11-28T11:09:58.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.897+0000 7f1698a94700 1 Processor -- start 2023-11-28T11:09:58.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.897+0000 7f1698a94700 1 -- start start 2023-11-28T11:09:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1698a94700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1694071750 0x7f169414cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1698a94700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16940cf360 0x7f16941e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1698a94700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1694154e70 0x7f169415d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1698a94700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169415a930 con 0x7f16940cf360 2023-11-28T11:09:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1698a94700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169414d280 con 0x7f1694154e70 2023-11-28T11:09:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1698a94700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16941e4b40 con 0x7f1694071750 2023-11-28T11:09:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1691d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16940cf360 0x7f16941e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1691d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16940cf360 0x7f16941e3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34738/0 (socket says 172.21.15.156:34738) 2023-11-28T11:09:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1691d9b700 1 -- 172.21.15.156:0/70745785 learned_addr learned my addr 172.21.15.156:0/70745785 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f169259c700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1694071750 0x7f169414cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1692d9d700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1694154e70 0x7f169415d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1691d9b700 1 -- 172.21.15.156:0/70745785 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1694071750 msgr2=0x7f169414cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1691d9b700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1694071750 0x7f169414cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1691d9b700 1 -- 172.21.15.156:0/70745785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1694154e70 msgr2=0x7f169415d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1691d9b700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1694154e70 0x7f169415d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1691d9b700 1 -- 172.21.15.156:0/70745785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16880094a0 con 0x7f16940cf360 2023-11-28T11:09:58.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1691d9b700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16940cf360 0x7f16941e3d80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f16840011e0 tx=0x7f168400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1682ffd700 1 -- 172.21.15.156:0/70745785 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1684004c80 con 0x7f16940cf360 2023-11-28T11:09:58.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1682ffd700 1 -- 172.21.15.156:0/70745785 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1684004e00 con 0x7f16940cf360 2023-11-28T11:09:58.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1698a94700 1 -- 172.21.15.156:0/70745785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16940af520 con 0x7f16940cf360 2023-11-28T11:09:58.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1682ffd700 1 -- 172.21.15.156:0/70745785 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1684007310 con 0x7f16940cf360 2023-11-28T11:09:58.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.898+0000 7f1698a94700 1 -- 172.21.15.156:0/70745785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169415b9a0 con 0x7f16940cf360 2023-11-28T11:09:58.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.899+0000 7f1698a94700 1 -- 172.21.15.156:0/70745785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169415adb0 con 0x7f16940cf360 2023-11-28T11:09:58.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.900+0000 7f1682ffd700 1 -- 172.21.15.156:0/70745785 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f168401a020 con 0x7f16940cf360 2023-11-28T11:09:58.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.900+0000 7f1682ffd700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f16740682f0 0x7f167406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.900+0000 7f1682ffd700 1 -- 172.21.15.156:0/70745785 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1684085960 con 0x7f16940cf360 2023-11-28T11:09:58.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.900+0000 7f169259c700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f16740682f0 0x7f167406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.901+0000 7f169259c700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f16740682f0 0x7f167406a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1688000c00 tx=0x7f1688008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:58.904+0000 7f1682ffd700 1 -- 172.21.15.156:0/70745785 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f16840521c0 con 0x7f16940cf360 2023-11-28T11:09:59.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.163+0000 7f1698a94700 1 -- 172.21.15.156:0/70745785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f169415aa70 con 0x7f16940cf360 2023-11-28T11:09:59.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.167+0000 7f1682ffd700 1 -- 172.21.15.156:0/70745785 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 184+0+0 (secure 0 0 0) 0x7f169415aa70 con 0x7f16940cf360 2023-11-28T11:09:59.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.169+0000 7f1698a94700 1 -- 172.21.15.156:0/70745785 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f16740682f0 msgr2=0x7f167406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.169+0000 7f1698a94700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f16740682f0 0x7f167406a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1688000c00 tx=0x7f1688008030 comp rx=0 tx=0).stop 2023-11-28T11:09:59.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.169+0000 7f1698a94700 1 -- 172.21.15.156:0/70745785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16940cf360 msgr2=0x7f16941e3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.169+0000 7f1698a94700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16940cf360 0x7f16941e3d80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f16840011e0 tx=0x7f168400cae0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.169+0000 7f1698a94700 1 -- 172.21.15.156:0/70745785 shutdown_connections 2023-11-28T11:09:59.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.169+0000 7f1698a94700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1694154e70 0x7f169415d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.169+0000 7f1698a94700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f16740682f0 0x7f167406a7a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.169+0000 7f1698a94700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1694071750 0x7f169414cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.169+0000 7f1698a94700 1 --2- 172.21.15.156:0/70745785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16940cf360 0x7f16941e3d80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.169+0000 7f1698a94700 1 -- 172.21.15.156:0/70745785 >> 172.21.15.156:0/70745785 conn(0x7f16940d0000 msgr2=0x7f16940ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:59.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.169+0000 7f1698a94700 1 -- 172.21.15.156:0/70745785 shutdown_connections 2023-11-28T11:09:59.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.169+0000 7f1698a94700 1 -- 172.21.15.156:0/70745785 wait complete. 2023-11-28T11:09:59.171 INFO:teuthology.orchestra.run.smithi156.stderr:updated caps for client.0 2023-11-28T11:09:59.182 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:59.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.302+0000 7f06073a8700 1 Processor -- start 2023-11-28T11:09:59.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.303+0000 7f06073a8700 1 -- start start 2023-11-28T11:09:59.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.303+0000 7f06073a8700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f060006e500 0x7f060015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:59.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.303+0000 7f06073a8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f060006a5e0 0x7f060006aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:59.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.303+0000 7f06073a8700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06000aee40 con 0x7f06000ae9d0 2023-11-28T11:09:59.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.303+0000 7f06073a8700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060015bf10 con 0x7f060006a5e0 2023-11-28T11:09:59.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.303+0000 7f06073a8700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060006c930 con 0x7f060006e500 2023-11-28T11:09:59.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f0605945700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f060006a5e0 0x7f060006aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:59.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f0605945700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f060006a5e0 0x7f060006aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34748/0 (socket says 172.21.15.156:34748) 2023-11-28T11:09:59.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f0605945700 1 -- 172.21.15.156:0/793051774 learned_addr learned my addr 172.21.15.156:0/793051774 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:59.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f0605144700 1 --2- 172.21.15.156:0/793051774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f060006e500 0x7f060015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:59.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f0605945700 1 -- 172.21.15.156:0/793051774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f060006e500 msgr2=0x7f060015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:59.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f0605945700 1 --2- 172.21.15.156:0/793051774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f060006e500 0x7f060015b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f0605945700 1 -- 172.21.15.156:0/793051774 >> v1:172.21.15.186:6789/0 conn(0x7f06000ae9d0 legacy=0x7f060006e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f0605945700 1 -- 172.21.15.156:0/793051774 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060006cc10 con 0x7f060006a5e0 2023-11-28T11:09:59.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f0605144700 1 --2- 172.21.15.156:0/793051774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f060006e500 0x7f060015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:59.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f0605945700 1 --2- 172.21.15.156:0/793051774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f060006a5e0 0x7f060006aa00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009e50 tx=0x7f05f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e30328c03cdf9bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:59.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f05f7fff700 1 -- 172.21.15.156:0/793051774 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800e070 con 0x7f060006a5e0 2023-11-28T11:09:59.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f05f7fff700 1 -- 172.21.15.156:0/793051774 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f800b9c0 con 0x7f060006a5e0 2023-11-28T11:09:59.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.304+0000 7f05f7fff700 1 -- 172.21.15.156:0/793051774 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800bd70 con 0x7f060006a5e0 2023-11-28T11:09:59.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.305+0000 7f06073a8700 1 -- 172.21.15.156:0/793051774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f060006a5e0 msgr2=0x7f060006aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.305+0000 7f06073a8700 1 --2- 172.21.15.156:0/793051774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f060006a5e0 0x7f060006aa00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009e50 tx=0x7f05f800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:59.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.305+0000 7f06073a8700 1 -- 172.21.15.156:0/793051774 shutdown_connections 2023-11-28T11:09:59.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.305+0000 7f06073a8700 1 --2- 172.21.15.156:0/793051774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f060006a5e0 0x7f060006aa00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.305+0000 7f06073a8700 1 --2- 172.21.15.156:0/793051774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f060006e500 0x7f060015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.305+0000 7f06073a8700 1 -- 172.21.15.156:0/793051774 >> 172.21.15.156:0/793051774 conn(0x7f06000b12b0 msgr2=0x7f06000b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:59.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.305+0000 7f06073a8700 1 -- 172.21.15.156:0/793051774 shutdown_connections 2023-11-28T11:09:59.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.305+0000 7f06073a8700 1 -- 172.21.15.156:0/793051774 wait complete. 2023-11-28T11:09:59.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.306+0000 7f06073a8700 1 Processor -- start 2023-11-28T11:09:59.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.306+0000 7f06073a8700 1 -- start start 2023-11-28T11:09:59.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.307+0000 7f06073a8700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f060006a5e0 0x7f060014dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:59.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.307+0000 7f06073a8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f060006e500 0x7f060014d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:59.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.307+0000 7f06073a8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06000ae9d0 0x7f06001f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:59.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.307+0000 7f06073a8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000c7f50 con 0x7f06000ae9d0 2023-11-28T11:09:59.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.307+0000 7f06073a8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060014d970 con 0x7f060006e500 2023-11-28T11:09:59.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.307+0000 7f06073a8700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06000c7340 con 0x7f060006a5e0 2023-11-28T11:09:59.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.307+0000 7f0605945700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06000ae9d0 0x7f06001f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:59.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.307+0000 7f0605945700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06000ae9d0 0x7f06001f3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34752/0 (socket says 172.21.15.156:34752) 2023-11-28T11:09:59.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.307+0000 7f0605945700 1 -- 172.21.15.156:0/1882339954 learned_addr learned my addr 172.21.15.156:0/1882339954 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:09:59.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f0605144700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f060006a5e0 0x7f060014dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:59.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f0605945700 1 -- 172.21.15.156:0/1882339954 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f060006a5e0 msgr2=0x7f060014dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f0605945700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f060006a5e0 0x7f060014dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f0605945700 1 -- 172.21.15.156:0/1882339954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f060006e500 msgr2=0x7f060014d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f0604943700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f060006e500 0x7f060014d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:59.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f0605945700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f060006e500 0x7f060014d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f0605945700 1 -- 172.21.15.156:0/1882339954 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8009b30 con 0x7f06000ae9d0 2023-11-28T11:09:59.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f0605945700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06000ae9d0 0x7f06001f3a70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f060006c510 tx=0x7f05f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:59.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f05f5ffb700 1 -- 172.21.15.156:0/1882339954 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8004960 con 0x7f06000ae9d0 2023-11-28T11:09:59.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f05f5ffb700 1 -- 172.21.15.156:0/1882339954 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f80099b0 con 0x7f06000ae9d0 2023-11-28T11:09:59.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f05f5ffb700 1 -- 172.21.15.156:0/1882339954 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f801c680 con 0x7f06000ae9d0 2023-11-28T11:09:59.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f06073a8700 1 -- 172.21.15.156:0/1882339954 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060006cef0 con 0x7f06000ae9d0 2023-11-28T11:09:59.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.308+0000 7f06073a8700 1 -- 172.21.15.156:0/1882339954 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06000664c0 con 0x7f06000ae9d0 2023-11-28T11:09:59.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.309+0000 7f06073a8700 1 -- 172.21.15.156:0/1882339954 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060015ac80 con 0x7f06000ae9d0 2023-11-28T11:09:59.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.310+0000 7f05f5ffb700 1 -- 172.21.15.156:0/1882339954 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f05f802c070 con 0x7f06000ae9d0 2023-11-28T11:09:59.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.310+0000 7f05f5ffb700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f05e40682f0 0x7f05e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:59.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.310+0000 7f05f5ffb700 1 -- 172.21.15.156:0/1882339954 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f05f80878a0 con 0x7f06000ae9d0 2023-11-28T11:09:59.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.310+0000 7f0605144700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f05e40682f0 0x7f05e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:59.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.310+0000 7f0605144700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f05e40682f0 0x7f05e406a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f06000c8720 tx=0x7f05f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:59.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.314+0000 7f05f5ffb700 1 -- 172.21.15.156:0/1882339954 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f05f8054100 con 0x7f06000ae9d0 2023-11-28T11:09:59.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.538+0000 7f06073a8700 1 -- 172.21.15.156:0/1882339954 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06000c82e0 con 0x7f06000ae9d0 2023-11-28T11:09:59.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.538+0000 7f05f5ffb700 1 -- 172.21.15.156:0/1882339954 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f05f8057720 con 0x7f06000ae9d0 2023-11-28T11:09:59.540 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:09:59.540 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/365552946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":365552946},{"type":"v1","addr":"172.21.15.186:6834","nonce":365552946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1918325985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":1918325985},{"type":"v1","addr":"172.21.15.156:6837","nonce":1918325985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/2889641252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2889641252},{"type":"v1","addr":"172.21.15.156:6836","nonce":2889641252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:54.274258+0000","modified":"2023-11-28T11:09:57.285496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3832587641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3832587641},{"type":"v1","addr":"172.21.15.186:6835","nonce":3832587641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:09:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.540+0000 7f06073a8700 1 -- 172.21.15.156:0/1882339954 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f05e40682f0 msgr2=0x7f05e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.540+0000 7f06073a8700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f05e40682f0 0x7f05e406a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f06000c8720 tx=0x7f05f0008040 comp rx=0 tx=0).stop 2023-11-28T11:09:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.540+0000 7f06073a8700 1 -- 172.21.15.156:0/1882339954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06000ae9d0 msgr2=0x7f06001f3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.540+0000 7f06073a8700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06000ae9d0 0x7f06001f3a70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f060006c510 tx=0x7f05f8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.540+0000 7f06073a8700 1 -- 172.21.15.156:0/1882339954 shutdown_connections 2023-11-28T11:09:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.540+0000 7f06073a8700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06000ae9d0 0x7f06001f3a70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.540+0000 7f06073a8700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f05e40682f0 0x7f05e406a7a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.540+0000 7f06073a8700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f060006a5e0 0x7f060014dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.540+0000 7f06073a8700 1 --2- 172.21.15.156:0/1882339954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f060006e500 0x7f060014d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.540+0000 7f06073a8700 1 -- 172.21.15.156:0/1882339954 >> 172.21.15.156:0/1882339954 conn(0x7f06000b12b0 msgr2=0x7f06000cb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:59.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.540+0000 7f06073a8700 1 -- 172.21.15.156:0/1882339954 shutdown_connections 2023-11-28T11:09:59.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:09:59.540+0000 7f06073a8700 1 -- 172.21.15.156:0/1882339954 wait complete. 2023-11-28T11:09:59.543 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T11:09:59.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:09:54.274258+0000', 'modified': '2023-11-28T11:09:57.285496+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5547': {'gid': 5547, 'name': 'd', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6835/3832587641', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6833', 'nonce': 3832587641}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 3832587641}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:09:59.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:09:59.956 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T11:10:00.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.078+0000 7f6569ebd700 1 Processor -- start 2023-11-28T11:10:00.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.078+0000 7f6569ebd700 1 -- start start 2023-11-28T11:10:00.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.079+0000 7f6569ebd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f65640cbdf0 0x7f65640af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.079+0000 7f6569ebd700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65640cc5f0 con 0x7f656415cef0 2023-11-28T11:10:00.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.079+0000 7f6569ebd700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656415d470 con 0x7f656406ba90 2023-11-28T11:10:00.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.079+0000 7f6569ebd700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656415d750 con 0x7f65640cbdf0 2023-11-28T11:10:00.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.079+0000 7f65637fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f656415cef0 0x7f656406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49636/0 (socket says 172.21.15.156:49636) 2023-11-28T11:10:00.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.079+0000 7f65637fe700 1 -- 172.21.15.156:0/2253305468 learned_addr learned my addr 172.21.15.156:0/2253305468 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:00.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.079+0000 7f6563fff700 1 --2- 172.21.15.156:0/2253305468 >> v2:172.21.15.186:3300/0 conn(0x7f65640cbdf0 0x7f65640af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.080+0000 7f65627fc700 1 -- 172.21.15.156:0/2253305468 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398836354 0 0) 0x7f65640cc5f0 con 0x7f656415cef0 2023-11-28T11:10:00.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.080+0000 7f65627fc700 1 -- 172.21.15.156:0/2253305468 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f654c0035b0 con 0x7f656415cef0 2023-11-28T11:10:00.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.080+0000 7f6563fff700 1 -- 172.21.15.156:0/2253305468 >> v1:172.21.15.186:6789/0 conn(0x7f656406ba90 legacy=0x7f65640ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.080+0000 7f6563fff700 1 -- 172.21.15.156:0/2253305468 >> v1:172.21.15.156:6789/0 conn(0x7f656415cef0 legacy=0x7f656406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.080+0000 7f6563fff700 1 -- 172.21.15.156:0/2253305468 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656415da30 con 0x7f65640cbdf0 2023-11-28T11:10:00.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.080+0000 7f6563fff700 1 --2- 172.21.15.156:0/2253305468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65640cbdf0 0x7f65640af400 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f655c009d90 tx=0x7f655c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb9fe6390f2ce3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.080+0000 7f65627fc700 1 -- 172.21.15.156:0/2253305468 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c00e070 con 0x7f65640cbdf0 2023-11-28T11:10:00.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.080+0000 7f65627fc700 1 -- 172.21.15.156:0/2253305468 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c00bb20 con 0x7f65640cbdf0 2023-11-28T11:10:00.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.080+0000 7f65627fc700 1 -- 172.21.15.156:0/2253305468 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c00be90 con 0x7f65640cbdf0 2023-11-28T11:10:00.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.081+0000 7f6569ebd700 1 -- 172.21.15.156:0/2253305468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65640cbdf0 msgr2=0x7f65640af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.081+0000 7f6569ebd700 1 --2- 172.21.15.156:0/2253305468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65640cbdf0 0x7f65640af400 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f655c009d90 tx=0x7f655c00b330 comp rx=0 tx=0).stop 2023-11-28T11:10:00.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.081+0000 7f6569ebd700 1 -- 172.21.15.156:0/2253305468 shutdown_connections 2023-11-28T11:10:00.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.081+0000 7f6569ebd700 1 --2- 172.21.15.156:0/2253305468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65640cbdf0 0x7f65640af400 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.081+0000 7f6569ebd700 1 -- 172.21.15.156:0/2253305468 >> 172.21.15.156:0/2253305468 conn(0x7f65640d0000 msgr2=0x7f65640d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:00.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.081+0000 7f6569ebd700 1 -- 172.21.15.156:0/2253305468 shutdown_connections 2023-11-28T11:10:00.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.081+0000 7f6569ebd700 1 -- 172.21.15.156:0/2253305468 wait complete. 2023-11-28T11:10:00.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.082+0000 7f6569ebd700 1 Processor -- start 2023-11-28T11:10:00.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.082+0000 7f6569ebd700 1 -- start start 2023-11-28T11:10:00.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6569ebd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f656406ba90 0x7f65640c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6569ebd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65640cbdf0 0x7f65640c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6569ebd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f656415cef0 0x7f6564153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6569ebd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65640c84f0 con 0x7f65640cbdf0 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6569ebd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65640c7610 con 0x7f656415cef0 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6569ebd700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65640c78e0 con 0x7f656406ba90 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6562ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65640cbdf0 0x7f65640c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6562ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65640cbdf0 0x7f65640c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34766/0 (socket says 172.21.15.156:34766) 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6562ffd700 1 -- 172.21.15.156:0/4000188397 learned_addr learned my addr 172.21.15.156:0/4000188397 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f65637fe700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f656406ba90 0x7f65640c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6563fff700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f656415cef0 0x7f6564153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6562ffd700 1 -- 172.21.15.156:0/4000188397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f656406ba90 msgr2=0x7f65640c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6562ffd700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f656406ba90 0x7f65640c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6562ffd700 1 -- 172.21.15.156:0/4000188397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f656415cef0 msgr2=0x7f6564153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6562ffd700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f656415cef0 0x7f6564153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6562ffd700 1 -- 172.21.15.156:0/4000188397 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c009a70 con 0x7f65640cbdf0 2023-11-28T11:10:00.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f6562ffd700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65640cbdf0 0x7f65640c70d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f655400c4e0 tx=0x7f655400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.083+0000 7f654bfff700 1 -- 172.21.15.156:0/4000188397 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65540052d0 con 0x7f65640cbdf0 2023-11-28T11:10:00.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.084+0000 7f654bfff700 1 -- 172.21.15.156:0/4000188397 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6554005430 con 0x7f65640cbdf0 2023-11-28T11:10:00.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.084+0000 7f6569ebd700 1 -- 172.21.15.156:0/4000188397 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656415dff0 con 0x7f65640cbdf0 2023-11-28T11:10:00.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.084+0000 7f654bfff700 1 -- 172.21.15.156:0/4000188397 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65540038c0 con 0x7f65640cbdf0 2023-11-28T11:10:00.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.084+0000 7f6569ebd700 1 -- 172.21.15.156:0/4000188397 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656415dd10 con 0x7f65640cbdf0 2023-11-28T11:10:00.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.085+0000 7f654bfff700 1 -- 172.21.15.156:0/4000188397 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6554019030 con 0x7f65640cbdf0 2023-11-28T11:10:00.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.085+0000 7f654bfff700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65500682f0 0x7f655006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.085+0000 7f654bfff700 1 -- 172.21.15.156:0/4000188397 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6554085100 con 0x7f65640cbdf0 2023-11-28T11:10:00.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.085+0000 7f65637fe700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65500682f0 0x7f655006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.086+0000 7f65637fe700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65500682f0 0x7f655006a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f65580011e0 tx=0x7f6558009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.087+0000 7f6576046b80 1 Processor -- start 2023-11-28T11:10:00.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.087+0000 7f6576046b80 1 -- start start 2023-11-28T11:10:00.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.088+0000 7f6576046b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be76810 0x56254be76be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.088+0000 7f6576046b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be4b1c0 0x56254be77120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.088+0000 7f6576046b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be6fce0 0x56254be70150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.088+0000 7f6576046b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254bd36bc0 con 0x56254be76810 2023-11-28T11:10:00.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.088+0000 7f6576046b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254bda5350 con 0x56254be4b1c0 2023-11-28T11:10:00.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.088+0000 7f6576046b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56254bd9e880 con 0x56254be6fce0 2023-11-28T11:10:00.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.088+0000 7f65637fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be76810 0x56254be76be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.088+0000 7f65637fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be76810 0x56254be76be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34772/0 (socket says 172.21.15.156:34772) 2023-11-28T11:10:00.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.088+0000 7f65637fe700 1 -- 172.21.15.156:0/1079270123 learned_addr learned my addr 172.21.15.156:0/1079270123 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:00.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.089+0000 7f6563fff700 1 --2- 172.21.15.156:0/1079270123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be6fce0 0x56254be70150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.089+0000 7f6562ffd700 1 --2- 172.21.15.156:0/1079270123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be4b1c0 0x56254be77120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.089+0000 7f65637fe700 1 -- 172.21.15.156:0/1079270123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be6fce0 msgr2=0x56254be70150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:00.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.089+0000 7f65637fe700 1 --2- 172.21.15.156:0/1079270123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be6fce0 0x56254be70150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.089+0000 7f65637fe700 1 -- 172.21.15.156:0/1079270123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be4b1c0 msgr2=0x56254be77120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:00.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.089+0000 7f65637fe700 1 --2- 172.21.15.156:0/1079270123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be4b1c0 0x56254be77120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.089+0000 7f65637fe700 1 -- 172.21.15.156:0/1079270123 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56254bc205e0 con 0x56254be76810 2023-11-28T11:10:00.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.089+0000 7f6562ffd700 1 --2- 172.21.15.156:0/1079270123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be4b1c0 0x56254be77120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:00.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.089+0000 7f65637fe700 1 --2- 172.21.15.156:0/1079270123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be76810 0x56254be76be0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6558005b80 tx=0x7f6558011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e1c29cd7db1128 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.089+0000 7f65497fa700 1 -- 172.21.15.156:0/1079270123 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558007770 con 0x56254be76810 2023-11-28T11:10:00.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.089+0000 7f65497fa700 1 -- 172.21.15.156:0/1079270123 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6558021450 con 0x56254be76810 2023-11-28T11:10:00.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.089+0000 7f65497fa700 1 -- 172.21.15.156:0/1079270123 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558021800 con 0x56254be76810 2023-11-28T11:10:00.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.090+0000 7f6576046b80 1 -- 172.21.15.156:0/1079270123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be76810 msgr2=0x56254be76be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.090+0000 7f6576046b80 1 --2- 172.21.15.156:0/1079270123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be76810 0x56254be76be0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6558005b80 tx=0x7f6558011000 comp rx=0 tx=0).stop 2023-11-28T11:10:00.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.090+0000 7f6576046b80 1 -- 172.21.15.156:0/1079270123 shutdown_connections 2023-11-28T11:10:00.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.090+0000 7f6576046b80 1 --2- 172.21.15.156:0/1079270123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be6fce0 0x56254be70150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.090+0000 7f6576046b80 1 --2- 172.21.15.156:0/1079270123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be76810 0x56254be76be0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.090+0000 7f6576046b80 1 --2- 172.21.15.156:0/1079270123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be4b1c0 0x56254be77120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.090+0000 7f6576046b80 1 -- 172.21.15.156:0/1079270123 >> 172.21.15.156:0/1079270123 conn(0x56254be49d60 msgr2=0x56254be4a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.090+0000 7f6576046b80 1 -- 172.21.15.156:0/1079270123 shutdown_connections 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.090+0000 7f6576046b80 1 -- 172.21.15.156:0/1079270123 wait complete. 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.090+0000 7f6576046b80 1 -- start start 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.090+0000 7f6576046b80 1 Processor -- start 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.091+0000 7f6576046b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 0x56254be6e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.091+0000 7f6576046b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be6fce0 0x56254be6e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.091+0000 7f6576046b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 0x56254be931f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.091+0000 7f6576046b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254bc228d0 con 0x56254be4b1c0 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.091+0000 7f6576046b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254be9b2b0 con 0x56254be6fce0 2023-11-28T11:10:00.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.091+0000 7f6576046b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56254be6ae90 con 0x56254be76810 2023-11-28T11:10:00.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.091+0000 7f6576046b80 10 client.0 _ll_register_callbacks cb 0x56254bc44240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:00.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.091+0000 7f65637fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 0x56254be6e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.091+0000 7f65637fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 0x56254be6e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34778/0 (socket says 172.21.15.156:34778) 2023-11-28T11:10:00.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.091+0000 7f65637fe700 1 -- 172.21.15.156:0/3202448327 learned_addr learned my addr 172.21.15.156:0/3202448327 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:00.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f65637fe700 1 -- 172.21.15.156:0/3202448327 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 msgr2=0x56254be931f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:00.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f65637fe700 1 --2- 172.21.15.156:0/3202448327 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 0x56254be931f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f65637fe700 1 -- 172.21.15.156:0/3202448327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be6fce0 msgr2=0x56254be6e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:00.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f6562ffd700 1 --2- 172.21.15.156:0/3202448327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be6fce0 0x56254be6e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f65637fe700 1 --2- 172.21.15.156:0/3202448327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be6fce0 0x56254be6e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f65637fe700 1 -- 172.21.15.156:0/3202448327 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655800f040 con 0x56254be4b1c0 2023-11-28T11:10:00.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f65637fe700 1 --2- 172.21.15.156:0/3202448327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 0x56254be6e380 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x56254be4e380 tx=0x7f6558013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2da282f55a2b0e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f65427fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f65427fc700 1 -- 172.21.15.156:0/3202448327 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558028070 con 0x56254be4b1c0 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f65427fc700 1 -- 172.21.15.156:0/3202448327 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6558023080 con 0x56254be4b1c0 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f6576046b80 1 -- 172.21.15.156:0/3202448327 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56254bd6c140 con 0x56254be4b1c0 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f65427fc700 1 -- 172.21.15.156:0/3202448327 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6558023240 con 0x56254be4b1c0 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f6576046b80 10 client.5685 fetch_fsmap learned FSMap version 23 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f6576046b80 1 -- 172.21.15.156:0/3202448327 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x56254bbccd90 con 0x56254be4b1c0 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f65427fc700 1 -- 172.21.15.156:0/3202448327 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56254bd6c140 con 0x56254be4b1c0 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f65427fc700 1 -- 172.21.15.156:0/3202448327 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6558023960 con 0x56254be4b1c0 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f6576046b80 10 client.5685 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f6576046b80 10 client.5685 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f6576046b80 20 client.5685 populate_metadata read hostname 'smithi156' 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f6576046b80 1 --2- 172.21.15.156:0/3202448327 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56254be66b40 0x56254be93be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f6576046b80 4 client.5685 mds_command: new command op to 5547 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:10:00.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.092+0000 7f6576046b80 1 -- 172.21.15.156:0/3202448327 --> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56254bc22f10 con 0x56254be66b40 2023-11-28T11:10:00.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.093+0000 7f6562ffd700 1 --2- 172.21.15.156:0/3202448327 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56254be66b40 0x56254be93be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.093+0000 7f65427fc700 10 client.5685 ms_handle_connect on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:00.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.093+0000 7f6562ffd700 1 --2- 172.21.15.156:0/3202448327 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56254be66b40 0x56254be93be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97fa977b29f663a3 server_cookie=1a61be7d24bba57e in_seq=0 out_seq=0 2023-11-28T11:10:00.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f65427fc700 1 -- 172.21.15.156:0/3202448327 <== mds.0 v2:172.21.15.186:6833/3832587641 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x56254bc22f10 con 0x56254be66b40 2023-11-28T11:10:00.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f65427fc700 10 client.5685 handle_command_reply: tid=0 2023-11-28T11:10:00.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f6576046b80 1 client.5685 shutdown 2023-11-28T11:10:00.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f6576046b80 1 -- 172.21.15.156:0/3202448327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 msgr2=0x56254be6e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f6576046b80 1 --2- 172.21.15.156:0/3202448327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 0x56254be6e380 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x56254be4e380 tx=0x7f6558013000 comp rx=0 tx=0).stop 2023-11-28T11:10:00.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f6576046b80 1 -- 172.21.15.156:0/3202448327 shutdown_connections 2023-11-28T11:10:00.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f6576046b80 1 --2- 172.21.15.156:0/3202448327 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56254be66b40 0x56254be93be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f6576046b80 1 --2- 172.21.15.156:0/3202448327 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 0x56254be931f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f65427fc700 0 client.5685 ms_handle_reset on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:00.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f6576046b80 1 --2- 172.21.15.156:0/3202448327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 0x56254be6e380 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f6576046b80 1 --2- 172.21.15.156:0/3202448327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be6fce0 0x56254be6e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f6576046b80 1 -- 172.21.15.156:0/3202448327 >> 172.21.15.156:0/3202448327 conn(0x56254be49d60 msgr2=0x56254be4a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:00.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f6576046b80 1 -- 172.21.15.156:0/3202448327 shutdown_connections 2023-11-28T11:10:00.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f6576046b80 1 -- 172.21.15.156:0/3202448327 wait complete. 2023-11-28T11:10:00.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.094+0000 7f6576046b80 20 client.5685 trim_cache size 0 max 16384 2023-11-28T11:10:00.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.112+0000 7f6576046b80 1 Processor -- start 2023-11-28T11:10:00.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.112+0000 7f6576046b80 1 -- start start 2023-11-28T11:10:00.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.112+0000 7f6576046b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 0x56254be4cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.112+0000 7f6576046b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be6fce0 0x56254bef47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.112+0000 7f6576046b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 0x56254be45f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.112+0000 7f6576046b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254be944f0 con 0x56254be4b1c0 2023-11-28T11:10:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.112+0000 7f6576046b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254be772c0 con 0x56254be6fce0 2023-11-28T11:10:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.112+0000 7f6576046b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56254be42f30 con 0x56254be76810 2023-11-28T11:10:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f65637fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 0x56254be4cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f65637fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 0x56254be4cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34786/0 (socket says 172.21.15.156:34786) 2023-11-28T11:10:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f65637fe700 1 -- 172.21.15.156:0/862520634 learned_addr learned my addr 172.21.15.156:0/862520634 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f6562ffd700 1 --2- 172.21.15.156:0/862520634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be6fce0 0x56254bef47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f6563fff700 1 --2- 172.21.15.156:0/862520634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 0x56254be45f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f65637fe700 1 -- 172.21.15.156:0/862520634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 msgr2=0x56254be45f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f65637fe700 1 --2- 172.21.15.156:0/862520634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 0x56254be45f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f65637fe700 1 -- 172.21.15.156:0/862520634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be6fce0 msgr2=0x56254bef47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f65637fe700 1 --2- 172.21.15.156:0/862520634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be6fce0 0x56254bef47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f65637fe700 1 -- 172.21.15.156:0/862520634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56254bd65140 con 0x56254be4b1c0 2023-11-28T11:10:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f65637fe700 1 --2- 172.21.15.156:0/862520634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 0x56254be4cb90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6558007da0 tx=0x7f655801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b37f75ce1da3e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f6549ffb700 1 -- 172.21.15.156:0/862520634 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558023960 con 0x56254be4b1c0 2023-11-28T11:10:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f6549ffb700 1 -- 172.21.15.156:0/862520634 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6558019070 con 0x56254be4b1c0 2023-11-28T11:10:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.113+0000 7f6549ffb700 1 -- 172.21.15.156:0/862520634 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65580232c0 con 0x56254be4b1c0 2023-11-28T11:10:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.114+0000 7f6576046b80 1 -- 172.21.15.156:0/862520634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 msgr2=0x56254be4cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.114+0000 7f6576046b80 1 --2- 172.21.15.156:0/862520634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 0x56254be4cb90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6558007da0 tx=0x7f655801d000 comp rx=0 tx=0).stop 2023-11-28T11:10:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.114+0000 7f6576046b80 1 -- 172.21.15.156:0/862520634 shutdown_connections 2023-11-28T11:10:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.114+0000 7f6576046b80 1 --2- 172.21.15.156:0/862520634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 0x56254be45f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.114+0000 7f6576046b80 1 --2- 172.21.15.156:0/862520634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be4b1c0 0x56254be4cb90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.114+0000 7f6576046b80 1 --2- 172.21.15.156:0/862520634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be6fce0 0x56254bef47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.114+0000 7f6576046b80 1 -- 172.21.15.156:0/862520634 >> 172.21.15.156:0/862520634 conn(0x56254be49d60 msgr2=0x56254be930f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.114+0000 7f6576046b80 1 -- 172.21.15.156:0/862520634 shutdown_connections 2023-11-28T11:10:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.114+0000 7f6576046b80 1 -- 172.21.15.156:0/862520634 wait complete. 2023-11-28T11:10:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.114+0000 7f6576046b80 1 -- start start 2023-11-28T11:10:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.114+0000 7f6576046b80 1 Processor -- start 2023-11-28T11:10:00.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.115+0000 7f6576046b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be4b1c0 0x56254be92520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.115+0000 7f6576046b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be6fce0 0x56254be4a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.115+0000 7f6576046b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 0x56254be4a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.115+0000 7f6576046b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254be46300 con 0x56254be6fce0 2023-11-28T11:10:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.115+0000 7f6576046b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254be91050 con 0x56254be4b1c0 2023-11-28T11:10:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.115+0000 7f6576046b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56254be87c20 con 0x56254be76810 2023-11-28T11:10:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.115+0000 7f6576046b80 10 client.0 _ll_register_callbacks cb 0x56254bc44240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.115+0000 7f6562ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be6fce0 0x56254be4a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.115+0000 7f6562ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be6fce0 0x56254be4a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34794/0 (socket says 172.21.15.156:34794) 2023-11-28T11:10:00.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.115+0000 7f6562ffd700 1 -- 172.21.15.156:0/90674682 learned_addr learned my addr 172.21.15.156:0/90674682 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:00.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.115+0000 7f65637fe700 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be4b1c0 0x56254be92520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.115+0000 7f6563fff700 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 0x56254be4a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6562ffd700 1 -- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 msgr2=0x56254be4a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:00.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6562ffd700 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 0x56254be4a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6562ffd700 1 -- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be4b1c0 msgr2=0x56254be92520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:00.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6562ffd700 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be4b1c0 0x56254be92520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6562ffd700 1 -- 172.21.15.156:0/90674682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56254bbccd90 con 0x56254be6fce0 2023-11-28T11:10:00.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6563fff700 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 0x56254be4a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:00.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6562ffd700 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be6fce0 0x56254be4a130 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f65540554d0 tx=0x7f6554005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73012d8ed8e3d01 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f65427fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:10:00.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f65427fc700 1 -- 172.21.15.156:0/90674682 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554054c90 con 0x56254be6fce0 2023-11-28T11:10:00.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f65427fc700 1 -- 172.21.15.156:0/90674682 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f655405d1a0 con 0x56254be6fce0 2023-11-28T11:10:00.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6576046b80 1 -- 172.21.15.156:0/90674682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56254be76440 con 0x56254be6fce0 2023-11-28T11:10:00.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f65427fc700 1 -- 172.21.15.156:0/90674682 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f655405d360 con 0x56254be6fce0 2023-11-28T11:10:00.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6576046b80 10 client.5691 fetch_fsmap learned FSMap version 23 2023-11-28T11:10:00.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6576046b80 1 -- 172.21.15.156:0/90674682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x56254bbf3200 con 0x56254be6fce0 2023-11-28T11:10:00.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f65427fc700 1 -- 172.21.15.156:0/90674682 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6554070020 con 0x56254be6fce0 2023-11-28T11:10:00.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f65427fc700 1 -- 172.21.15.156:0/90674682 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f655407db20 con 0x56254be6fce0 2023-11-28T11:10:00.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6576046b80 10 client.5691 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T11:10:00.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6576046b80 10 client.5691 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:10:00.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6576046b80 20 client.5691 populate_metadata read hostname 'smithi156' 2023-11-28T11:10:00.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6576046b80 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56254be45450 0x56254be45820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6576046b80 4 client.5691 mds_command: new command op to 5547 tid=0[{"prefix": "status"}] 2023-11-28T11:10:00.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f6576046b80 1 -- 172.21.15.156:0/90674682 --> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56254bd5c160 con 0x56254be45450 2023-11-28T11:10:00.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.116+0000 7f65637fe700 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56254be45450 0x56254be45820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f65427fc700 10 client.5691 ms_handle_connect on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:00.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f65637fe700 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56254be45450 0x56254be45820 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c4375091089703 server_cookie=88d84b06bcec5b4a in_seq=0 out_seq=0 2023-11-28T11:10:00.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f65427fc700 1 -- 172.21.15.156:0/90674682 <== mds.0 v2:172.21.15.186:6833/3832587641 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x56254bd5c160 con 0x56254be45450 2023-11-28T11:10:00.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f65427fc700 10 client.5691 handle_command_reply: tid=0 2023-11-28T11:10:00.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f6576046b80 1 client.5691 shutdown 2023-11-28T11:10:00.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f6576046b80 1 -- 172.21.15.156:0/90674682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be6fce0 msgr2=0x56254be4a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f6576046b80 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be6fce0 0x56254be4a130 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f65540554d0 tx=0x7f6554005040 comp rx=0 tx=0).stop 2023-11-28T11:10:00.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f6576046b80 1 -- 172.21.15.156:0/90674682 shutdown_connections 2023-11-28T11:10:00.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f6576046b80 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56254be45450 0x56254be45820 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f6576046b80 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56254be76810 0x56254be4a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f6576046b80 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56254be4b1c0 0x56254be92520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f65427fc700 0 client.5691 ms_handle_reset on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:00.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f6576046b80 1 --2- 172.21.15.156:0/90674682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254be6fce0 0x56254be4a130 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f6576046b80 1 -- 172.21.15.156:0/90674682 >> 172.21.15.156:0/90674682 conn(0x56254be49d60 msgr2=0x56254be4b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:00.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f6576046b80 1 -- 172.21.15.156:0/90674682 shutdown_connections 2023-11-28T11:10:00.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f6576046b80 1 -- 172.21.15.156:0/90674682 wait complete. 2023-11-28T11:10:00.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.117+0000 7f6576046b80 20 client.5691 trim_cache size 0 max 16384 2023-11-28T11:10:00.134 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:10:00.134 INFO:teuthology.orchestra.run.smithi156.stdout: "cluster_fsid": "351da896-8dbf-473b-9472-c3454d9368bf", 2023-11-28T11:10:00.134 INFO:teuthology.orchestra.run.smithi156.stdout: "whoami": 0, 2023-11-28T11:10:00.134 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 5547, 2023-11-28T11:10:00.134 INFO:teuthology.orchestra.run.smithi156.stdout: "want_state": "up:active", 2023-11-28T11:10:00.134 INFO:teuthology.orchestra.run.smithi156.stdout: "state": "up:active", 2023-11-28T11:10:00.134 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs", 2023-11-28T11:10:00.135 INFO:teuthology.orchestra.run.smithi156.stdout: "rank_uptime": 5.842148044, 2023-11-28T11:10:00.135 INFO:teuthology.orchestra.run.smithi156.stdout: "mdsmap_epoch": 23, 2023-11-28T11:10:00.135 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch": 49, 2023-11-28T11:10:00.135 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch_barrier": 49, 2023-11-28T11:10:00.135 INFO:teuthology.orchestra.run.smithi156.stdout: "uptime": 11.132731732 2023-11-28T11:10:00.135 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:10:00.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.118+0000 7f6569ebd700 1 -- 172.21.15.156:0/4000188397 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65500682f0 msgr2=0x7f655006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.118+0000 7f6569ebd700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65500682f0 0x7f655006a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f65580011e0 tx=0x7f6558009040 comp rx=0 tx=0).stop 2023-11-28T11:10:00.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.118+0000 7f6569ebd700 1 -- 172.21.15.156:0/4000188397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65640cbdf0 msgr2=0x7f65640c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.118+0000 7f6569ebd700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65640cbdf0 0x7f65640c70d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f655400c4e0 tx=0x7f655400c8a0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.118+0000 7f6569ebd700 1 -- 172.21.15.156:0/4000188397 shutdown_connections 2023-11-28T11:10:00.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.118+0000 7f6569ebd700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f656415cef0 0x7f6564153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.118+0000 7f6569ebd700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f65500682f0 0x7f655006a7a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.118+0000 7f6569ebd700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f656406ba90 0x7f65640c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.118+0000 7f6569ebd700 1 --2- 172.21.15.156:0/4000188397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65640cbdf0 0x7f65640c70d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.118+0000 7f6569ebd700 1 -- 172.21.15.156:0/4000188397 >> 172.21.15.156:0/4000188397 conn(0x7f65640d0000 msgr2=0x7f65640cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:00.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.118+0000 7f6569ebd700 1 -- 172.21.15.156:0/4000188397 shutdown_connections 2023-11-28T11:10:00.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.118+0000 7f6569ebd700 1 -- 172.21.15.156:0/4000188397 wait complete. 2023-11-28T11:10:00.139 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:10:00.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.260+0000 7fbe365c9700 1 Processor -- start 2023-11-28T11:10:00.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.260+0000 7fbe365c9700 1 -- start start 2023-11-28T11:10:00.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.260+0000 7fbe365c9700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbe30158480 0x7fbe300ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.260+0000 7fbe365c9700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe3006c7a0 0x7fbe300caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.260+0000 7fbe365c9700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe30071cb0 con 0x7fbe300b0750 2023-11-28T11:10:00.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.260+0000 7fbe365c9700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3015bc00 con 0x7fbe30158480 2023-11-28T11:10:00.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.260+0000 7fbe365c9700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300cb0b0 con 0x7fbe3006c7a0 2023-11-28T11:10:00.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.261+0000 7fbe2ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbe30158480 0x7fbe300ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.261+0000 7fbe34b66700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fbe300b0750 0x7fbe300c9320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49680/0 (socket says 172.21.15.156:49680) 2023-11-28T11:10:00.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.261+0000 7fbe34b66700 1 -- 172.21.15.156:0/3459814319 learned_addr learned my addr 172.21.15.156:0/3459814319 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:00.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.261+0000 7fbe2f7fe700 1 --2- 172.21.15.156:0/3459814319 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe3006c7a0 0x7fbe300caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.261+0000 7fbe2ffff700 1 -- 172.21.15.156:0/3459814319 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe3006c7a0 msgr2=0x7fbe300caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:00.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.261+0000 7fbe2ffff700 1 --2- 172.21.15.156:0/3459814319 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe3006c7a0 0x7fbe300caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.261+0000 7fbe2ffff700 1 -- 172.21.15.156:0/3459814319 >> v1:172.21.15.156:6789/0 conn(0x7fbe300b0750 legacy=0x7fbe300c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.261+0000 7fbe2ffff700 1 -- 172.21.15.156:0/3459814319 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3006b9e0 con 0x7fbe30158480 2023-11-28T11:10:00.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.261+0000 7fbe2ffff700 1 --2- 172.21.15.156:0/3459814319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe30158480 0x7fbe300ca540 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009840 tx=0x7fbe18009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33014bb468463a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.261+0000 7fbe2effd700 1 -- 172.21.15.156:0/3459814319 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18011830 con 0x7fbe30158480 2023-11-28T11:10:00.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.261+0000 7fbe2effd700 1 -- 172.21.15.156:0/3459814319 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe18011990 con 0x7fbe30158480 2023-11-28T11:10:00.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.261+0000 7fbe2effd700 1 -- 172.21.15.156:0/3459814319 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18011d20 con 0x7fbe30158480 2023-11-28T11:10:00.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.262+0000 7fbe365c9700 1 -- 172.21.15.156:0/3459814319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe30158480 msgr2=0x7fbe300ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.262+0000 7fbe365c9700 1 --2- 172.21.15.156:0/3459814319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe30158480 0x7fbe300ca540 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009840 tx=0x7fbe18009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:00.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.262+0000 7fbe365c9700 1 -- 172.21.15.156:0/3459814319 shutdown_connections 2023-11-28T11:10:00.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.262+0000 7fbe365c9700 1 --2- 172.21.15.156:0/3459814319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe30158480 0x7fbe300ca540 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.262+0000 7fbe365c9700 1 --2- 172.21.15.156:0/3459814319 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe3006c7a0 0x7fbe300caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.262+0000 7fbe365c9700 1 -- 172.21.15.156:0/3459814319 >> 172.21.15.156:0/3459814319 conn(0x7fbe300cb750 msgr2=0x7fbe300cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:00.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.262+0000 7fbe365c9700 1 -- 172.21.15.156:0/3459814319 shutdown_connections 2023-11-28T11:10:00.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.262+0000 7fbe365c9700 1 -- 172.21.15.156:0/3459814319 wait complete. 2023-11-28T11:10:00.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.263+0000 7fbe365c9700 1 Processor -- start 2023-11-28T11:10:00.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.263+0000 7fbe365c9700 1 -- start start 2023-11-28T11:10:00.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe365c9700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe3006c7a0 0x7fbe300c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe365c9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe300b0750 0x7fbe30144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe365c9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe30158480 0x7fbe300c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe365c9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300cda10 con 0x7fbe300b0750 2023-11-28T11:10:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe365c9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300ce910 con 0x7fbe30158480 2023-11-28T11:10:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe365c9700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300c7980 con 0x7fbe3006c7a0 2023-11-28T11:10:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe2f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe300b0750 0x7fbe30144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe2f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe300b0750 0x7fbe30144fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34808/0 (socket says 172.21.15.156:34808) 2023-11-28T11:10:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe2f7fe700 1 -- 172.21.15.156:0/2000007206 learned_addr learned my addr 172.21.15.156:0/2000007206 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe2ffff700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe3006c7a0 0x7fbe300c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe34b66700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe30158480 0x7fbe300c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe2f7fe700 1 -- 172.21.15.156:0/2000007206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe3006c7a0 msgr2=0x7fbe300c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe2f7fe700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe3006c7a0 0x7fbe300c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe2f7fe700 1 -- 172.21.15.156:0/2000007206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe30158480 msgr2=0x7fbe300c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe2f7fe700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe30158480 0x7fbe300c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.264+0000 7fbe2f7fe700 1 -- 172.21.15.156:0/2000007206 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe180094a0 con 0x7fbe300b0750 2023-11-28T11:10:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.265+0000 7fbe2f7fe700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe300b0750 0x7fbe30144fa0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000c820 tx=0x7fbe2000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.265+0000 7fbe2cff9700 1 -- 172.21.15.156:0/2000007206 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20011800 con 0x7fbe300b0750 2023-11-28T11:10:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.265+0000 7fbe2cff9700 1 -- 172.21.15.156:0/2000007206 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe20011960 con 0x7fbe300b0750 2023-11-28T11:10:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.265+0000 7fbe365c9700 1 -- 172.21.15.156:0/2000007206 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3006bfa0 con 0x7fbe300b0750 2023-11-28T11:10:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.265+0000 7fbe2cff9700 1 -- 172.21.15.156:0/2000007206 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20011cf0 con 0x7fbe300b0750 2023-11-28T11:10:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.265+0000 7fbe365c9700 1 -- 172.21.15.156:0/2000007206 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3006bcc0 con 0x7fbe300b0750 2023-11-28T11:10:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.266+0000 7fbe365c9700 1 -- 172.21.15.156:0/2000007206 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3006c280 con 0x7fbe300b0750 2023-11-28T11:10:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.266+0000 7fbe2cff9700 1 -- 172.21.15.156:0/2000007206 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbe20021070 con 0x7fbe300b0750 2023-11-28T11:10:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.266+0000 7fbe2cff9700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbe10068320 0x7fbe1006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.267+0000 7fbe2cff9700 1 -- 172.21.15.156:0/2000007206 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe20085aa0 con 0x7fbe300b0750 2023-11-28T11:10:00.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.267+0000 7fbe2ffff700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbe10068320 0x7fbe1006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.267+0000 7fbe2ffff700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbe10068320 0x7fbe1006a7d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbe180090b0 tx=0x7fbe18008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.271+0000 7fbe2cff9700 1 -- 172.21.15.156:0/2000007206 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbe20052300 con 0x7fbe300b0750 2023-11-28T11:10:00.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.496+0000 7fbe365c9700 1 -- 172.21.15.156:0/2000007206 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbe301509e0 con 0x7fbe300b0750 2023-11-28T11:10:00.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.497+0000 7fbe2cff9700 1 -- 172.21.15.156:0/2000007206 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbe2001f020 con 0x7fbe300b0750 2023-11-28T11:10:00.498 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:10:00.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.499+0000 7fbe365c9700 1 -- 172.21.15.156:0/2000007206 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbe10068320 msgr2=0x7fbe1006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.499+0000 7fbe365c9700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbe10068320 0x7fbe1006a7d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbe180090b0 tx=0x7fbe18008030 comp rx=0 tx=0).stop 2023-11-28T11:10:00.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.499+0000 7fbe365c9700 1 -- 172.21.15.156:0/2000007206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe300b0750 msgr2=0x7fbe30144fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.499+0000 7fbe365c9700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe300b0750 0x7fbe30144fa0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000c820 tx=0x7fbe2000cbe0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.499+0000 7fbe365c9700 1 -- 172.21.15.156:0/2000007206 shutdown_connections 2023-11-28T11:10:00.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.499+0000 7fbe365c9700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe30158480 0x7fbe300c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.499+0000 7fbe365c9700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbe10068320 0x7fbe1006a7d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.499+0000 7fbe365c9700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe3006c7a0 0x7fbe300c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.499+0000 7fbe365c9700 1 --2- 172.21.15.156:0/2000007206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe300b0750 0x7fbe30144fa0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.499+0000 7fbe365c9700 1 -- 172.21.15.156:0/2000007206 >> 172.21.15.156:0/2000007206 conn(0x7fbe300cb750 msgr2=0x7fbe30159ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:00.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.499+0000 7fbe365c9700 1 -- 172.21.15.156:0/2000007206 shutdown_connections 2023-11-28T11:10:00.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:00.499+0000 7fbe365c9700 1 -- 172.21.15.156:0/2000007206 wait complete. 2023-11-28T11:10:00.512 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:10:00.513 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:10:00.513 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-28T11:10:00.513 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:10:00.513 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:00.513 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:10:00.513 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:10:00.513 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:10:00.513 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:10:00.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:00.513 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-28T11:10:00.531 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:10:00.531 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:10:00.531 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:10:00.531 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:00.531 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42282sec preferred_lft 42282sec 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::c260:d776:a692:b0cc/64 scope link noprefixroute 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 8e:07:21:2f:e8:e5 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::a871:90ff:feb9:397d/64 scope link 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 8e:07:21:2f:e8:e5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::8c07:21ff:fe2f:e8e5/64 scope link 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi156.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether d2:39:d6:e9:98:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::d039:d6ff:fee9:985b/64 scope link 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:00.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:00.535 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:10:00.597 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-28T11:10:00.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:00.598 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-28T11:10:00.658 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-28T11:10:00.658 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:10:00.658 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:10:00.658 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:10:00.659 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:10:00.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:00.659 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T11:10:00.659 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T11:10:00.660 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:10:00.660 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:10:00.660 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:10:00.660 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T11:10:00.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:00.764 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T11:10:00.781 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:10:00.781 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:10:00.781 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:10:00.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:00.782 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-28T11:10:00.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:10:00.847 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:00.905 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:10:00.905 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:10:00.905 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:00.961 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:10:00.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:00.961 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:10:01.047 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:10:01.047 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:10:01.047 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:10:01.056 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:10:01.056 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:10:01.056 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:10:01.056 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:10:01.056 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:10:01.086 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:10:01.086 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:10:01.086 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:10:01.087 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:10:01.087 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:10:01.087 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T11:10:01.087 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T11:10:01.087 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=351da896-8dbf-473b-9472-c3454d9368bf" will pass to kernel 2023-11-28T11:10:01.106 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:10:01.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:01.106 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:01.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:01.135 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:10:01.221 INFO:teuthology.orchestra.run.smithi156.stdout:client5703 2023-11-28T11:10:01.222 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:10:01.222 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:01.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:01.239 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2023-11-28T11:10:01.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:01.305 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2023-11-28T11:10:01.364 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:10:01.364 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T11:10:01.365 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T11:10:01.365 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T11:10:01.365 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2023-11-28T11:10:01.365 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T11:10:01.470 INFO:teuthology.orchestra.run.smithi156.stdout:1099511627777 2023-11-28T11:10:01.659 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:01.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.819+0000 7fb12e613700 1 Processor -- start 2023-11-28T11:10:01.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.820+0000 7fb12e613700 1 -- start start 2023-11-28T11:10:01.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.820+0000 7fb12e613700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb128071040 0x7fb128071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.820+0000 7fb12e613700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb128160540 0x7fb1281609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.820+0000 7fb12e613700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb128071950 con 0x7fb12806dee0 2023-11-28T11:10:01.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.820+0000 7fb12e613700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12815d060 con 0x7fb128160540 2023-11-28T11:10:01.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.820+0000 7fb12e613700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12815d340 con 0x7fb128071040 2023-11-28T11:10:01.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.820+0000 7fb12cbb0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb128160540 0x7fb1281609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.821+0000 7fb1277fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb12806dee0 0x7fb12806e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49698/0 (socket says 172.21.15.156:49698) 2023-11-28T11:10:01.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.821+0000 7fb1277fe700 1 -- 172.21.15.156:0/2663312353 learned_addr learned my addr 172.21.15.156:0/2663312353 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:01.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.821+0000 7fb127fff700 1 --2- 172.21.15.156:0/2663312353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb128071040 0x7fb128071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.821+0000 7fb12cbb0700 1 -- 172.21.15.156:0/2663312353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb128071040 msgr2=0x7fb128071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:01.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.821+0000 7fb12cbb0700 1 --2- 172.21.15.156:0/2663312353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb128071040 0x7fb128071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.821+0000 7fb12cbb0700 1 -- 172.21.15.156:0/2663312353 >> v1:172.21.15.156:6789/0 conn(0x7fb12806dee0 legacy=0x7fb12806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:01.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.821+0000 7fb126ffd700 1 -- 172.21.15.156:0/2663312353 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627704768 0 0) 0x7fb128071950 con 0x7fb12806dee0 2023-11-28T11:10:01.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.821+0000 7fb12cbb0700 1 -- 172.21.15.156:0/2663312353 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12815d620 con 0x7fb128160540 2023-11-28T11:10:01.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.821+0000 7fb12cbb0700 1 --2- 172.21.15.156:0/2663312353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb128160540 0x7fb1281609c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb120009d90 tx=0x7fb12000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f6b0051d63c981 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:01.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.821+0000 7fb126ffd700 1 -- 172.21.15.156:0/2663312353 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12000e070 con 0x7fb128160540 2023-11-28T11:10:01.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.821+0000 7fb126ffd700 1 -- 172.21.15.156:0/2663312353 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12000bbd0 con 0x7fb128160540 2023-11-28T11:10:01.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.821+0000 7fb126ffd700 1 -- 172.21.15.156:0/2663312353 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb120012430 con 0x7fb128160540 2023-11-28T11:10:01.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.822+0000 7fb12e613700 1 -- 172.21.15.156:0/2663312353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb128160540 msgr2=0x7fb1281609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:01.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.822+0000 7fb12e613700 1 --2- 172.21.15.156:0/2663312353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb128160540 0x7fb1281609c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb120009d90 tx=0x7fb12000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:01.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.822+0000 7fb12e613700 1 -- 172.21.15.156:0/2663312353 shutdown_connections 2023-11-28T11:10:01.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.822+0000 7fb12e613700 1 --2- 172.21.15.156:0/2663312353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb128160540 0x7fb1281609c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.822+0000 7fb12e613700 1 --2- 172.21.15.156:0/2663312353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb128071040 0x7fb128071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.822+0000 7fb12e613700 1 -- 172.21.15.156:0/2663312353 >> 172.21.15.156:0/2663312353 conn(0x7fb1280d0000 msgr2=0x7fb1280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:01.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.822+0000 7fb12e613700 1 -- 172.21.15.156:0/2663312353 shutdown_connections 2023-11-28T11:10:01.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.822+0000 7fb12e613700 1 -- 172.21.15.156:0/2663312353 wait complete. 2023-11-28T11:10:01.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.823+0000 7fb12e613700 1 Processor -- start 2023-11-28T11:10:01.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.823+0000 7fb12e613700 1 -- start start 2023-11-28T11:10:01.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb12e613700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb12806dee0 0x7fb1280b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb12e613700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb128071040 0x7fb1280ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb12e613700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb128160540 0x7fb1280ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb12e613700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1281f3570 con 0x7fb12806dee0 2023-11-28T11:10:01.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb12e613700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1280c69b0 con 0x7fb128071040 2023-11-28T11:10:01.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb12e613700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1281f63b0 con 0x7fb128160540 2023-11-28T11:10:01.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb127fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb12806dee0 0x7fb1280b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb127fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb12806dee0 0x7fb1280b5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34824/0 (socket says 172.21.15.156:34824) 2023-11-28T11:10:01.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb127fff700 1 -- 172.21.15.156:0/3279880130 learned_addr learned my addr 172.21.15.156:0/3279880130 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:01.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb1277fe700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb128071040 0x7fb1280ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb12cbb0700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb128160540 0x7fb1280ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb127fff700 1 -- 172.21.15.156:0/3279880130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb128160540 msgr2=0x7fb1280ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:01.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb127fff700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb128160540 0x7fb1280ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb127fff700 1 -- 172.21.15.156:0/3279880130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb128071040 msgr2=0x7fb1280ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:01.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb127fff700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb128071040 0x7fb1280ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb127fff700 1 -- 172.21.15.156:0/3279880130 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb120009a70 con 0x7fb12806dee0 2023-11-28T11:10:01.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb127fff700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb12806dee0 0x7fb1280b5920 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb11800c7f0 tx=0x7fb11800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:01.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb124ff9700 1 -- 172.21.15.156:0/3279880130 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118011800 con 0x7fb12806dee0 2023-11-28T11:10:01.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb124ff9700 1 -- 172.21.15.156:0/3279880130 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb118011960 con 0x7fb12806dee0 2023-11-28T11:10:01.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb12e613700 1 -- 172.21.15.156:0/3279880130 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12815ccd0 con 0x7fb12806dee0 2023-11-28T11:10:01.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.824+0000 7fb124ff9700 1 -- 172.21.15.156:0/3279880130 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118011cf0 con 0x7fb12806dee0 2023-11-28T11:10:01.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.825+0000 7fb12e613700 1 -- 172.21.15.156:0/3279880130 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12815d900 con 0x7fb12806dee0 2023-11-28T11:10:01.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.825+0000 7fb12e613700 1 -- 172.21.15.156:0/3279880130 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12806a270 con 0x7fb12806dee0 2023-11-28T11:10:01.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.826+0000 7fb124ff9700 1 -- 172.21.15.156:0/3279880130 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb118021070 con 0x7fb12806dee0 2023-11-28T11:10:01.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.826+0000 7fb124ff9700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb1140682f0 0x7fb11406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.826+0000 7fb1277fe700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb1140682f0 0x7fb11406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.826+0000 7fb124ff9700 1 -- 172.21.15.156:0/3279880130 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb118085aa0 con 0x7fb12806dee0 2023-11-28T11:10:01.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.827+0000 7fb1277fe700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb1140682f0 0x7fb11406a7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb128157150 tx=0x7fb11c007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:01.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:01.831+0000 7fb124ff9700 1 -- 172.21.15.156:0/3279880130 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb1180522c0 con 0x7fb12806dee0 2023-11-28T11:10:02.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.058+0000 7fb12e613700 1 -- 172.21.15.156:0/3279880130 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb12806c360 con 0x7fb12806dee0 2023-11-28T11:10:02.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.058+0000 7fb124ff9700 1 -- 172.21.15.156:0/3279880130 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fb11801f020 con 0x7fb12806dee0 2023-11-28T11:10:02.060 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:02.060 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/365552946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":365552946},{"type":"v1","addr":"172.21.15.186:6834","nonce":365552946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1918325985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":1918325985},{"type":"v1","addr":"172.21.15.156:6837","nonce":1918325985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/2889641252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2889641252},{"type":"v1","addr":"172.21.15.156:6836","nonce":2889641252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:54.274258+0000","modified":"2023-11-28T11:09:57.285496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3832587641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3832587641},{"type":"v1","addr":"172.21.15.186:6835","nonce":3832587641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:10:02.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.061+0000 7fb12e613700 1 -- 172.21.15.156:0/3279880130 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb1140682f0 msgr2=0x7fb11406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.061+0000 7fb12e613700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb1140682f0 0x7fb11406a7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb128157150 tx=0x7fb11c007c90 comp rx=0 tx=0).stop 2023-11-28T11:10:02.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.061+0000 7fb12e613700 1 -- 172.21.15.156:0/3279880130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb12806dee0 msgr2=0x7fb1280b5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.061+0000 7fb12e613700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb12806dee0 0x7fb1280b5920 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb11800c7f0 tx=0x7fb11800cbb0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.061+0000 7fb12e613700 1 -- 172.21.15.156:0/3279880130 shutdown_connections 2023-11-28T11:10:02.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.061+0000 7fb12e613700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb1140682f0 0x7fb11406a7a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.061+0000 7fb12e613700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb128160540 0x7fb1280ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.061+0000 7fb12e613700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb12806dee0 0x7fb1280b5920 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.061+0000 7fb12e613700 1 --2- 172.21.15.156:0/3279880130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb128071040 0x7fb1280ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.061+0000 7fb12e613700 1 -- 172.21.15.156:0/3279880130 >> 172.21.15.156:0/3279880130 conn(0x7fb1280d0000 msgr2=0x7fb1280cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.061+0000 7fb12e613700 1 -- 172.21.15.156:0/3279880130 shutdown_connections 2023-11-28T11:10:02.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.061+0000 7fb12e613700 1 -- 172.21.15.156:0/3279880130 wait complete. 2023-11-28T11:10:02.064 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T11:10:02.074 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:10:02.234 INFO:teuthology.orchestra.run.smithi186.stdout:{"message":"","return_code":0} 2023-11-28T11:10:02.234 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:10:02.234 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:10:02.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.278+0000 7f922c69c080 1 Processor -- start 2023-11-28T11:10:02.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.278+0000 7f922c69c080 1 -- start start 2023-11-28T11:10:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f922c69c080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d13df48c30 0x55d13e01a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f922c69c080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df4b8c0 0x55d13df793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f922c69c080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d13df78fe0 0x55d13e01c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f922c69c080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13de82fe0 con 0x55d13df4b8c0 2023-11-28T11:10:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f922c69c080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13df78820 con 0x55d13df48c30 2023-11-28T11:10:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f922c69c080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d13df798f0 con 0x55d13df78fe0 2023-11-28T11:10:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9221f9e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df4b8c0 0x55d13df793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9221f9e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df4b8c0 0x55d13df793b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34840/0 (socket says 172.21.15.156:34840) 2023-11-28T11:10:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9221f9e700 1 -- 172.21.15.156:0/2570137497 learned_addr learned my addr 172.21.15.156:0/2570137497 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f922279f700 1 --2- 172.21.15.156:0/2570137497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d13df48c30 0x55d13e01a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9222fa0700 1 --2- 172.21.15.156:0/2570137497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d13df78fe0 0x55d13e01c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9221f9e700 1 -- 172.21.15.156:0/2570137497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d13df78fe0 msgr2=0x55d13e01c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9221f9e700 1 --2- 172.21.15.156:0/2570137497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d13df78fe0 0x55d13e01c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9221f9e700 1 -- 172.21.15.156:0/2570137497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d13df48c30 msgr2=0x55d13e01a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9221f9e700 1 --2- 172.21.15.156:0/2570137497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d13df48c30 0x55d13e01a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9221f9e700 1 -- 172.21.15.156:0/2570137497 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c00a100 con 0x55d13df4b8c0 2023-11-28T11:10:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9222fa0700 1 --2- 172.21.15.156:0/2570137497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d13df78fe0 0x55d13e01c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9221f9e700 1 --2- 172.21.15.156:0/2570137497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df4b8c0 0x55d13df793b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f921800b2e0 tx=0x7f921800b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c923f941903072 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9220f9c700 1 -- 172.21.15.156:0/2570137497 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9218014070 con 0x55d13df4b8c0 2023-11-28T11:10:02.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9220f9c700 1 -- 172.21.15.156:0/2570137497 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921800be40 con 0x55d13df4b8c0 2023-11-28T11:10:02.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.279+0000 7f9220f9c700 1 -- 172.21.15.156:0/2570137497 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921800f5f0 con 0x55d13df4b8c0 2023-11-28T11:10:02.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.280+0000 7f922c69c080 1 -- 172.21.15.156:0/2570137497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df4b8c0 msgr2=0x55d13df793b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.280+0000 7f922c69c080 1 --2- 172.21.15.156:0/2570137497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df4b8c0 0x55d13df793b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f921800b2e0 tx=0x7f921800b5f0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.280+0000 7f922c69c080 1 -- 172.21.15.156:0/2570137497 shutdown_connections 2023-11-28T11:10:02.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.280+0000 7f922c69c080 1 --2- 172.21.15.156:0/2570137497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d13df78fe0 0x55d13e01c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.280+0000 7f922c69c080 1 --2- 172.21.15.156:0/2570137497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d13df48c30 0x55d13e01a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.280+0000 7f922c69c080 1 --2- 172.21.15.156:0/2570137497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df4b8c0 0x55d13df793b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.280+0000 7f922c69c080 1 -- 172.21.15.156:0/2570137497 >> 172.21.15.156:0/2570137497 conn(0x55d13df753e0 msgr2=0x55d13df77030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.280+0000 7f922c69c080 1 -- 172.21.15.156:0/2570137497 shutdown_connections 2023-11-28T11:10:02.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.280+0000 7f922c69c080 1 -- 172.21.15.156:0/2570137497 wait complete. 2023-11-28T11:10:02.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.280+0000 7f922c69c080 1 Processor -- start 2023-11-28T11:10:02.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.280+0000 7f922c69c080 1 -- start start 2023-11-28T11:10:02.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922c69c080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df48c30 0x55d13e014eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922c69c080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d13df4b8c0 0x55d13e01b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922c69c080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d13df78fe0 0x55d13e01bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922c69c080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13e017b30 con 0x55d13df48c30 2023-11-28T11:10:02.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922c69c080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13df781e0 con 0x55d13df4b8c0 2023-11-28T11:10:02.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922c69c080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d13df8f9d0 con 0x55d13df78fe0 2023-11-28T11:10:02.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922279f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df48c30 0x55d13e014eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922279f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df48c30 0x55d13e014eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34844/0 (socket says 172.21.15.156:34844) 2023-11-28T11:10:02.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922279f700 1 -- 172.21.15.156:0/2613453229 learned_addr learned my addr 172.21.15.156:0/2613453229 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f9222fa0700 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d13df78fe0 0x55d13e01bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f9221f9e700 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d13df4b8c0 0x55d13e01b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922279f700 1 -- 172.21.15.156:0/2613453229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d13df78fe0 msgr2=0x55d13e01bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922279f700 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d13df78fe0 0x55d13e01bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922279f700 1 -- 172.21.15.156:0/2613453229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d13df4b8c0 msgr2=0x55d13e01b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922279f700 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d13df4b8c0 0x55d13e01b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.281+0000 7f922279f700 1 -- 172.21.15.156:0/2613453229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9218001c10 con 0x55d13df48c30 2023-11-28T11:10:02.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.282+0000 7f922279f700 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df48c30 0x55d13e014eb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9214009f00 tx=0x7f92140086e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.282+0000 7f92137fe700 1 -- 172.21.15.156:0/2613453229 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214011040 con 0x55d13df48c30 2023-11-28T11:10:02.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.282+0000 7f92137fe700 1 -- 172.21.15.156:0/2613453229 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9214015620 con 0x55d13df48c30 2023-11-28T11:10:02.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.282+0000 7f922c69c080 1 -- 172.21.15.156:0/2613453229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d13de9b120 con 0x55d13df48c30 2023-11-28T11:10:02.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.282+0000 7f922c69c080 1 -- 172.21.15.156:0/2613453229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d13de8e830 con 0x55d13df48c30 2023-11-28T11:10:02.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.282+0000 7f92137fe700 1 -- 172.21.15.156:0/2613453229 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92140159b0 con 0x55d13df48c30 2023-11-28T11:10:02.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.283+0000 7f92137fe700 1 -- 172.21.15.156:0/2613453229 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f921401c030 con 0x55d13df48c30 2023-11-28T11:10:02.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.284+0000 7f92137fe700 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9204066c10 0x7f92040690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.284+0000 7f92137fe700 1 -- 172.21.15.156:0/2613453229 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9214084090 con 0x55d13df48c30 2023-11-28T11:10:02.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.284+0000 7f9221f9e700 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9204066c10 0x7f92040690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.284+0000 7f922c69c080 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x55d13df483b0 0x55d13e018a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.284+0000 7f922c69c080 1 -- 172.21.15.156:0/2613453229 --> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50) v8 -- 0x55d13e018f60 con 0x55d13df483b0 2023-11-28T11:10:02.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.284+0000 7f9222fa0700 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x55d13df483b0 0x55d13e018a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.284+0000 7f9221f9e700 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9204066c10 0x7f92040690c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f921800b2e0 tx=0x7f9218011040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.284+0000 7f9222fa0700 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x55d13df483b0 0x55d13e018a20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.285+0000 7f9222fa0700 1 -- 172.21.15.156:0/2613453229 <== osd.1 v2:172.21.15.156:6820/103566 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f921c00fa60 con 0x55d13df483b0 2023-11-28T11:10:02.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.285+0000 7f922c69c080 1 -- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x55d13df483b0 msgr2=0x55d13e018a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.285+0000 7f922c69c080 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x55d13df483b0 0x55d13e018a20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.285+0000 7f922c69c080 1 -- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9204066c10 msgr2=0x7f92040690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.285+0000 7f922c69c080 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9204066c10 0x7f92040690c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f921800b2e0 tx=0x7f9218011040 comp rx=0 tx=0).stop 2023-11-28T11:10:02.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.285+0000 7f922c69c080 1 -- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df48c30 msgr2=0x55d13e014eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.285+0000 7f922c69c080 1 --2- 172.21.15.156:0/2613453229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d13df48c30 0x55d13e014eb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9214009f00 tx=0x7f92140086e0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.285+0000 7f9222fa0700 1 -- 172.21.15.156:0/2613453229 reap_dead start 2023-11-28T11:10:02.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.285+0000 7f922c69c080 1 -- 172.21.15.156:0/2613453229 shutdown_connections 2023-11-28T11:10:02.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.285+0000 7f922c69c080 1 -- 172.21.15.156:0/2613453229 >> 172.21.15.156:0/2613453229 conn(0x55d13df753e0 msgr2=0x55d13e016150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.285+0000 7f922c69c080 1 -- 172.21.15.156:0/2613453229 shutdown_connections 2023-11-28T11:10:02.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.285+0000 7f922c69c080 1 -- 172.21.15.156:0/2613453229 wait complete. 2023-11-28T11:10:02.297 INFO:teuthology.orchestra.run.smithi156.stdout:Yalphaparent_a 2023-11-28T11:10:02.298 DEBUG:teuthology.orchestra.run.smithi156:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:10:02.409 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:10:02.409 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 1099511627777, 2023-11-28T11:10:02.409 INFO:teuthology.orchestra.run.smithi156.stdout: "ancestors": [ 2023-11-28T11:10:02.409 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:02.409 INFO:teuthology.orchestra.run.smithi156.stdout: "dirino": 1099511627776, 2023-11-28T11:10:02.409 INFO:teuthology.orchestra.run.smithi156.stdout: "dname": "alpha", 2023-11-28T11:10:02.409 INFO:teuthology.orchestra.run.smithi156.stdout: "version": 2 2023-11-28T11:10:02.409 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:02.409 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:02.409 INFO:teuthology.orchestra.run.smithi156.stdout: "dirino": 1, 2023-11-28T11:10:02.410 INFO:teuthology.orchestra.run.smithi156.stdout: "dname": "parent_a", 2023-11-28T11:10:02.410 INFO:teuthology.orchestra.run.smithi156.stdout: "version": 4 2023-11-28T11:10:02.410 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:02.410 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2023-11-28T11:10:02.410 INFO:teuthology.orchestra.run.smithi156.stdout: "pool": 8, 2023-11-28T11:10:02.410 INFO:teuthology.orchestra.run.smithi156.stdout: "old_pools": [] 2023-11-28T11:10:02.410 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:10:02.410 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:02.411 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2023-11-28T11:10:02.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.455+0000 7f54b7f80080 1 Processor -- start 2023-11-28T11:10:02.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.455+0000 7f54b7f80080 1 -- start start 2023-11-28T11:10:02.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54b7f80080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a6c9779e0 0x563a6c95c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54b7f80080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563a6ca05460 0x563a6c96bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54b7f80080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca09df0 0x563a6c96c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54b7f80080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6c867fe0 con 0x563a6ca09df0 2023-11-28T11:10:02.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54b7f80080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ca09a40 con 0x563a6c9779e0 2023-11-28T11:10:02.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54b7f80080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a6c974460 con 0x563a6ca05460 2023-11-28T11:10:02.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ae884700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca09df0 0x563a6c96c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ae083700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a6c9779e0 0x563a6c95c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ae884700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca09df0 0x563a6c96c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34864/0 (socket says 172.21.15.156:34864) 2023-11-28T11:10:02.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ae884700 1 -- 172.21.15.156:0/3679223747 learned_addr learned my addr 172.21.15.156:0/3679223747 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ad882700 1 --2- 172.21.15.156:0/3679223747 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563a6ca05460 0x563a6c96bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ae884700 1 -- 172.21.15.156:0/3679223747 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563a6ca05460 msgr2=0x563a6c96bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ae884700 1 --2- 172.21.15.156:0/3679223747 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563a6ca05460 0x563a6c96bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ae884700 1 -- 172.21.15.156:0/3679223747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a6c9779e0 msgr2=0x563a6c95c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ae884700 1 --2- 172.21.15.156:0/3679223747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a6c9779e0 0x563a6c95c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ae884700 1 -- 172.21.15.156:0/3679223747 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a800a1c0 con 0x563a6ca09df0 2023-11-28T11:10:02.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ae884700 1 --2- 172.21.15.156:0/3679223747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca09df0 0x563a6c96c0b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f54a8013040 tx=0x7f54a800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d855a38c5b9ac99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ac880700 1 -- 172.21.15.156:0/3679223747 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a800e710 con 0x563a6ca09df0 2023-11-28T11:10:02.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ac880700 1 -- 172.21.15.156:0/3679223747 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a800e9e0 con 0x563a6ca09df0 2023-11-28T11:10:02.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.456+0000 7f54ac880700 1 -- 172.21.15.156:0/3679223747 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a800ed90 con 0x563a6ca09df0 2023-11-28T11:10:02.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.457+0000 7f54b7f80080 1 -- 172.21.15.156:0/3679223747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca09df0 msgr2=0x563a6c96c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.457+0000 7f54b7f80080 1 --2- 172.21.15.156:0/3679223747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca09df0 0x563a6c96c0b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f54a8013040 tx=0x7f54a800e000 comp rx=0 tx=0).stop 2023-11-28T11:10:02.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.457+0000 7f54b7f80080 1 -- 172.21.15.156:0/3679223747 shutdown_connections 2023-11-28T11:10:02.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.457+0000 7f54b7f80080 1 --2- 172.21.15.156:0/3679223747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca09df0 0x563a6c96c0b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.457+0000 7f54b7f80080 1 --2- 172.21.15.156:0/3679223747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a6c9779e0 0x563a6c95c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.457+0000 7f54b7f80080 1 --2- 172.21.15.156:0/3679223747 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563a6ca05460 0x563a6c96bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.457+0000 7f54b7f80080 1 -- 172.21.15.156:0/3679223747 >> 172.21.15.156:0/3679223747 conn(0x563a6c959b70 msgr2=0x563a6c95aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.457+0000 7f54b7f80080 1 -- 172.21.15.156:0/3679223747 shutdown_connections 2023-11-28T11:10:02.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.457+0000 7f54b7f80080 1 -- 172.21.15.156:0/3679223747 wait complete. 2023-11-28T11:10:02.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.458+0000 7f54b7f80080 1 Processor -- start 2023-11-28T11:10:02.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.458+0000 7f54b7f80080 1 -- start start 2023-11-28T11:10:02.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.458+0000 7f54b7f80080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a6c9779e0 0x563a6ca01ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.458+0000 7f54b7f80080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca05460 0x563a6ca02530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.458+0000 7f54b7f80080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563a6ca09df0 0x563a6c96f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.458+0000 7f54b7f80080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ca9e420 con 0x563a6ca05460 2023-11-28T11:10:02.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.458+0000 7f54b7f80080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ca021e0 con 0x563a6c9779e0 2023-11-28T11:10:02.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.458+0000 7f54b7f80080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a6ca047c0 con 0x563a6ca09df0 2023-11-28T11:10:02.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.458+0000 7f54ad882700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca05460 0x563a6ca02530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.458+0000 7f54ad882700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca05460 0x563a6ca02530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34868/0 (socket says 172.21.15.156:34868) 2023-11-28T11:10:02.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.458+0000 7f54ad882700 1 -- 172.21.15.156:0/956645091 learned_addr learned my addr 172.21.15.156:0/956645091 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f54ae884700 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563a6ca09df0 0x563a6c96f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f54ae083700 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a6c9779e0 0x563a6ca01ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f54ad882700 1 -- 172.21.15.156:0/956645091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563a6ca09df0 msgr2=0x563a6c96f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f54ad882700 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563a6ca09df0 0x563a6c96f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f54ad882700 1 -- 172.21.15.156:0/956645091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a6c9779e0 msgr2=0x563a6ca01ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f54ad882700 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a6c9779e0 0x563a6ca01ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f54ad882700 1 -- 172.21.15.156:0/956645091 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a6c89a4f0 con 0x563a6ca05460 2023-11-28T11:10:02.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f54ae083700 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a6c9779e0 0x563a6ca01ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:02.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f54ad882700 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca05460 0x563a6ca02530 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f54a0002cd0 tx=0x7f54a000c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f549f7fe700 1 -- 172.21.15.156:0/956645091 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0016030 con 0x563a6ca05460 2023-11-28T11:10:02.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f549f7fe700 1 -- 172.21.15.156:0/956645091 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a0008070 con 0x563a6ca05460 2023-11-28T11:10:02.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f54b7f80080 1 -- 172.21.15.156:0/956645091 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563a6c880120 con 0x563a6ca05460 2023-11-28T11:10:02.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f549f7fe700 1 -- 172.21.15.156:0/956645091 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0001bf0 con 0x563a6ca05460 2023-11-28T11:10:02.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.459+0000 7f54b7f80080 1 -- 172.21.15.156:0/956645091 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a6c873830 con 0x563a6ca05460 2023-11-28T11:10:02.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.460+0000 7f549f7fe700 1 -- 172.21.15.156:0/956645091 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f54a0001d50 con 0x563a6ca05460 2023-11-28T11:10:02.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.461+0000 7f549f7fe700 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5490066c10 0x7f54900690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.461+0000 7f549f7fe700 1 -- 172.21.15.156:0/956645091 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f54a0017070 con 0x563a6ca05460 2023-11-28T11:10:02.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.461+0000 7f54ae083700 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5490066c10 0x7f54900690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.461+0000 7f54b7f80080 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x563a6ca9cb10 0x563a6c96f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.461+0000 7f54b7f80080 1 -- 172.21.15.156:0/956645091 --> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51) v8 -- 0x563a6c96fbb0 con 0x563a6ca9cb10 2023-11-28T11:10:02.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.461+0000 7f54ae884700 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x563a6ca9cb10 0x563a6c96f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.461+0000 7f54ae083700 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5490066c10 0x7f54900690c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005fd0 tx=0x7f54a400c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.461+0000 7f54ae884700 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x563a6ca9cb10 0x563a6c96f670 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.462+0000 7f54ae884700 1 -- 172.21.15.156:0/956645091 <== osd.1 v2:172.21.15.156:6820/103566 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v51'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f54a8017040 con 0x563a6ca9cb10 2023-11-28T11:10:02.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.463+0000 7f54b7f80080 1 -- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x563a6ca9cb10 msgr2=0x563a6c96f670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.463+0000 7f54b7f80080 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x563a6ca9cb10 0x563a6c96f670 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.463+0000 7f54b7f80080 1 -- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5490066c10 msgr2=0x7f54900690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.463+0000 7f54b7f80080 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5490066c10 0x7f54900690c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005fd0 tx=0x7f54a400c040 comp rx=0 tx=0).stop 2023-11-28T11:10:02.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.463+0000 7f54b7f80080 1 -- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca05460 msgr2=0x563a6ca02530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.463+0000 7f54b7f80080 1 --2- 172.21.15.156:0/956645091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a6ca05460 0x563a6ca02530 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f54a0002cd0 tx=0x7f54a000c250 comp rx=0 tx=0).stop 2023-11-28T11:10:02.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.463+0000 7f54ae884700 1 -- 172.21.15.156:0/956645091 reap_dead start 2023-11-28T11:10:02.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.463+0000 7f54b7f80080 1 -- 172.21.15.156:0/956645091 shutdown_connections 2023-11-28T11:10:02.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.463+0000 7f54b7f80080 1 -- 172.21.15.156:0/956645091 >> 172.21.15.156:0/956645091 conn(0x563a6c959b70 msgr2=0x563a6ca083f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.463+0000 7f54b7f80080 1 -- 172.21.15.156:0/956645091 shutdown_connections 2023-11-28T11:10:02.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.463+0000 7f54b7f80080 1 -- 172.21.15.156:0/956645091 wait complete. 2023-11-28T11:10:02.471 DEBUG:teuthology.orchestra.run.smithi156:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-28T11:10:02.473 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-28T11:10:02.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.599+0000 7f04c5642700 1 Processor -- start 2023-11-28T11:10:02.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.599+0000 7f04c5642700 1 -- start start 2023-11-28T11:10:02.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04c5642700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f04c00af060 0x7f04c0070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04c5642700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f04c00af430 0x7f04c006d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04c5642700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04c006a270 0x7f04c006a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04c5642700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c006dee0 con 0x7f04c00af430 2023-11-28T11:10:02.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04c5642700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c006abd0 con 0x7f04c00af060 2023-11-28T11:10:02.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04c5642700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00713c0 con 0x7f04c006a270 2023-11-28T11:10:02.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04be7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f04c00af430 0x7f04c006d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04be7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f04c00af430 0x7f04c006d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34876/0 (socket says 172.21.15.156:34876) 2023-11-28T11:10:02.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04be7fc700 1 -- 172.21.15.156:0/645719274 learned_addr learned my addr 172.21.15.156:0/645719274 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04beffd700 1 --2- 172.21.15.156:0/645719274 >> v2:172.21.15.186:3300/0 conn(0x7f04c00af060 0x7f04c0070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04bf7fe700 1 --2- 172.21.15.156:0/645719274 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04c006a270 0x7f04c006a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04be7fc700 1 -- 172.21.15.156:0/645719274 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04c006a270 msgr2=0x7f04c006a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04be7fc700 1 --2- 172.21.15.156:0/645719274 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04c006a270 0x7f04c006a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04be7fc700 1 -- 172.21.15.156:0/645719274 >> v2:172.21.15.186:3300/0 conn(0x7f04c00af060 msgr2=0x7f04c0070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04be7fc700 1 --2- 172.21.15.156:0/645719274 >> v2:172.21.15.186:3300/0 conn(0x7f04c00af060 0x7f04c0070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.600+0000 7f04be7fc700 1 -- 172.21.15.156:0/645719274 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c00716a0 con 0x7f04c00af430 2023-11-28T11:10:02.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.601+0000 7f04be7fc700 1 --2- 172.21.15.156:0/645719274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c00af430 0x7f04c006d9a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f04c00725d0 tx=0x7f04b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e7e01d9ab69eb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.601+0000 7f04bdffb700 1 -- 172.21.15.156:0/645719274 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4011830 con 0x7f04c00af430 2023-11-28T11:10:02.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.601+0000 7f04bdffb700 1 -- 172.21.15.156:0/645719274 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b4011990 con 0x7f04c00af430 2023-11-28T11:10:02.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.601+0000 7f04bdffb700 1 -- 172.21.15.156:0/645719274 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4011d20 con 0x7f04c00af430 2023-11-28T11:10:02.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.601+0000 7f04c5642700 1 -- 172.21.15.156:0/645719274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c00af430 msgr2=0x7f04c006d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.601+0000 7f04c5642700 1 --2- 172.21.15.156:0/645719274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c00af430 0x7f04c006d9a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f04c00725d0 tx=0x7f04b4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.601+0000 7f04c5642700 1 -- 172.21.15.156:0/645719274 shutdown_connections 2023-11-28T11:10:02.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.601+0000 7f04c5642700 1 --2- 172.21.15.156:0/645719274 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04c006a270 0x7f04c006a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.601+0000 7f04c5642700 1 --2- 172.21.15.156:0/645719274 >> v2:172.21.15.186:3300/0 conn(0x7f04c00af060 0x7f04c0070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.601+0000 7f04c5642700 1 --2- 172.21.15.156:0/645719274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c00af430 0x7f04c006d9a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.601+0000 7f04c5642700 1 -- 172.21.15.156:0/645719274 >> 172.21.15.156:0/645719274 conn(0x7f04c006b380 msgr2=0x7f04c006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.602+0000 7f04c5642700 1 -- 172.21.15.156:0/645719274 shutdown_connections 2023-11-28T11:10:02.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.602+0000 7f04c5642700 1 -- 172.21.15.156:0/645719274 wait complete. 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.603+0000 7f04c5642700 1 Processor -- start 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.603+0000 7f04c5642700 1 -- start start 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04c5642700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04c006a270 0x7f04c015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04c5642700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c00af060 0x7f04c014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04c5642700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c00af430 0x7f04c015f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04c5642700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c014c7c0 con 0x7f04c00af430 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04c5642700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c014c320 con 0x7f04c00af060 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04c5642700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c0160100 con 0x7f04c006a270 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04bf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c00af430 0x7f04c015f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04bf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c00af430 0x7f04c015f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34886/0 (socket says 172.21.15.156:34886) 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04bf7fe700 1 -- 172.21.15.156:0/1646377796 learned_addr learned my addr 172.21.15.156:0/1646377796 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04beffd700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04c006a270 0x7f04c015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04be7fc700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c00af060 0x7f04c014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04bf7fe700 1 -- 172.21.15.156:0/1646377796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04c006a270 msgr2=0x7f04c015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04bf7fe700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04c006a270 0x7f04c015dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04bf7fe700 1 -- 172.21.15.156:0/1646377796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c00af060 msgr2=0x7f04c014bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04bf7fe700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c00af060 0x7f04c014bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04bf7fe700 1 -- 172.21.15.156:0/1646377796 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b4009550 con 0x7f04c00af430 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04bf7fe700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c00af430 0x7f04c015f080 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f04b800efc0 tx=0x7f04b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.604+0000 7f04a77fe700 1 -- 172.21.15.156:0/1646377796 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8009ba0 con 0x7f04c00af430 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.605+0000 7f04a77fe700 1 -- 172.21.15.156:0/1646377796 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b800f040 con 0x7f04c00af430 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.605+0000 7f04c5642700 1 -- 172.21.15.156:0/1646377796 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c0071c60 con 0x7f04c00af430 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.605+0000 7f04a77fe700 1 -- 172.21.15.156:0/1646377796 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8014480 con 0x7f04c00af430 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.605+0000 7f04c5642700 1 -- 172.21.15.156:0/1646377796 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c0071980 con 0x7f04c00af430 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.606+0000 7f04a77fe700 1 -- 172.21.15.156:0/1646377796 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f04b80145e0 con 0x7f04c00af430 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.606+0000 7f04a77fe700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04ac0682f0 0x7f04ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.607+0000 7f04a77fe700 1 -- 172.21.15.156:0/1646377796 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f04b80866e0 con 0x7f04c00af430 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.607+0000 7f04beffd700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04ac0682f0 0x7f04ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.607+0000 7f04beffd700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04ac0682f0 0x7f04ac06a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009a50 tx=0x7f04b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.608+0000 7f04d17cbb80 1 Processor -- start 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.609+0000 7f04d17cbb80 1 -- start start 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.609+0000 7f04d17cbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691caa8e0 0x561691c80af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.609+0000 7f04d17cbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 0x561691caacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.609+0000 7f04d17cbb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691cab1f0 0x561691cc6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.609+0000 7f04d17cbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561691b6ac90 con 0x561691ca8200 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.609+0000 7f04d17cbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561691bd9420 con 0x561691caa8e0 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.609+0000 7f04d17cbb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561691bd2950 con 0x561691cab1f0 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.609+0000 7f04be7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 0x561691caacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.609+0000 7f04be7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 0x561691caacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34892/0 (socket says 172.21.15.156:34892) 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.609+0000 7f04be7fc700 1 -- 172.21.15.156:0/3706309168 learned_addr learned my addr 172.21.15.156:0/3706309168 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.610+0000 7f04bf7fe700 1 --2- 172.21.15.156:0/3706309168 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691cab1f0 0x561691cc6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.610+0000 7f04beffd700 1 --2- 172.21.15.156:0/3706309168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691caa8e0 0x561691c80af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.610+0000 7f04be7fc700 1 -- 172.21.15.156:0/3706309168 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691cab1f0 msgr2=0x561691cc6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.610+0000 7f04be7fc700 1 --2- 172.21.15.156:0/3706309168 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691cab1f0 0x561691cc6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.610+0000 7f04be7fc700 1 -- 172.21.15.156:0/3706309168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691caa8e0 msgr2=0x561691c80af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.610+0000 7f04be7fc700 1 --2- 172.21.15.156:0/3706309168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691caa8e0 0x561691c80af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.610+0000 7f04be7fc700 1 -- 172.21.15.156:0/3706309168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561691a546b0 con 0x561691ca8200 2023-11-28T11:10:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.610+0000 7f04be7fc700 1 --2- 172.21.15.156:0/3706309168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 0x561691caacb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f04b4014040 tx=0x7f04b4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dd5fcc3c05fa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.610+0000 7f04a4ff9700 1 -- 172.21.15.156:0/3706309168 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b40116b0 con 0x561691ca8200 2023-11-28T11:10:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.610+0000 7f04a4ff9700 1 -- 172.21.15.156:0/3706309168 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b4011b00 con 0x561691ca8200 2023-11-28T11:10:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.610+0000 7f04a4ff9700 1 -- 172.21.15.156:0/3706309168 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4011830 con 0x561691ca8200 2023-11-28T11:10:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.611+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3706309168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 msgr2=0x561691caacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.611+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/3706309168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 0x561691caacb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f04b4014040 tx=0x7f04b4010000 comp rx=0 tx=0).stop 2023-11-28T11:10:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.611+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3706309168 shutdown_connections 2023-11-28T11:10:02.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.611+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/3706309168 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691cab1f0 0x561691cc6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.611+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/3706309168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691caa8e0 0x561691c80af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.611+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/3706309168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 0x561691caacb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.611+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3706309168 >> 172.21.15.156:0/3706309168 conn(0x561691c7de30 msgr2=0x561691c7e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.611+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3706309168 shutdown_connections 2023-11-28T11:10:02.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.611+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3706309168 wait complete. 2023-11-28T11:10:02.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.612+0000 7f04d17cbb80 1 -- start start 2023-11-28T11:10:02.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.612+0000 7f04d17cbb80 1 Processor -- start 2023-11-28T11:10:02.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.612+0000 7f04d17cbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691ca8200 0x561691cadc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.612+0000 7f04d17cbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691caa8e0 0x561691caca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.612+0000 7f04d17cbb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691cab1f0 0x561691cacfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.612+0000 7f04d17cbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561691a569a0 con 0x561691caa8e0 2023-11-28T11:10:02.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.612+0000 7f04d17cbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561691cac650 con 0x561691ca8200 2023-11-28T11:10:02.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.612+0000 7f04d17cbb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561691ca80b0 con 0x561691cab1f0 2023-11-28T11:10:02.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04d17cbb80 10 client.0 _ll_register_callbacks cb 0x561691a78310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:02.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04be7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691caa8e0 0x561691caca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04be7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691caa8e0 0x561691caca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34898/0 (socket says 172.21.15.156:34898) 2023-11-28T11:10:02.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04be7fc700 1 -- 172.21.15.156:0/3713830292 learned_addr learned my addr 172.21.15.156:0/3713830292 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04beffd700 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691ca8200 0x561691cadc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04bf7fe700 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691cab1f0 0x561691cacfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04be7fc700 1 -- 172.21.15.156:0/3713830292 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691cab1f0 msgr2=0x561691cacfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04be7fc700 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691cab1f0 0x561691cacfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04be7fc700 1 -- 172.21.15.156:0/3713830292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691ca8200 msgr2=0x561691cadc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04be7fc700 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691ca8200 0x561691cadc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04be7fc700 1 -- 172.21.15.156:0/3713830292 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c00716a0 con 0x561691caa8e0 2023-11-28T11:10:02.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04be7fc700 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691caa8e0 0x561691caca70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f04c006c720 tx=0x7f04b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b86d1349207069f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f049dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:10:02.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f049dffb700 1 -- 172.21.15.156:0/3713830292 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4011d90 con 0x561691caa8e0 2023-11-28T11:10:02.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f049dffb700 1 -- 172.21.15.156:0/3713830292 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f04b402dc60 con 0x561691caa8e0 2023-11-28T11:10:02.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3713830292 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561691ba0210 con 0x561691caa8e0 2023-11-28T11:10:02.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f049dffb700 1 -- 172.21.15.156:0/3713830292 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b402de20 con 0x561691caa8e0 2023-11-28T11:10:02.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04d17cbb80 10 client.5745 fetch_fsmap learned FSMap version 23 2023-11-28T11:10:02.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3713830292 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x561691a00e60 con 0x561691caa8e0 2023-11-28T11:10:02.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.613+0000 7f049dffb700 1 -- 172.21.15.156:0/3713830292 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561691ba0210 con 0x561691caa8e0 2023-11-28T11:10:02.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.614+0000 7f049dffb700 1 -- 172.21.15.156:0/3713830292 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f04b4004070 con 0x561691caa8e0 2023-11-28T11:10:02.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.614+0000 7f04d17cbb80 10 client.5745 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T11:10:02.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.614+0000 7f04d17cbb80 10 client.5745 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:10:02.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.614+0000 7f04d17cbb80 20 client.5745 populate_metadata read hostname 'smithi156' 2023-11-28T11:10:02.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.614+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x561691cc9da0 0x561691ca1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.614+0000 7f04d17cbb80 4 client.5745 mds_command: new command op to 5547 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:10:02.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.614+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3713830292 --> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561691a56fe0 con 0x561691cc9da0 2023-11-28T11:10:02.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.614+0000 7f04beffd700 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x561691cc9da0 0x561691ca1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.614+0000 7f049dffb700 10 client.5745 ms_handle_connect on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:02.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.614+0000 7f04beffd700 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x561691cc9da0 0x561691ca1f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a670bbe4ccbef577 server_cookie=bd1961962649082a in_seq=0 out_seq=0 2023-11-28T11:10:02.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f049dffb700 1 -- 172.21.15.156:0/3713830292 <== mds.0 v2:172.21.15.186:6833/3832587641 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x561691a56fe0 con 0x561691cc9da0 2023-11-28T11:10:02.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f049dffb700 10 client.5745 handle_command_reply: tid=0 2023-11-28T11:10:02.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f04d17cbb80 1 client.5745 shutdown 2023-11-28T11:10:02.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3713830292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691caa8e0 msgr2=0x561691caca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691caa8e0 0x561691caca70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f04c006c720 tx=0x7f04b40047c0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3713830292 shutdown_connections 2023-11-28T11:10:02.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x561691cc9da0 0x561691ca1f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691cab1f0 0x561691cacfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f049dffb700 0 client.5745 ms_handle_reset on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:02.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691ca8200 0x561691cadc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/3713830292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691caa8e0 0x561691caca70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3713830292 >> 172.21.15.156:0/3713830292 conn(0x561691c7de30 msgr2=0x561691c80a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3713830292 shutdown_connections 2023-11-28T11:10:02.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f04d17cbb80 1 -- 172.21.15.156:0/3713830292 wait complete. 2023-11-28T11:10:02.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.615+0000 7f04d17cbb80 20 client.5745 trim_cache size 0 max 16384 2023-11-28T11:10:02.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.622+0000 7fb4b431a700 1 Processor -- start 2023-11-28T11:10:02.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.622+0000 7fb4b431a700 1 -- start start 2023-11-28T11:10:02.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b431a700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb4ac0b6840 0x7fb4ac15d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b431a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4ac0d0bb0 0x7fb4ac0d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b431a700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4ac0b79c0 con 0x7fb4ac15cc80 2023-11-28T11:10:02.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b431a700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4ac0b4610 con 0x7fb4ac0b6840 2023-11-28T11:10:02.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b431a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac0d1570 con 0x7fb4ac0d0bb0 2023-11-28T11:10:02.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b18b5700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb4ac0b6840 0x7fb4ac15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b28b7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4ac0d0bb0 0x7fb4ac0d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b18b5700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb4ac0b6840 0x7fb4ac15d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:45920/0 (socket says 172.21.15.156:45920) 2023-11-28T11:10:02.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b20b6700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fb4ac15cc80 0x7fb4ac0b3f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:60142/0 (socket says 172.21.15.156:60142) 2023-11-28T11:10:02.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b18b5700 1 -- 172.21.15.156:0/2346514383 learned_addr learned my addr 172.21.15.156:0/2346514383 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b28b7700 1 -- 172.21.15.156:0/2346514383 >> v2:172.21.15.186:3300/0 conn(0x7fb4ac0b6840 msgr2=0x7fb4ac15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b28b7700 1 --2- 172.21.15.156:0/2346514383 >> v2:172.21.15.186:3300/0 conn(0x7fb4ac0b6840 0x7fb4ac15d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b10b4700 1 -- 172.21.15.156:0/2346514383 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048917710 0 0) 0x7fb4ac0b79c0 con 0x7fb4ac15cc80 2023-11-28T11:10:02.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b28b7700 1 -- 172.21.15.156:0/2346514383 >> v1:172.21.15.186:6789/0 conn(0x7fb4ac15cc80 legacy=0x7fb4ac0b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.624+0000 7fb4b28b7700 1 -- 172.21.15.156:0/2346514383 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac0b6db0 con 0x7fb4ac0d0bb0 2023-11-28T11:10:02.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.625+0000 7fb4b28b7700 1 --2- 172.21.15.156:0/2346514383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4ac0d0bb0 0x7fb4ac0d1030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009d90 tx=0x7fb4a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9223e1b69b1c7b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.625+0000 7fb4b10b4700 1 -- 172.21.15.156:0/2346514383 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a400e070 con 0x7fb4ac0d0bb0 2023-11-28T11:10:02.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.625+0000 7fb4b10b4700 1 -- 172.21.15.156:0/2346514383 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a400bb20 con 0x7fb4ac0d0bb0 2023-11-28T11:10:02.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.625+0000 7fb4b10b4700 1 -- 172.21.15.156:0/2346514383 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4012430 con 0x7fb4ac0d0bb0 2023-11-28T11:10:02.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.625+0000 7fb4b431a700 1 -- 172.21.15.156:0/2346514383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4ac0d0bb0 msgr2=0x7fb4ac0d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.625+0000 7fb4b431a700 1 --2- 172.21.15.156:0/2346514383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4ac0d0bb0 0x7fb4ac0d1030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009d90 tx=0x7fb4a400b330 comp rx=0 tx=0).stop 2023-11-28T11:10:02.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.626+0000 7fb4b431a700 1 -- 172.21.15.156:0/2346514383 shutdown_connections 2023-11-28T11:10:02.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.626+0000 7fb4b431a700 1 --2- 172.21.15.156:0/2346514383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4ac0d0bb0 0x7fb4ac0d1030 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.626+0000 7fb4b431a700 1 --2- 172.21.15.156:0/2346514383 >> v2:172.21.15.186:3300/0 conn(0x7fb4ac0b6840 0x7fb4ac15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.626+0000 7fb4b431a700 1 -- 172.21.15.156:0/2346514383 >> 172.21.15.156:0/2346514383 conn(0x7fb4ac09c360 msgr2=0x7fb4ac0d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.626+0000 7fb4b431a700 1 -- 172.21.15.156:0/2346514383 shutdown_connections 2023-11-28T11:10:02.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.626+0000 7fb4b431a700 1 -- 172.21.15.156:0/2346514383 wait complete. 2023-11-28T11:10:02.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.628+0000 7fb4b431a700 1 Processor -- start 2023-11-28T11:10:02.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.628+0000 7fb4b431a700 1 -- start start 2023-11-28T11:10:02.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b431a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4ac0b6840 0x7fb4ac14e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b431a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4ac0d0bb0 0x7fb4ac15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b431a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4ac15cc80 0x7fb4ac0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b431a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac14ea70 con 0x7fb4ac0b6840 2023-11-28T11:10:02.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b431a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac15fa40 con 0x7fb4ac0d0bb0 2023-11-28T11:10:02.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b431a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac15e9a0 con 0x7fb4ac15cc80 2023-11-28T11:10:02.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b20b6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4ac0b6840 0x7fb4ac14e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b20b6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4ac0b6840 0x7fb4ac14e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34910/0 (socket says 172.21.15.156:34910) 2023-11-28T11:10:02.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b20b6700 1 -- 172.21.15.156:0/4012278286 learned_addr learned my addr 172.21.15.156:0/4012278286 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b18b5700 1 --2- 172.21.15.156:0/4012278286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4ac0d0bb0 0x7fb4ac15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b28b7700 1 --2- 172.21.15.156:0/4012278286 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4ac15cc80 0x7fb4ac0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b20b6700 1 -- 172.21.15.156:0/4012278286 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4ac15cc80 msgr2=0x7fb4ac0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b20b6700 1 --2- 172.21.15.156:0/4012278286 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4ac15cc80 0x7fb4ac0ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b20b6700 1 -- 172.21.15.156:0/4012278286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4ac0d0bb0 msgr2=0x7fb4ac15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b20b6700 1 --2- 172.21.15.156:0/4012278286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4ac0d0bb0 0x7fb4ac15e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b20b6700 1 -- 172.21.15.156:0/4012278286 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a4009a70 con 0x7fb4ac0b6840 2023-11-28T11:10:02.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b28b7700 1 --2- 172.21.15.156:0/4012278286 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4ac15cc80 0x7fb4ac0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:02.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.629+0000 7fb4b20b6700 1 --2- 172.21.15.156:0/4012278286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4ac0b6840 0x7fb4ac14e530 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800c470 tx=0x7fb4a800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.630+0000 7fb4a2ffd700 1 -- 172.21.15.156:0/4012278286 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8004cd0 con 0x7fb4ac0b6840 2023-11-28T11:10:02.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.630+0000 7fb4a2ffd700 1 -- 172.21.15.156:0/4012278286 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a8004e30 con 0x7fb4ac0b6840 2023-11-28T11:10:02.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.630+0000 7fb4b431a700 1 -- 172.21.15.156:0/4012278286 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ac0b7370 con 0x7fb4ac0b6840 2023-11-28T11:10:02.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.630+0000 7fb4a2ffd700 1 -- 172.21.15.156:0/4012278286 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8007310 con 0x7fb4ac0b6840 2023-11-28T11:10:02.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.630+0000 7fb4b431a700 1 -- 172.21.15.156:0/4012278286 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ac0b7090 con 0x7fb4ac0b6840 2023-11-28T11:10:02.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.630+0000 7fb4b431a700 1 -- 172.21.15.156:0/4012278286 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fb4ac0b7650 con 0x7fb4ac0b6840 2023-11-28T11:10:02.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.631+0000 7fb4a2ffd700 1 -- 172.21.15.156:0/4012278286 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb4a801a020 con 0x7fb4ac0b6840 2023-11-28T11:10:02.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.632+0000 7fb4a2ffd700 1 --2- 172.21.15.156:0/4012278286 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb4900682f0 0x7fb49006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.632+0000 7fb4a2ffd700 1 -- 172.21.15.156:0/4012278286 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb4a8086180 con 0x7fb4ac0b6840 2023-11-28T11:10:02.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.632+0000 7fb4b18b5700 1 --2- 172.21.15.156:0/4012278286 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb4900682f0 0x7fb49006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.632+0000 7fb4a2ffd700 1 -- 172.21.15.156:0/4012278286 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+504 (secure 0 0 0) 0x7fb4a8086510 con 0x7fb4ac0b6840 2023-11-28T11:10:02.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.632+0000 7fb4b431a700 1 -- 172.21.15.156:0/4012278286 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fb4ac15f890 con 0x7fb4ac0b6840 2023-11-28T11:10:02.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.632+0000 7fb4b18b5700 1 --2- 172.21.15.156:0/4012278286 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb4900682f0 0x7fb49006a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0a0de0 tx=0x7fb49c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.632+0000 7fb4a2ffd700 1 -- 172.21.15.156:0/4012278286 <== mon.0 v2:172.21.15.156:3300/0 7 ==== log(2 entries from seq 520 at 2023-11-28T11:10:01.301089+0000) v1 ==== 601+0+0 (secure 0 0 0) 0x7fb4a801a2b0 con 0x7fb4ac0b6840 2023-11-28T11:10:02.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.635+0000 7f04d17cbb80 1 Processor -- start 2023-11-28T11:10:02.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.636+0000 7f04d17cbb80 1 -- start start 2023-11-28T11:10:02.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.636+0000 7f04d17cbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 0x561691c7e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.636+0000 7f04d17cbb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691caa8e0 0x561691c80830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.636+0000 7f04d17cbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691cab1f0 0x561691ca9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.636+0000 7f04d17cbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561691c7c7e0 con 0x561691ca8200 2023-11-28T11:10:02.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.636+0000 7f04d17cbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561691c7bb10 con 0x561691cab1f0 2023-11-28T11:10:02.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.636+0000 7f04d17cbb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561691c7ebc0 con 0x561691caa8e0 2023-11-28T11:10:02.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.636+0000 7f04beffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 0x561691c7e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.636+0000 7f04beffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 0x561691c7e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34918/0 (socket says 172.21.15.156:34918) 2023-11-28T11:10:02.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.636+0000 7f04beffd700 1 -- 172.21.15.156:0/637953970 learned_addr learned my addr 172.21.15.156:0/637953970 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.636+0000 7f04be7fc700 1 --2- 172.21.15.156:0/637953970 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691caa8e0 0x561691c80830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.636+0000 7f04bf7fe700 1 --2- 172.21.15.156:0/637953970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691cab1f0 0x561691ca9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.637+0000 7f04beffd700 1 -- 172.21.15.156:0/637953970 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691caa8e0 msgr2=0x561691c80830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.637+0000 7f04beffd700 1 --2- 172.21.15.156:0/637953970 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691caa8e0 0x561691c80830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.637+0000 7f04beffd700 1 -- 172.21.15.156:0/637953970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691cab1f0 msgr2=0x561691ca9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.637+0000 7f04beffd700 1 --2- 172.21.15.156:0/637953970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691cab1f0 0x561691ca9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.637+0000 7f04beffd700 1 -- 172.21.15.156:0/637953970 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561691b99210 con 0x561691ca8200 2023-11-28T11:10:02.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.637+0000 7f04bf7fe700 1 --2- 172.21.15.156:0/637953970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691cab1f0 0x561691ca9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:02.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.637+0000 7f04beffd700 1 --2- 172.21.15.156:0/637953970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 0x561691c7e340 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009550 tx=0x7f04b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fb5d7db383963e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.637+0000 7f04a57fa700 1 -- 172.21.15.156:0/637953970 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0028040 con 0x561691ca8200 2023-11-28T11:10:02.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.637+0000 7f04a57fa700 1 -- 172.21.15.156:0/637953970 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b002a2c0 con 0x561691ca8200 2023-11-28T11:10:02.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.637+0000 7f04a57fa700 1 -- 172.21.15.156:0/637953970 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0015db0 con 0x561691ca8200 2023-11-28T11:10:02.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.638+0000 7f04d17cbb80 1 -- 172.21.15.156:0/637953970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 msgr2=0x561691c7e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.638+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/637953970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 0x561691c7e340 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009550 tx=0x7f04b0017000 comp rx=0 tx=0).stop 2023-11-28T11:10:02.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.638+0000 7f04d17cbb80 1 -- 172.21.15.156:0/637953970 shutdown_connections 2023-11-28T11:10:02.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.638+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/637953970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691cab1f0 0x561691ca9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.638+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/637953970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691ca8200 0x561691c7e340 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.638+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/637953970 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691caa8e0 0x561691c80830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.638+0000 7f04d17cbb80 1 -- 172.21.15.156:0/637953970 >> 172.21.15.156:0/637953970 conn(0x561691c7de30 msgr2=0x561691c7f950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.638+0000 7f04d17cbb80 1 -- 172.21.15.156:0/637953970 shutdown_connections 2023-11-28T11:10:02.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.638+0000 7f04d17cbb80 1 -- 172.21.15.156:0/637953970 wait complete. 2023-11-28T11:10:02.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.638+0000 7f04d17cbb80 1 -- start start 2023-11-28T11:10:02.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.638+0000 7f04d17cbb80 1 Processor -- start 2023-11-28T11:10:02.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04d17cbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691ca8200 0x561691d2c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04d17cbb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691caa8e0 0x561691ca85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04d17cbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691cab1f0 0x561691cddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04d17cbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561691c79750 con 0x561691cab1f0 2023-11-28T11:10:02.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04d17cbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561691ca8d40 con 0x561691ca8200 2023-11-28T11:10:02.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04d17cbb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561691ca8b10 con 0x561691caa8e0 2023-11-28T11:10:02.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04d17cbb80 10 client.0 _ll_register_callbacks cb 0x561691a78310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:02.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04bf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691cab1f0 0x561691cddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04bf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691cab1f0 0x561691cddfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34928/0 (socket says 172.21.15.156:34928) 2023-11-28T11:10:02.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04bf7fe700 1 -- 172.21.15.156:0/2726067186 learned_addr learned my addr 172.21.15.156:0/2726067186 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04be7fc700 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691caa8e0 0x561691ca85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04beffd700 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691ca8200 0x561691d2c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04bf7fe700 1 -- 172.21.15.156:0/2726067186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691caa8e0 msgr2=0x561691ca85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04bf7fe700 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691caa8e0 0x561691ca85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04bf7fe700 1 -- 172.21.15.156:0/2726067186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691ca8200 msgr2=0x561691d2c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04bf7fe700 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691ca8200 0x561691d2c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04bf7fe700 1 -- 172.21.15.156:0/2726067186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b0011d00 con 0x561691cab1f0 2023-11-28T11:10:02.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04bf7fe700 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691cab1f0 0x561691cddfe0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f04b80568c0 tx=0x7f04b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d19811376a597a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f049dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:10:02.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f049dffb700 1 -- 172.21.15.156:0/2726067186 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b805eea0 con 0x561691cab1f0 2023-11-28T11:10:02.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f049dffb700 1 -- 172.21.15.156:0/2726067186 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f04b80529b0 con 0x561691cab1f0 2023-11-28T11:10:02.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f04d17cbb80 1 -- 172.21.15.156:0/2726067186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561691b90230 con 0x561691cab1f0 2023-11-28T11:10:02.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.640+0000 7f049dffb700 1 -- 172.21.15.156:0/2726067186 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b8052b70 con 0x561691cab1f0 2023-11-28T11:10:02.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f04d17cbb80 10 client.5754 fetch_fsmap learned FSMap version 23 2023-11-28T11:10:02.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f04d17cbb80 1 -- 172.21.15.156:0/2726067186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x561691a272d0 con 0x561691cab1f0 2023-11-28T11:10:02.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f049dffb700 1 -- 172.21.15.156:0/2726067186 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561691b90230 con 0x561691cab1f0 2023-11-28T11:10:02.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f049dffb700 1 -- 172.21.15.156:0/2726067186 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f04b8055b20 con 0x561691cab1f0 2023-11-28T11:10:02.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f04d17cbb80 10 client.5754 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T11:10:02.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f04d17cbb80 10 client.5754 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:10:02.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f04d17cbb80 20 client.5754 populate_metadata read hostname 'smithi156' 2023-11-28T11:10:02.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x561691ca20d0 0x561691ca24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f04d17cbb80 4 client.5754 mds_command: new command op to 5547 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-28T11:10:02.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f04d17cbb80 1 -- 172.21.15.156:0/2726067186 --> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x561691b7a240 con 0x561691ca20d0 2023-11-28T11:10:02.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f04beffd700 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x561691ca20d0 0x561691ca24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f049dffb700 10 client.5754 ms_handle_connect on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:02.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.641+0000 7f04beffd700 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x561691ca20d0 0x561691ca24d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1cdba8d8bf49fb server_cookie=9c6b6ccaafbbbfac in_seq=0 out_seq=0 2023-11-28T11:10:02.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f049dffb700 1 -- 172.21.15.156:0/2726067186 <== mds.0 v2:172.21.15.186:6833/3832587641 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561691b7a240 con 0x561691ca20d0 2023-11-28T11:10:02.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f049dffb700 10 client.5754 handle_command_reply: tid=0 2023-11-28T11:10:02.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f04d17cbb80 1 client.5754 shutdown 2023-11-28T11:10:02.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f04d17cbb80 1 -- 172.21.15.156:0/2726067186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691cab1f0 msgr2=0x561691cddfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691cab1f0 0x561691cddfe0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f04b80568c0 tx=0x7f04b8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f04d17cbb80 1 -- 172.21.15.156:0/2726067186 shutdown_connections 2023-11-28T11:10:02.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561691cab1f0 0x561691cddfe0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561691ca8200 0x561691d2c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x561691ca20d0 0x561691ca24d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f04d17cbb80 1 --2- 172.21.15.156:0/2726067186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561691caa8e0 0x561691ca85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f04d17cbb80 1 -- 172.21.15.156:0/2726067186 >> 172.21.15.156:0/2726067186 conn(0x561691c7de30 msgr2=0x561691c7f950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f049dffb700 0 client.5754 ms_handle_reset on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:02.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f04d17cbb80 1 -- 172.21.15.156:0/2726067186 shutdown_connections 2023-11-28T11:10:02.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f04d17cbb80 1 -- 172.21.15.156:0/2726067186 wait complete. 2023-11-28T11:10:02.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.642+0000 7f04d17cbb80 20 client.5754 trim_cache size 0 max 16384 2023-11-28T11:10:02.651 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:10:02.651 INFO:teuthology.orchestra.run.smithi156.stdout: "return_code": 0, 2023-11-28T11:10:02.651 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_tag": "3b3ef99b-8250-42c5-b5e8-414d052dc90f", 2023-11-28T11:10:02.651 INFO:teuthology.orchestra.run.smithi156.stdout: "mode": "asynchronous" 2023-11-28T11:10:02.651 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:10:02.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.643+0000 7f04c5642700 1 -- 172.21.15.156:0/1646377796 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04ac0682f0 msgr2=0x7f04ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.643+0000 7f04c5642700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04ac0682f0 0x7f04ac06a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009a50 tx=0x7f04b0008040 comp rx=0 tx=0).stop 2023-11-28T11:10:02.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.643+0000 7f04c5642700 1 -- 172.21.15.156:0/1646377796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c00af430 msgr2=0x7f04c015f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.643+0000 7f04c5642700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c00af430 0x7f04c015f080 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f04b800efc0 tx=0x7f04b800c3a0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.643+0000 7f04c5642700 1 -- 172.21.15.156:0/1646377796 shutdown_connections 2023-11-28T11:10:02.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.643+0000 7f04c5642700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c00af430 0x7f04c015f080 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.643+0000 7f04c5642700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f04ac0682f0 0x7f04ac06a7a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.643+0000 7f04c5642700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04c006a270 0x7f04c015dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.643+0000 7f04c5642700 1 --2- 172.21.15.156:0/1646377796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04c00af060 0x7f04c014bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.643+0000 7f04c5642700 1 -- 172.21.15.156:0/1646377796 >> 172.21.15.156:0/1646377796 conn(0x7f04c006b380 msgr2=0x7f04c00d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.643+0000 7f04c5642700 1 -- 172.21.15.156:0/1646377796 shutdown_connections 2023-11-28T11:10:02.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.643+0000 7f04c5642700 1 -- 172.21.15.156:0/1646377796 wait complete. 2023-11-28T11:10:02.653 INFO:tasks.ceph.mds.d.smithi186.stderr:2023-11-28T11:10:02.642+0000 7f78f4e50700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5703=0-4194304@1} caps={5703=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5703 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55cb5f27d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:10:02.656 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T11:10:02.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.776+0000 7ff24ab97700 1 Processor -- start 2023-11-28T11:10:02.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.776+0000 7ff24ab97700 1 -- start start 2023-11-28T11:10:02.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff24ab97700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff244160740 0x7ff2440cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff24ab97700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff244160b10 0x7ff2440cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff24ab97700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff24406b060 0x7ff24406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff24ab97700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2440ca720 con 0x7ff244160b10 2023-11-28T11:10:02.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff24ab97700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2440cc030 con 0x7ff24406b060 2023-11-28T11:10:02.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff24ab97700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440c9e10 con 0x7ff244160740 2023-11-28T11:10:02.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff243fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff244160b10 0x7ff2440cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff243fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff244160b10 0x7ff2440cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34934/0 (socket says 172.21.15.156:34934) 2023-11-28T11:10:02.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff243fff700 1 -- 172.21.15.156:0/2984436811 learned_addr learned my addr 172.21.15.156:0/2984436811 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff249134700 1 --2- 172.21.15.156:0/2984436811 >> v2:172.21.15.186:3300/0 conn(0x7ff24406b060 0x7ff24406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff248933700 1 --2- 172.21.15.156:0/2984436811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff244160740 0x7ff2440cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff243fff700 1 -- 172.21.15.156:0/2984436811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff244160740 msgr2=0x7ff2440cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff243fff700 1 --2- 172.21.15.156:0/2984436811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff244160740 0x7ff2440cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff243fff700 1 -- 172.21.15.156:0/2984436811 >> v2:172.21.15.186:3300/0 conn(0x7ff24406b060 msgr2=0x7ff24406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff243fff700 1 --2- 172.21.15.156:0/2984436811 >> v2:172.21.15.186:3300/0 conn(0x7ff24406b060 0x7ff24406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff243fff700 1 -- 172.21.15.156:0/2984436811 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24406efc0 con 0x7ff244160b10 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff248933700 1 --2- 172.21.15.156:0/2984436811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff244160740 0x7ff2440cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.777+0000 7ff243fff700 1 --2- 172.21.15.156:0/2984436811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff244160b10 0x7ff2440cd390 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff24415e2e0 tx=0x7ff230009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8363b706817d8db2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.778+0000 7ff2437fe700 1 -- 172.21.15.156:0/2984436811 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230011830 con 0x7ff244160b10 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.778+0000 7ff2437fe700 1 -- 172.21.15.156:0/2984436811 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230011990 con 0x7ff244160b10 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.778+0000 7ff2437fe700 1 -- 172.21.15.156:0/2984436811 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230011d20 con 0x7ff244160b10 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.778+0000 7ff24ab97700 1 -- 172.21.15.156:0/2984436811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff244160b10 msgr2=0x7ff2440cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.778+0000 7ff24ab97700 1 --2- 172.21.15.156:0/2984436811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff244160b10 0x7ff2440cd390 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff24415e2e0 tx=0x7ff230009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.778+0000 7ff24ab97700 1 -- 172.21.15.156:0/2984436811 shutdown_connections 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.778+0000 7ff24ab97700 1 --2- 172.21.15.156:0/2984436811 >> v2:172.21.15.186:3300/0 conn(0x7ff24406b060 0x7ff24406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.778+0000 7ff24ab97700 1 --2- 172.21.15.156:0/2984436811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff244160740 0x7ff2440cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.778+0000 7ff24ab97700 1 --2- 172.21.15.156:0/2984436811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff244160b10 0x7ff2440cd390 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.778+0000 7ff24ab97700 1 -- 172.21.15.156:0/2984436811 >> 172.21.15.156:0/2984436811 conn(0x7ff2440b12b0 msgr2=0x7ff2440b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.778+0000 7ff24ab97700 1 -- 172.21.15.156:0/2984436811 shutdown_connections 2023-11-28T11:10:02.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.778+0000 7ff24ab97700 1 -- 172.21.15.156:0/2984436811 wait complete. 2023-11-28T11:10:02.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.779+0000 7ff24ab97700 1 Processor -- start 2023-11-28T11:10:02.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.779+0000 7ff24ab97700 1 -- start start 2023-11-28T11:10:02.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.780+0000 7ff24ab97700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff24406b060 0x7ff2440c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.780+0000 7ff24ab97700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff244160740 0x7ff2440c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.780+0000 7ff24ab97700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff244160b10 0x7ff24406da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.780+0000 7ff24ab97700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440c8050 con 0x7ff244160b10 2023-11-28T11:10:02.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.780+0000 7ff24ab97700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24406df40 con 0x7ff244160740 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.780+0000 7ff24ab97700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24414b0f0 con 0x7ff24406b060 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.780+0000 7ff249134700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff244160b10 0x7ff24406da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.780+0000 7ff249134700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff244160b10 0x7ff24406da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34942/0 (socket says 172.21.15.156:34942) 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.780+0000 7ff249134700 1 -- 172.21.15.156:0/1840683890 learned_addr learned my addr 172.21.15.156:0/1840683890 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff248933700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff24406b060 0x7ff2440c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff243fff700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff244160740 0x7ff2440c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff249134700 1 -- 172.21.15.156:0/1840683890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff24406b060 msgr2=0x7ff2440c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff249134700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff24406b060 0x7ff2440c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff249134700 1 -- 172.21.15.156:0/1840683890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff244160740 msgr2=0x7ff2440c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff249134700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff244160740 0x7ff2440c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff249134700 1 -- 172.21.15.156:0/1840683890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff230009550 con 0x7ff244160b10 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff243fff700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff244160740 0x7ff2440c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:02.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff249134700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff244160b10 0x7ff24406da00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00efc0 tx=0x7ff23c00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff2417fa700 1 -- 172.21.15.156:0/1840683890 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c00cb80 con 0x7ff244160b10 2023-11-28T11:10:02.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff2417fa700 1 -- 172.21.15.156:0/1840683890 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c00f040 con 0x7ff244160b10 2023-11-28T11:10:02.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff24ab97700 1 -- 172.21.15.156:0/1840683890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2440ca3b0 con 0x7ff244160b10 2023-11-28T11:10:02.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff2417fa700 1 -- 172.21.15.156:0/1840683890 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c014480 con 0x7ff244160b10 2023-11-28T11:10:02.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.781+0000 7ff24ab97700 1 -- 172.21.15.156:0/1840683890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2440ca0d0 con 0x7ff244160b10 2023-11-28T11:10:02.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.782+0000 7ff2417fa700 1 -- 172.21.15.156:0/1840683890 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff23c0145e0 con 0x7ff244160b10 2023-11-28T11:10:02.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.783+0000 7ff2417fa700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff2280682f0 0x7ff22806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.783+0000 7ff2417fa700 1 -- 172.21.15.156:0/1840683890 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff23c086620 con 0x7ff244160b10 2023-11-28T11:10:02.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.783+0000 7ff248933700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff2280682f0 0x7ff22806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.783+0000 7ff248933700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff2280682f0 0x7ff22806a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff238006f10 tx=0x7ff238008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.785+0000 7ff256d20b80 1 Processor -- start 2023-11-28T11:10:02.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.785+0000 7ff256d20b80 1 -- start start 2023-11-28T11:10:02.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff256d20b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee25860 0x559adedfba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff256d20b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee23180 0x559adee25c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff256d20b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 0x559adee417c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff256d20b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559adece5c10 con 0x559adee26170 2023-11-28T11:10:02.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff256d20b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aded543a0 con 0x559adee25860 2023-11-28T11:10:02.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff256d20b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x559aded4d8d0 con 0x559adee23180 2023-11-28T11:10:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff249134700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 0x559adee417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff249134700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 0x559adee417c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34950/0 (socket says 172.21.15.156:34950) 2023-11-28T11:10:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff249134700 1 -- 172.21.15.156:0/1631370693 learned_addr learned my addr 172.21.15.156:0/1631370693 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff248933700 1 --2- 172.21.15.156:0/1631370693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee25860 0x559adedfba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff243fff700 1 --2- 172.21.15.156:0/1631370693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee23180 0x559adee25c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff249134700 1 -- 172.21.15.156:0/1631370693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee23180 msgr2=0x559adee25c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff249134700 1 --2- 172.21.15.156:0/1631370693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee23180 0x559adee25c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff249134700 1 -- 172.21.15.156:0/1631370693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee25860 msgr2=0x559adedfba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff249134700 1 --2- 172.21.15.156:0/1631370693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee25860 0x559adedfba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff249134700 1 -- 172.21.15.156:0/1631370693 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559adebcf630 con 0x559adee26170 2023-11-28T11:10:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff249134700 1 --2- 172.21.15.156:0/1631370693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 0x559adee417c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00c4c0 tx=0x7ff23c05ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624a15a3527e572e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff22e7fc700 1 -- 172.21.15.156:0/1631370693 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c05e900 con 0x559adee26170 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff22e7fc700 1 -- 172.21.15.156:0/1631370693 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c057e90 con 0x559adee26170 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.786+0000 7ff22e7fc700 1 -- 172.21.15.156:0/1631370693 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c052b30 con 0x559adee26170 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.787+0000 7ff256d20b80 1 -- 172.21.15.156:0/1631370693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 msgr2=0x559adee417c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.787+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1631370693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 0x559adee417c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00c4c0 tx=0x7ff23c05ec60 comp rx=0 tx=0).stop 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.787+0000 7ff256d20b80 1 -- 172.21.15.156:0/1631370693 shutdown_connections 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.787+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1631370693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 0x559adee417c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.787+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1631370693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee25860 0x559adedfba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.787+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1631370693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee23180 0x559adee25c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.787+0000 7ff256d20b80 1 -- 172.21.15.156:0/1631370693 >> 172.21.15.156:0/1631370693 conn(0x559adedf8db0 msgr2=0x559adedf9740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.787+0000 7ff256d20b80 1 -- 172.21.15.156:0/1631370693 shutdown_connections 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.787+0000 7ff256d20b80 1 -- 172.21.15.156:0/1631370693 wait complete. 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.788+0000 7ff256d20b80 1 -- start start 2023-11-28T11:10:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.788+0000 7ff256d20b80 1 Processor -- start 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.788+0000 7ff256d20b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee23180 0x559adedf4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.788+0000 7ff256d20b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee25860 0x559adedf5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff256d20b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 0x559adee1c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff256d20b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559adebd1920 con 0x559adee26170 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff256d20b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559adee390f0 con 0x559adee25860 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff256d20b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x559adee1d920 con 0x559adee23180 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff256d20b80 10 client.0 _ll_register_callbacks cb 0x559adebf3290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff249134700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 0x559adee1c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff243fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee25860 0x559adedf5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff248933700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee23180 0x559adedf4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff249134700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 0x559adee1c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34956/0 (socket says 172.21.15.156:34956) 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff249134700 1 -- 172.21.15.156:0/1224425834 learned_addr learned my addr 172.21.15.156:0/1224425834 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff243fff700 1 -- 172.21.15.156:0/1224425834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee23180 msgr2=0x559adedf4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff243fff700 1 --2- 172.21.15.156:0/1224425834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee23180 0x559adedf4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff243fff700 1 -- 172.21.15.156:0/1224425834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 msgr2=0x559adee1c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff243fff700 1 --2- 172.21.15.156:0/1224425834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 0x559adee1c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff243fff700 1 -- 172.21.15.156:0/1224425834 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2440ca0d0 con 0x559adee25860 2023-11-28T11:10:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff249134700 1 --2- 172.21.15.156:0/1224425834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 0x559adee1c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff243fff700 1 --2- 172.21.15.156:0/1224425834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee25860 0x559adedf5480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff230016000 tx=0x7ff23000e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d36ea88abe456a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff2237fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-28T11:10:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff2237fe700 1 -- 172.21.15.156:0/1224425834 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230021040 con 0x559adee25860 2023-11-28T11:10:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff2237fe700 1 -- 172.21.15.156:0/1224425834 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff2300117b0 con 0x559adee25860 2023-11-28T11:10:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff256d20b80 1 -- 172.21.15.156:0/1224425834 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559aded1b190 con 0x559adee25860 2023-11-28T11:10:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff2237fe700 1 -- 172.21.15.156:0/1224425834 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff230004450 con 0x559adee25860 2023-11-28T11:10:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.789+0000 7ff256d20b80 10 client.5410 fetch_fsmap learned FSMap version 23 2023-11-28T11:10:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.790+0000 7ff256d20b80 1 -- 172.21.15.156:0/1224425834 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x559adeb7bde0 con 0x559adee25860 2023-11-28T11:10:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.790+0000 7ff2237fe700 1 -- 172.21.15.156:0/1224425834 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559aded1b190 con 0x559adee25860 2023-11-28T11:10:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.790+0000 7ff2237fe700 1 -- 172.21.15.156:0/1224425834 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff2300036a0 con 0x559adee25860 2023-11-28T11:10:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.790+0000 7ff256d20b80 10 client.5410 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T11:10:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.790+0000 7ff256d20b80 10 client.5410 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:10:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.790+0000 7ff256d20b80 20 client.5410 populate_metadata read hostname 'smithi156' 2023-11-28T11:10:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.790+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1224425834 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x559adee3a420 0x559adee1ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.790+0000 7ff256d20b80 4 client.5410 mds_command: new command op to 5547 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:10:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.790+0000 7ff256d20b80 1 -- 172.21.15.156:0/1224425834 --> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559adebd1f60 con 0x559adee3a420 2023-11-28T11:10:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.790+0000 7ff248933700 1 --2- 172.21.15.156:0/1224425834 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x559adee3a420 0x559adee1ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.790+0000 7ff2237fe700 10 client.5410 ms_handle_connect on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.790+0000 7ff248933700 1 --2- 172.21.15.156:0/1224425834 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x559adee3a420 0x559adee1ce70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80957b69295eb6ca server_cookie=31f87f741b0dea16 in_seq=0 out_seq=0 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff2237fe700 1 -- 172.21.15.156:0/1224425834 <== mds.0 v2:172.21.15.186:6833/3832587641 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x559adebd1f60 con 0x559adee3a420 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff2237fe700 10 client.5410 handle_command_reply: tid=0 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff256d20b80 1 client.5410 shutdown 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff256d20b80 1 -- 172.21.15.156:0/1224425834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee25860 msgr2=0x559adedf5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1224425834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee25860 0x559adedf5480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff230016000 tx=0x7ff23000e000 comp rx=0 tx=0).stop 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff256d20b80 1 -- 172.21.15.156:0/1224425834 shutdown_connections 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1224425834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee26170 0x559adee1c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1224425834 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x559adee3a420 0x559adee1ce70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1224425834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee23180 0x559adedf4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1224425834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee25860 0x559adedf5480 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff2237fe700 0 client.5410 ms_handle_reset on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff256d20b80 1 -- 172.21.15.156:0/1224425834 >> 172.21.15.156:0/1224425834 conn(0x559adedf8db0 msgr2=0x559adedfa280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff256d20b80 1 -- 172.21.15.156:0/1224425834 shutdown_connections 2023-11-28T11:10:02.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff256d20b80 1 -- 172.21.15.156:0/1224425834 wait complete. 2023-11-28T11:10:02.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.791+0000 7ff256d20b80 20 client.5410 trim_cache size 0 max 16384 2023-11-28T11:10:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.807+0000 7ff256d20b80 1 Processor -- start 2023-11-28T11:10:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.807+0000 7ff256d20b80 1 -- start start 2023-11-28T11:10:02.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff256d20b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee23180 0x559adedef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff256d20b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 0x559adee1ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff256d20b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee26170 0x559adee37ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff256d20b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559adee24f00 con 0x559adee25860 2023-11-28T11:10:02.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff256d20b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559adee44940 con 0x559adee23180 2023-11-28T11:10:02.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff256d20b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x559adee38400 con 0x559adee26170 2023-11-28T11:10:02.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff243fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 0x559adee1ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff243fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 0x559adee1ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34962/0 (socket says 172.21.15.156:34962) 2023-11-28T11:10:02.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff243fff700 1 -- 172.21.15.156:0/3833274356 learned_addr learned my addr 172.21.15.156:0/3833274356 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff248933700 1 --2- 172.21.15.156:0/3833274356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee23180 0x559adedef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff249134700 1 --2- 172.21.15.156:0/3833274356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee26170 0x559adee37ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff243fff700 1 -- 172.21.15.156:0/3833274356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee26170 msgr2=0x559adee37ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff243fff700 1 --2- 172.21.15.156:0/3833274356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee26170 0x559adee37ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff243fff700 1 -- 172.21.15.156:0/3833274356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee23180 msgr2=0x559adedef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff243fff700 1 --2- 172.21.15.156:0/3833274356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee23180 0x559adedef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.808+0000 7ff243fff700 1 -- 172.21.15.156:0/3833274356 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559adedf7f30 con 0x559adee25860 2023-11-28T11:10:02.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff249134700 1 --2- 172.21.15.156:0/3833274356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee26170 0x559adee37ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:02.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff243fff700 1 --2- 172.21.15.156:0/3833274356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 0x559adee1ca80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff230009f60 tx=0x7ff230011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76cd74197a07cf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff22effd700 1 -- 172.21.15.156:0/3833274356 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230021050 con 0x559adee25860 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff22effd700 1 -- 172.21.15.156:0/3833274356 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230003dc0 con 0x559adee25860 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff22effd700 1 -- 172.21.15.156:0/3833274356 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230011d10 con 0x559adee25860 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff256d20b80 1 -- 172.21.15.156:0/3833274356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 msgr2=0x559adee1ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff256d20b80 1 --2- 172.21.15.156:0/3833274356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 0x559adee1ca80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff230009f60 tx=0x7ff230011000 comp rx=0 tx=0).stop 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff256d20b80 1 -- 172.21.15.156:0/3833274356 shutdown_connections 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff256d20b80 1 --2- 172.21.15.156:0/3833274356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee26170 0x559adee37ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff256d20b80 1 --2- 172.21.15.156:0/3833274356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee23180 0x559adedef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff256d20b80 1 --2- 172.21.15.156:0/3833274356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 0x559adee1ca80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff256d20b80 1 -- 172.21.15.156:0/3833274356 >> 172.21.15.156:0/3833274356 conn(0x559adedf8db0 msgr2=0x559adee1e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff256d20b80 1 -- 172.21.15.156:0/3833274356 shutdown_connections 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.809+0000 7ff256d20b80 1 -- 172.21.15.156:0/3833274356 wait complete. 2023-11-28T11:10:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.810+0000 7ff256d20b80 1 -- start start 2023-11-28T11:10:02.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.810+0000 7ff256d20b80 1 Processor -- start 2023-11-28T11:10:02.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff256d20b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee23180 0x559adeea6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff256d20b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 0x559adee1b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff256d20b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee26170 0x559adeea2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff256d20b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559adee23a20 con 0x559adee25860 2023-11-28T11:10:02.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff256d20b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559adedf5250 con 0x559adee23180 2023-11-28T11:10:02.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff256d20b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x559adee23550 con 0x559adee26170 2023-11-28T11:10:02.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff256d20b80 10 client.0 _ll_register_callbacks cb 0x559adebf3290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:02.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff243fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 0x559adee1b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff243fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 0x559adee1b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34968/0 (socket says 172.21.15.156:34968) 2023-11-28T11:10:02.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff243fff700 1 -- 172.21.15.156:0/1203712986 learned_addr learned my addr 172.21.15.156:0/1203712986 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:02.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff248933700 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee23180 0x559adeea6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff249134700 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee26170 0x559adeea2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff243fff700 1 -- 172.21.15.156:0/1203712986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee26170 msgr2=0x559adeea2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff243fff700 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee26170 0x559adeea2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff243fff700 1 -- 172.21.15.156:0/1203712986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee23180 msgr2=0x559adeea6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff243fff700 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee23180 0x559adeea6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff243fff700 1 -- 172.21.15.156:0/1203712986 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559adeb7bde0 con 0x559adee25860 2023-11-28T11:10:02.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff243fff700 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 0x559adee1b910 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff230009130 tx=0x7ff2300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb0d9331e46f42e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.811+0000 7ff2237fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff2237fe700 1 -- 172.21.15.156:0/1203712986 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230021050 con 0x559adee25860 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff2237fe700 1 -- 172.21.15.156:0/1203712986 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff230000e40 con 0x559adee25860 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff256d20b80 1 -- 172.21.15.156:0/1203712986 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559aded0b1b0 con 0x559adee25860 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff2237fe700 1 -- 172.21.15.156:0/1203712986 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff230026480 con 0x559adee25860 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff256d20b80 10 client.5772 fetch_fsmap learned FSMap version 23 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff256d20b80 1 -- 172.21.15.156:0/1203712986 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x559adeb64c60 con 0x559adee25860 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff2237fe700 1 -- 172.21.15.156:0/1203712986 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559aded0b1b0 con 0x559adee25860 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff2237fe700 1 -- 172.21.15.156:0/1203712986 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff2300265e0 con 0x559adee25860 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff256d20b80 10 client.5772 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff256d20b80 10 client.5772 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff256d20b80 20 client.5772 populate_metadata read hostname 'smithi156' 2023-11-28T11:10:02.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x559adee46470 0x559adeea2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff256d20b80 4 client.5772 mds_command: new command op to 5547 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:10:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff256d20b80 1 -- 172.21.15.156:0/1203712986 --> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559adecf51c0 con 0x559adee46470 2023-11-28T11:10:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff248933700 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x559adee46470 0x559adeea2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff2237fe700 10 client.5772 ms_handle_connect on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.812+0000 7ff248933700 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x559adee46470 0x559adeea2bd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9567884b2d352b server_cookie=5751c8e297715923 in_seq=0 out_seq=0 2023-11-28T11:10:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff2237fe700 1 -- 172.21.15.156:0/1203712986 <== mds.0 v2:172.21.15.186:6833/3832587641 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x559adecf51c0 con 0x559adee46470 2023-11-28T11:10:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff2237fe700 10 client.5772 handle_command_reply: tid=0 2023-11-28T11:10:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff256d20b80 1 client.5772 shutdown 2023-11-28T11:10:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff256d20b80 1 -- 172.21.15.156:0/1203712986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 msgr2=0x559adee1b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 0x559adee1b910 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff230009130 tx=0x7ff2300047c0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff256d20b80 1 -- 172.21.15.156:0/1203712986 shutdown_connections 2023-11-28T11:10:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x559adee46470 0x559adeea2bd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559adee26170 0x559adeea2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff2237fe700 0 client.5772 ms_handle_reset on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:02.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559adee23180 0x559adeea6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff256d20b80 1 --2- 172.21.15.156:0/1203712986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559adee25860 0x559adee1b910 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff256d20b80 1 -- 172.21.15.156:0/1203712986 >> 172.21.15.156:0/1203712986 conn(0x559adedf8db0 msgr2=0x559adee43aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff256d20b80 1 -- 172.21.15.156:0/1203712986 shutdown_connections 2023-11-28T11:10:02.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff256d20b80 1 -- 172.21.15.156:0/1203712986 wait complete. 2023-11-28T11:10:02.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff256d20b80 20 client.5772 trim_cache size 0 max 16384 2023-11-28T11:10:02.821 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:10:02.821 INFO:teuthology.orchestra.run.smithi156.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T11:10:02.822 INFO:teuthology.orchestra.run.smithi156.stdout: "scrubs": { 2023-11-28T11:10:02.822 INFO:teuthology.orchestra.run.smithi156.stdout: "3b3ef99b-8250-42c5-b5e8-414d052dc90f": { 2023-11-28T11:10:02.822 INFO:teuthology.orchestra.run.smithi156.stdout: "path": "/", 2023-11-28T11:10:02.822 INFO:teuthology.orchestra.run.smithi156.stdout: "tag": "3b3ef99b-8250-42c5-b5e8-414d052dc90f", 2023-11-28T11:10:02.822 INFO:teuthology.orchestra.run.smithi156.stdout: "options": "recursive,repair" 2023-11-28T11:10:02.822 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:02.822 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:02.822 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:10:02.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff24ab97700 1 -- 172.21.15.156:0/1840683890 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff2280682f0 msgr2=0x7ff22806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff24ab97700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff2280682f0 0x7ff22806a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff238006f10 tx=0x7ff238008040 comp rx=0 tx=0).stop 2023-11-28T11:10:02.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff24ab97700 1 -- 172.21.15.156:0/1840683890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff244160b10 msgr2=0x7ff24406da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff24ab97700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff244160b10 0x7ff24406da00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00efc0 tx=0x7ff23c00c0c0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff24ab97700 1 -- 172.21.15.156:0/1840683890 shutdown_connections 2023-11-28T11:10:02.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff24ab97700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff244160b10 0x7ff24406da00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff24ab97700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff2280682f0 0x7ff22806a7a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff24ab97700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff24406b060 0x7ff2440c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff24ab97700 1 --2- 172.21.15.156:0/1840683890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff244160740 0x7ff2440c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.813+0000 7ff24ab97700 1 -- 172.21.15.156:0/1840683890 >> 172.21.15.156:0/1840683890 conn(0x7ff2440b12b0 msgr2=0x7ff2440cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.814+0000 7ff24ab97700 1 -- 172.21.15.156:0/1840683890 shutdown_connections 2023-11-28T11:10:02.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:02.814+0000 7ff24ab97700 1 -- 172.21.15.156:0/1840683890 wait complete. 2023-11-28T11:10:02.826 INFO:tasks.cephfs.filesystem:scrub status for tag:3b3ef99b-8250-42c5-b5e8-414d052dc90f - {'path': '/', 'tag': '3b3ef99b-8250-42c5-b5e8-414d052dc90f', 'options': 'recursive,repair'} 2023-11-28T11:10:03.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:03.310+0000 7fb4a2ffd700 1 -- 172.21.15.156:0/4012278286 <== mon.0 v2:172.21.15.156:3300/0 8 ==== log(6 entries from seq 1 at 2023-11-28T11:10:02.642907+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7fb4a80864f0 con 0x7fb4ac0b6840 2023-11-28T11:10:03.312 INFO:teuthology.orchestra.run.smithi156.stdout: cluster: 2023-11-28T11:10:03.312 INFO:teuthology.orchestra.run.smithi156.stdout: id: 351da896-8dbf-473b-9472-c3454d9368bf 2023-11-28T11:10:03.312 INFO:teuthology.orchestra.run.smithi156.stdout: health: HEALTH_OK 2023-11-28T11:10:03.312 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: services: 2023-11-28T11:10:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-28T11:10:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: mgr: y(active, since 2m), standbys: x 2023-11-28T11:10:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: mds: 1/1 daemons up, 3 standby 2023-11-28T11:10:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: osd: 8 osds: 8 up (since 116s), 8 in (since 2m) 2023-11-28T11:10:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: data: 2023-11-28T11:10:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: volumes: 1/1 healthy 2023-11-28T11:10:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: pools: 4 pools, 105 pgs 2023-11-28T11:10:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: objects: 23 objects, 579 KiB 2023-11-28T11:10:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: usage: 73 MiB used, 720 GiB / 720 GiB avail 2023-11-28T11:10:03.313 INFO:teuthology.orchestra.run.smithi156.stdout: pgs: 105 active+clean 2023-11-28T11:10:03.314 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:03.314 INFO:teuthology.orchestra.run.smithi156.stdout: io: 2023-11-28T11:10:03.314 INFO:teuthology.orchestra.run.smithi156.stdout: client: 2.1 KiB/s wr, 0 op/s rd, 6 op/s wr 2023-11-28T11:10:03.314 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:03.314 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:03.314 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T11:10:02.642907+0000 mds.d [INF] scrub queued for path: / 2023-11-28T11:10:03.314 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T11:10:02.642910+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2023-11-28T11:10:03.314 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T11:10:02.642915+0000 mds.d [INF] scrub summary: active paths [/] 2023-11-28T11:10:03.314 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T11:10:02.643718+0000 mds.d [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2023-11-28T11:10:03.314 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T11:10:02.643725+0000 mds.d [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2023-11-28T11:10:03.314 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T11:10:02.644027+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2023-11-28T11:10:05.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:05.769+0000 7fb4a2ffd700 1 -- 172.21.15.156:0/4012278286 <== mon.0 v2:172.21.15.156:3300/0 9 ==== log(1 entries from seq 7 at 2023-11-28T11:10:03.988474+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7fb4a805c0e0 con 0x7fb4ac0b6840 2023-11-28T11:10:05.770 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T11:10:03.988474+0000 mds.d [INF] scrub summary: idle 2023-11-28T11:10:21.151 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:10:21.154 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:10:32.827 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T11:10:32.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.963+0000 7f71a2e71700 1 Processor -- start 2023-11-28T11:10:32.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.963+0000 7f71a2e71700 1 -- start start 2023-11-28T11:10:32.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f71a2e71700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f719c071040 0x7f719c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f71a2e71700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c071950 con 0x7f719c06dee0 2023-11-28T11:10:32.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f71a2e71700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c15d060 con 0x7f719c160540 2023-11-28T11:10:32.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f71a2e71700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f719c15d340 con 0x7f719c071040 2023-11-28T11:10:32.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f71a0c0d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f719c071040 0x7f719c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f71a0c0d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f719c071040 0x7f719c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34974/0 (socket says 172.21.15.156:34974) 2023-11-28T11:10:32.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f719bfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f719c06dee0 0x7f719c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49854/0 (socket says 172.21.15.156:49854) 2023-11-28T11:10:32.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f71a0c0d700 1 -- 172.21.15.156:0/379128142 learned_addr learned my addr 172.21.15.156:0/379128142 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:32.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f71a0c0d700 1 -- 172.21.15.156:0/379128142 >> v1:172.21.15.186:6789/0 conn(0x7f719c160540 legacy=0x7f719c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f71a0c0d700 1 -- 172.21.15.156:0/379128142 >> v1:172.21.15.156:6789/0 conn(0x7f719c06dee0 legacy=0x7f719c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f71a0c0d700 1 -- 172.21.15.156:0/379128142 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c15d620 con 0x7f719c071040 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f719b7fe700 1 -- 172.21.15.156:0/379128142 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732365187 0 0) 0x7f719c071950 con 0x7f719c06dee0 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.964+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/379128142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c071040 0x7f719c071410 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f71900098f0 tx=0x7f7190009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0f64ce710a25dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.965+0000 7f719b7fe700 1 -- 172.21.15.156:0/379128142 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719000b8a0 con 0x7f719c071040 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.965+0000 7f719b7fe700 1 -- 172.21.15.156:0/379128142 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719000ba00 con 0x7f719c071040 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.965+0000 7f719b7fe700 1 -- 172.21.15.156:0/379128142 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719000bd90 con 0x7f719c071040 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.965+0000 7f71a2e71700 1 -- 172.21.15.156:0/379128142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c071040 msgr2=0x7f719c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.965+0000 7f71a2e71700 1 --2- 172.21.15.156:0/379128142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c071040 0x7f719c071410 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f71900098f0 tx=0x7f7190009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.965+0000 7f71a2e71700 1 -- 172.21.15.156:0/379128142 shutdown_connections 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.965+0000 7f71a2e71700 1 --2- 172.21.15.156:0/379128142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c071040 0x7f719c071410 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.965+0000 7f71a2e71700 1 -- 172.21.15.156:0/379128142 >> 172.21.15.156:0/379128142 conn(0x7f719c0d0000 msgr2=0x7f719c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.965+0000 7f71a2e71700 1 -- 172.21.15.156:0/379128142 shutdown_connections 2023-11-28T11:10:32.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.965+0000 7f71a2e71700 1 -- 172.21.15.156:0/379128142 wait complete. 2023-11-28T11:10:32.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.966+0000 7f71a2e71700 1 Processor -- start 2023-11-28T11:10:32.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.966+0000 7f71a2e71700 1 -- start start 2023-11-28T11:10:32.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.967+0000 7f71a2e71700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c06dee0 0x7f719c1f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.967+0000 7f71a2e71700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f719c071040 0x7f719c15f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.967+0000 7f71a2e71700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f719c160540 0x7f719c15f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.967+0000 7f71a2e71700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c159ea0 con 0x7f719c06dee0 2023-11-28T11:10:32.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.967+0000 7f71a2e71700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c0b4830 con 0x7f719c160540 2023-11-28T11:10:32.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.967+0000 7f71a2e71700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c0b3cb0 con 0x7f719c071040 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.967+0000 7f71a0c0d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c06dee0 0x7f719c1f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.967+0000 7f71a0c0d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c06dee0 0x7f719c1f1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34980/0 (socket says 172.21.15.156:34980) 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71a0c0d700 1 -- 172.21.15.156:0/2289743974 learned_addr learned my addr 172.21.15.156:0/2289743974 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71a140e700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f719c160540 0x7f719c15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f719bfff700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f719c071040 0x7f719c15f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71a0c0d700 1 -- 172.21.15.156:0/2289743974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f719c071040 msgr2=0x7f719c15f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f719c071040 0x7f719c15f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71a0c0d700 1 -- 172.21.15.156:0/2289743974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f719c160540 msgr2=0x7f719c15f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f719c160540 0x7f719c15f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71a0c0d700 1 -- 172.21.15.156:0/2289743974 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7190009550 con 0x7f719c06dee0 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f719bfff700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f719c071040 0x7f719c15f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c06dee0 0x7f719c1f1fb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7190014040 tx=0x7f7190004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71997fa700 1 -- 172.21.15.156:0/2289743974 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71900119f0 con 0x7f719c06dee0 2023-11-28T11:10:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71997fa700 1 -- 172.21.15.156:0/2289743974 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719001e070 con 0x7f719c06dee0 2023-11-28T11:10:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71a2e71700 1 -- 172.21.15.156:0/2289743974 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c15ccd0 con 0x7f719c06dee0 2023-11-28T11:10:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71997fa700 1 -- 172.21.15.156:0/2289743974 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7190011d70 con 0x7f719c06dee0 2023-11-28T11:10:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.968+0000 7f71a2e71700 1 -- 172.21.15.156:0/2289743974 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c15d900 con 0x7f719c06dee0 2023-11-28T11:10:32.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.970+0000 7f71997fa700 1 -- 172.21.15.156:0/2289743974 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f719000f050 con 0x7f719c06dee0 2023-11-28T11:10:32.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.970+0000 7f71997fa700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71800682f0 0x7f718006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.970+0000 7f71997fa700 1 -- 172.21.15.156:0/2289743974 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7190086d20 con 0x7f719c06dee0 2023-11-28T11:10:32.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.970+0000 7f719bfff700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71800682f0 0x7f718006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.971+0000 7f719bfff700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71800682f0 0x7f718006a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f719c071710 tx=0x7f7188009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.972+0000 7f71aeffab80 1 Processor -- start 2023-11-28T11:10:32.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.972+0000 7f71aeffab80 1 -- start start 2023-11-28T11:10:32.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71aeffab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f506860 0x56178f4dca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71aeffab80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f504180 0x56178f506c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71aeffab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f507170 0x56178f5227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71aeffab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178f3c6c10 con 0x56178f507170 2023-11-28T11:10:32.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71aeffab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178f4353a0 con 0x56178f506860 2023-11-28T11:10:32.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71aeffab80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56178f42e8d0 con 0x56178f504180 2023-11-28T11:10:32.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71a140e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f507170 0x56178f5227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71a140e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f507170 0x56178f5227c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34992/0 (socket says 172.21.15.156:34992) 2023-11-28T11:10:32.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71a140e700 1 -- 172.21.15.156:0/1682625716 learned_addr learned my addr 172.21.15.156:0/1682625716 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:32.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f719bfff700 1 --2- 172.21.15.156:0/1682625716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f504180 0x56178f506c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71a140e700 1 -- 172.21.15.156:0/1682625716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f504180 msgr2=0x56178f506c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71a140e700 1 --2- 172.21.15.156:0/1682625716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f504180 0x56178f506c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71a140e700 1 -- 172.21.15.156:0/1682625716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f506860 msgr2=0x56178f4dca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/1682625716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f506860 0x56178f4dca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71a140e700 1 --2- 172.21.15.156:0/1682625716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f506860 0x56178f4dca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71a140e700 1 -- 172.21.15.156:0/1682625716 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56178f2b0630 con 0x56178f507170 2023-11-28T11:10:32.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f719bfff700 1 --2- 172.21.15.156:0/1682625716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f504180 0x56178f506c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:32.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.973+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/1682625716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f506860 0x56178f4dca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:32.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71a140e700 1 --2- 172.21.15.156:0/1682625716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f507170 0x56178f5227c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f71940020f0 tx=0x7f719400e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2e2bb2d9b59b5a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71867fc700 1 -- 172.21.15.156:0/1682625716 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719400ec60 con 0x56178f507170 2023-11-28T11:10:32.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71867fc700 1 -- 172.21.15.156:0/1682625716 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719400edc0 con 0x56178f507170 2023-11-28T11:10:32.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71867fc700 1 -- 172.21.15.156:0/1682625716 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194003c20 con 0x56178f507170 2023-11-28T11:10:32.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71aeffab80 1 -- 172.21.15.156:0/1682625716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f507170 msgr2=0x56178f5227c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71aeffab80 1 --2- 172.21.15.156:0/1682625716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f507170 0x56178f5227c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f71940020f0 tx=0x7f719400e440 comp rx=0 tx=0).stop 2023-11-28T11:10:32.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71aeffab80 1 -- 172.21.15.156:0/1682625716 shutdown_connections 2023-11-28T11:10:32.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71aeffab80 1 --2- 172.21.15.156:0/1682625716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f507170 0x56178f5227c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71aeffab80 1 --2- 172.21.15.156:0/1682625716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f506860 0x56178f4dca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71aeffab80 1 --2- 172.21.15.156:0/1682625716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f504180 0x56178f506c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71aeffab80 1 -- 172.21.15.156:0/1682625716 >> 172.21.15.156:0/1682625716 conn(0x56178f4d9db0 msgr2=0x56178f4da740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71aeffab80 1 -- 172.21.15.156:0/1682625716 shutdown_connections 2023-11-28T11:10:32.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.974+0000 7f71aeffab80 1 -- 172.21.15.156:0/1682625716 wait complete. 2023-11-28T11:10:32.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.975+0000 7f71aeffab80 1 -- start start 2023-11-28T11:10:32.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.975+0000 7f71aeffab80 1 Processor -- start 2023-11-28T11:10:32.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71aeffab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 0x56178f50a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71aeffab80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f506860 0x56178f4fdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71aeffab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f507170 0x56178f519f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71aeffab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178f2b2920 con 0x56178f504180 2023-11-28T11:10:32.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71aeffab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178f4fe810 con 0x56178f507170 2023-11-28T11:10:32.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71aeffab80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56178f527bd0 con 0x56178f506860 2023-11-28T11:10:32.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71aeffab80 10 client.0 _ll_register_callbacks cb 0x56178f2d4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:32.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71a0c0d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 0x56178f50a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71a0c0d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 0x56178f50a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34994/0 (socket says 172.21.15.156:34994) 2023-11-28T11:10:32.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71a0c0d700 1 -- 172.21.15.156:0/4209432880 learned_addr learned my addr 172.21.15.156:0/4209432880 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:32.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f719bfff700 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f506860 0x56178f4fdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71a140e700 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f507170 0x56178f519f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71a0c0d700 1 -- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f506860 msgr2=0x56178f4fdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f506860 0x56178f4fdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71a0c0d700 1 -- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f507170 msgr2=0x56178f519f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:32.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f507170 0x56178f519f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71a0c0d700 1 -- 172.21.15.156:0/4209432880 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719400f040 con 0x56178f504180 2023-11-28T11:10:32.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f719bfff700 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f506860 0x56178f4fdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.976+0000 7f71a140e700 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f507170 0x56178f519f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 0x56178f50a0a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7190056c50 tx=0x7f719002b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5a5416d524c578 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f717b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f717b7fe700 1 -- 172.21.15.156:0/4209432880 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719005ee00 con 0x56178f504180 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f717b7fe700 1 -- 172.21.15.156:0/4209432880 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f719005ef60 con 0x56178f504180 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f71aeffab80 1 -- 172.21.15.156:0/4209432880 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56178f3fc190 con 0x56178f504180 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f717b7fe700 1 -- 172.21.15.156:0/4209432880 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f719006e1b0 con 0x56178f504180 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f71aeffab80 10 client.5787 fetch_fsmap learned FSMap version 23 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f71aeffab80 1 -- 172.21.15.156:0/4209432880 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x56178f25cde0 con 0x56178f504180 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f717b7fe700 1 -- 172.21.15.156:0/4209432880 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56178f3fc190 con 0x56178f504180 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f717b7fe700 1 -- 172.21.15.156:0/4209432880 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f719006e310 con 0x56178f504180 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f71aeffab80 10 client.5787 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T11:10:32.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f71aeffab80 10 client.5787 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:10:32.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f71aeffab80 20 client.5787 populate_metadata read hostname 'smithi156' 2023-11-28T11:10:32.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f71aeffab80 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56178f5276d0 0x56178f4fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f71aeffab80 4 client.5787 mds_command: new command op to 5547 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:10:32.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f71aeffab80 1 -- 172.21.15.156:0/4209432880 --> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56178f2b2f60 con 0x56178f5276d0 2023-11-28T11:10:32.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.977+0000 7f719bfff700 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56178f5276d0 0x56178f4fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.978+0000 7f717b7fe700 10 client.5787 ms_handle_connect on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:32.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.978+0000 7f719bfff700 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56178f5276d0 0x56178f4fd1c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7877281a4bf8eaf9 server_cookie=a26538bc3d4d4e1c in_seq=0 out_seq=0 2023-11-28T11:10:32.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.978+0000 7f717b7fe700 1 -- 172.21.15.156:0/4209432880 <== mds.0 v2:172.21.15.186:6833/3832587641 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x56178f2b2f60 con 0x56178f5276d0 2023-11-28T11:10:32.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.978+0000 7f717b7fe700 10 client.5787 handle_command_reply: tid=0 2023-11-28T11:10:32.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f71aeffab80 1 client.5787 shutdown 2023-11-28T11:10:32.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f71aeffab80 1 -- 172.21.15.156:0/4209432880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 msgr2=0x56178f50a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f71aeffab80 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 0x56178f50a0a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7190056c50 tx=0x7f719002b400 comp rx=0 tx=0).stop 2023-11-28T11:10:32.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f71aeffab80 1 -- 172.21.15.156:0/4209432880 shutdown_connections 2023-11-28T11:10:32.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f71aeffab80 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f507170 0x56178f519f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f71aeffab80 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 0x56178f50a0a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f71aeffab80 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56178f5276d0 0x56178f4fd1c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f71aeffab80 1 --2- 172.21.15.156:0/4209432880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f506860 0x56178f4fdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f71aeffab80 1 -- 172.21.15.156:0/4209432880 >> 172.21.15.156:0/4209432880 conn(0x56178f4d9db0 msgr2=0x56178f4db280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f717b7fe700 0 client.5787 ms_handle_reset on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:32.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f71aeffab80 1 -- 172.21.15.156:0/4209432880 shutdown_connections 2023-11-28T11:10:32.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f71aeffab80 1 -- 172.21.15.156:0/4209432880 wait complete. 2023-11-28T11:10:32.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.979+0000 7f71aeffab80 20 client.5787 trim_cache size 0 max 16384 2023-11-28T11:10:32.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.995+0000 7f71aeffab80 1 Processor -- start 2023-11-28T11:10:32.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.996+0000 7f71aeffab80 1 -- start start 2023-11-28T11:10:33.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.996+0000 7f71aeffab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f504180 0x56178f519f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.996+0000 7f71aeffab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f506860 0x56178f51a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.996+0000 7f71aeffab80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f507170 0x56178f5185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.996+0000 7f71aeffab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178f4dd110 con 0x56178f506860 2023-11-28T11:10:33.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.996+0000 7f71aeffab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178f5242e0 con 0x56178f504180 2023-11-28T11:10:33.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.996+0000 7f71aeffab80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56178f525440 con 0x56178f507170 2023-11-28T11:10:33.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f719bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f506860 0x56178f51a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f719bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f506860 0x56178f51a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35004/0 (socket says 172.21.15.156:35004) 2023-11-28T11:10:33.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f719bfff700 1 -- 172.21.15.156:0/561630077 learned_addr learned my addr 172.21.15.156:0/561630077 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:33.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f71a140e700 1 --2- 172.21.15.156:0/561630077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f507170 0x56178f5185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/561630077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f504180 0x56178f519f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f719bfff700 1 -- 172.21.15.156:0/561630077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f507170 msgr2=0x56178f5185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:33.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f719bfff700 1 --2- 172.21.15.156:0/561630077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f507170 0x56178f5185b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f719bfff700 1 -- 172.21.15.156:0/561630077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f504180 msgr2=0x56178f519f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:33.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f719bfff700 1 --2- 172.21.15.156:0/561630077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f504180 0x56178f519f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f719bfff700 1 -- 172.21.15.156:0/561630077 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56178f3f5190 con 0x56178f506860 2023-11-28T11:10:33.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f719bfff700 1 --2- 172.21.15.156:0/561630077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f506860 0x56178f51a470 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7188000c00 tx=0x7f7188016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a4a8bb0df13267 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:33.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f7186ffd700 1 -- 172.21.15.156:0/561630077 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188019040 con 0x56178f506860 2023-11-28T11:10:33.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f7186ffd700 1 -- 172.21.15.156:0/561630077 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718801a040 con 0x56178f506860 2023-11-28T11:10:33.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.997+0000 7f7186ffd700 1 -- 172.21.15.156:0/561630077 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188022ae0 con 0x56178f506860 2023-11-28T11:10:33.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.998+0000 7f71aeffab80 1 -- 172.21.15.156:0/561630077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f506860 msgr2=0x56178f51a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.998+0000 7f71aeffab80 1 --2- 172.21.15.156:0/561630077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f506860 0x56178f51a470 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7188000c00 tx=0x7f7188016040 comp rx=0 tx=0).stop 2023-11-28T11:10:33.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.998+0000 7f71aeffab80 1 -- 172.21.15.156:0/561630077 shutdown_connections 2023-11-28T11:10:33.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.998+0000 7f71aeffab80 1 --2- 172.21.15.156:0/561630077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f507170 0x56178f5185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.998+0000 7f71aeffab80 1 --2- 172.21.15.156:0/561630077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f504180 0x56178f519f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.998+0000 7f71aeffab80 1 --2- 172.21.15.156:0/561630077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f506860 0x56178f51a470 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.998+0000 7f71aeffab80 1 -- 172.21.15.156:0/561630077 >> 172.21.15.156:0/561630077 conn(0x56178f4d9db0 msgr2=0x56178f4dda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.998+0000 7f71aeffab80 1 -- 172.21.15.156:0/561630077 shutdown_connections 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.998+0000 7f71aeffab80 1 -- 172.21.15.156:0/561630077 wait complete. 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.998+0000 7f71aeffab80 1 -- start start 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:32.999+0000 7f71aeffab80 1 Processor -- start 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71aeffab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 0x56178f586960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71aeffab80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f506860 0x56178f4d54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71aeffab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f507170 0x56178f4d59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71aeffab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178f587590 con 0x56178f504180 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71aeffab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178f4da8e0 con 0x56178f507170 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71aeffab80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56178f4dab30 con 0x56178f506860 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71aeffab80 10 client.0 _ll_register_callbacks cb 0x7f7190057000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:33.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71a0c0d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 0x56178f586960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71a0c0d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 0x56178f586960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35008/0 (socket says 172.21.15.156:35008) 2023-11-28T11:10:33.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71a0c0d700 1 -- 172.21.15.156:0/1085208111 learned_addr learned my addr 172.21.15.156:0/1085208111 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:33.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71a140e700 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f507170 0x56178f4d59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f719bfff700 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f506860 0x56178f4d54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71a0c0d700 1 -- 172.21.15.156:0/1085208111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f506860 msgr2=0x56178f4d54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:33.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f506860 0x56178f4d54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71a0c0d700 1 -- 172.21.15.156:0/1085208111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f507170 msgr2=0x56178f4d59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:33.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f507170 0x56178f4d59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71a0c0d700 1 -- 172.21.15.156:0/1085208111 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56178f283250 con 0x56178f504180 2023-11-28T11:10:33.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71a0c0d700 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 0x56178f586960 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f719005f450 tx=0x7f7190000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d61f7297fe0c37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:33.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f717b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:10:33.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f717b7fe700 1 -- 172.21.15.156:0/1085208111 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719006e3b0 con 0x56178f504180 2023-11-28T11:10:33.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f717b7fe700 1 -- 172.21.15.156:0/1085208111 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7190056160 con 0x56178f504180 2023-11-28T11:10:33.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.000+0000 7f71aeffab80 1 -- 172.21.15.156:0/1085208111 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56178f3ec1b0 con 0x56178f504180 2023-11-28T11:10:33.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f717b7fe700 1 -- 172.21.15.156:0/1085208111 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f719005f010 con 0x56178f504180 2023-11-28T11:10:33.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f71aeffab80 10 client.5793 fetch_fsmap learned FSMap version 23 2023-11-28T11:10:33.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f71aeffab80 1 -- 172.21.15.156:0/1085208111 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x56178f245c60 con 0x56178f504180 2023-11-28T11:10:33.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f717b7fe700 1 -- 172.21.15.156:0/1085208111 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56178f3ec1b0 con 0x56178f504180 2023-11-28T11:10:33.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f717b7fe700 1 -- 172.21.15.156:0/1085208111 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7190056320 con 0x56178f504180 2023-11-28T11:10:33.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f71aeffab80 10 client.5793 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T11:10:33.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f71aeffab80 10 client.5793 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:10:33.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f71aeffab80 20 client.5793 populate_metadata read hostname 'smithi156' 2023-11-28T11:10:33.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f71aeffab80 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56178f586ea0 0x56178f53a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f71aeffab80 4 client.5793 mds_command: new command op to 5547 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f71aeffab80 1 -- 172.21.15.156:0/1085208111 --> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56178f3d61c0 con 0x56178f586ea0 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f719bfff700 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56178f586ea0 0x56178f53a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f717b7fe700 10 client.5793 ms_handle_connect on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.001+0000 7f719bfff700 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56178f586ea0 0x56178f53a890 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a9f3f18ef53ce40 server_cookie=75e30d4fc3738a4b in_seq=0 out_seq=0 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f717b7fe700 1 -- 172.21.15.156:0/1085208111 <== mds.0 v2:172.21.15.186:6833/3832587641 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56178f3d61c0 con 0x56178f586ea0 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f717b7fe700 10 client.5793 handle_command_reply: tid=0 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71aeffab80 1 client.5793 shutdown 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71aeffab80 1 -- 172.21.15.156:0/1085208111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 msgr2=0x56178f586960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71aeffab80 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 0x56178f586960 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f719005f450 tx=0x7f7190000bc0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71aeffab80 1 -- 172.21.15.156:0/1085208111 shutdown_connections 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71aeffab80 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56178f507170 0x56178f4d59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71aeffab80 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56178f504180 0x56178f586960 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71aeffab80 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.186:6833/3832587641,v1:172.21.15.186:6835/3832587641] conn(0x56178f586ea0 0x56178f53a890 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71aeffab80 1 --2- 172.21.15.156:0/1085208111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56178f506860 0x56178f4d54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f717b7fe700 0 client.5793 ms_handle_reset on v2:172.21.15.186:6833/3832587641 2023-11-28T11:10:33.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71aeffab80 1 -- 172.21.15.156:0/1085208111 >> 172.21.15.156:0/1085208111 conn(0x56178f4d9db0 msgr2=0x56178f517000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:33.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71aeffab80 1 -- 172.21.15.156:0/1085208111 shutdown_connections 2023-11-28T11:10:33.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71aeffab80 1 -- 172.21.15.156:0/1085208111 wait complete. 2023-11-28T11:10:33.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71aeffab80 20 client.5793 trim_cache size 0 max 16384 2023-11-28T11:10:33.013 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:10:33.013 INFO:teuthology.orchestra.run.smithi156.stdout: "status": "no active scrubs running", 2023-11-28T11:10:33.014 INFO:teuthology.orchestra.run.smithi156.stdout: "scrubs": {} 2023-11-28T11:10:33.014 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:10:33.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71a2e71700 1 -- 172.21.15.156:0/2289743974 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71800682f0 msgr2=0x7f718006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71a2e71700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71800682f0 0x7f718006a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f719c071710 tx=0x7f7188009040 comp rx=0 tx=0).stop 2023-11-28T11:10:33.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71a2e71700 1 -- 172.21.15.156:0/2289743974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c06dee0 msgr2=0x7f719c1f1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.002+0000 7f71a2e71700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c06dee0 0x7f719c1f1fb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7190014040 tx=0x7f7190004750 comp rx=0 tx=0).stop 2023-11-28T11:10:33.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.003+0000 7f71a2e71700 1 -- 172.21.15.156:0/2289743974 shutdown_connections 2023-11-28T11:10:33.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.003+0000 7f71a2e71700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f719c160540 0x7f719c15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.003+0000 7f71a2e71700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c06dee0 0x7f719c1f1fb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.003+0000 7f71a2e71700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71800682f0 0x7f718006a7a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.003+0000 7f71a2e71700 1 --2- 172.21.15.156:0/2289743974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f719c071040 0x7f719c15f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.003+0000 7f71a2e71700 1 -- 172.21.15.156:0/2289743974 >> 172.21.15.156:0/2289743974 conn(0x7f719c0d0000 msgr2=0x7f719c06a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:33.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.003+0000 7f71a2e71700 1 -- 172.21.15.156:0/2289743974 shutdown_connections 2023-11-28T11:10:33.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.003+0000 7f71a2e71700 1 -- 172.21.15.156:0/2289743974 wait complete. 2023-11-28T11:10:33.016 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T11:10:33.118 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:33.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.237+0000 7f60fb36b700 1 Processor -- start 2023-11-28T11:10:33.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.238+0000 7f60fb36b700 1 -- start start 2023-11-28T11:10:33.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60fb36b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f60f406dee0 0x7f60f406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60fb36b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f60f4160540 0x7f60f4160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60fb36b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f415d060 con 0x7f60f4071040 2023-11-28T11:10:33.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60fb36b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f415d340 con 0x7f60f4160540 2023-11-28T11:10:33.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60fb36b700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f415d620 con 0x7f60f406dee0 2023-11-28T11:10:33.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60f9908700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f60f4160540 0x7f60f4160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60f9107700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f60f4071040 0x7f60f4071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49894/0 (socket says 172.21.15.156:49894) 2023-11-28T11:10:33.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60f9107700 1 -- 172.21.15.156:0/1474336100 learned_addr learned my addr 172.21.15.156:0/1474336100 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:33.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60f9908700 1 -- 172.21.15.156:0/1474336100 >> v2:172.21.15.186:3300/0 conn(0x7f60f406dee0 msgr2=0x7f60f406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:33.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60f9908700 1 --2- 172.21.15.156:0/1474336100 >> v2:172.21.15.186:3300/0 conn(0x7f60f406dee0 0x7f60f406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60f9908700 1 -- 172.21.15.156:0/1474336100 >> v1:172.21.15.156:6789/0 conn(0x7f60f4071040 legacy=0x7f60f4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60f9908700 1 -- 172.21.15.156:0/1474336100 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f415d900 con 0x7f60f4160540 2023-11-28T11:10:33.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60ebfff700 1 -- 172.21.15.156:0/1474336100 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644790929 0 0) 0x7f60f415d060 con 0x7f60f4071040 2023-11-28T11:10:33.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60f9908700 1 --2- 172.21.15.156:0/1474336100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60f4160540 0x7f60f4160980 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f60f406fb00 tx=0x7f60ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1517016fde6311a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:33.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.239+0000 7f60ebfff700 1 -- 172.21.15.156:0/1474336100 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00e070 con 0x7f60f4160540 2023-11-28T11:10:33.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.240+0000 7f60ebfff700 1 -- 172.21.15.156:0/1474336100 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec00bbd0 con 0x7f60f4160540 2023-11-28T11:10:33.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.240+0000 7f60ebfff700 1 -- 172.21.15.156:0/1474336100 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec012430 con 0x7f60f4160540 2023-11-28T11:10:33.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.240+0000 7f60fb36b700 1 -- 172.21.15.156:0/1474336100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60f4160540 msgr2=0x7f60f4160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.240+0000 7f60fb36b700 1 --2- 172.21.15.156:0/1474336100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60f4160540 0x7f60f4160980 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f60f406fb00 tx=0x7f60ec00b330 comp rx=0 tx=0).stop 2023-11-28T11:10:33.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.240+0000 7f60fb36b700 1 -- 172.21.15.156:0/1474336100 shutdown_connections 2023-11-28T11:10:33.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.240+0000 7f60fb36b700 1 --2- 172.21.15.156:0/1474336100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60f4160540 0x7f60f4160980 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.240+0000 7f60fb36b700 1 --2- 172.21.15.156:0/1474336100 >> v2:172.21.15.186:3300/0 conn(0x7f60f406dee0 0x7f60f406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.240+0000 7f60fb36b700 1 -- 172.21.15.156:0/1474336100 >> 172.21.15.156:0/1474336100 conn(0x7f60f40d0000 msgr2=0x7f60f40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:33.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.240+0000 7f60fb36b700 1 -- 172.21.15.156:0/1474336100 shutdown_connections 2023-11-28T11:10:33.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.240+0000 7f60fb36b700 1 -- 172.21.15.156:0/1474336100 wait complete. 2023-11-28T11:10:33.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.241+0000 7f60fb36b700 1 Processor -- start 2023-11-28T11:10:33.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.241+0000 7f60fb36b700 1 -- start start 2023-11-28T11:10:33.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.242+0000 7f60fb36b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60f406dee0 0x7f60f414b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.242+0000 7f60fb36b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60f4071040 0x7f60f40c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.242+0000 7f60fb36b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60f4160540 0x7f60f415f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.242+0000 7f60fb36b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40c4a70 con 0x7f60f406dee0 2023-11-28T11:10:33.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.242+0000 7f60fb36b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f414ba20 con 0x7f60f4160540 2023-11-28T11:10:33.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.242+0000 7f60fb36b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40c3e60 con 0x7f60f4071040 2023-11-28T11:10:33.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.242+0000 7f60f9107700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60f406dee0 0x7f60f414b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.242+0000 7f60f9107700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60f406dee0 0x7f60f414b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35022/0 (socket says 172.21.15.156:35022) 2023-11-28T11:10:33.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.242+0000 7f60f9107700 1 -- 172.21.15.156:0/613738420 learned_addr learned my addr 172.21.15.156:0/613738420 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:33.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.242+0000 7f60f8906700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60f4071040 0x7f60f40c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.242+0000 7f60f9908700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60f4160540 0x7f60f415f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.243+0000 7f60f9107700 1 -- 172.21.15.156:0/613738420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60f4071040 msgr2=0x7f60f40c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.243+0000 7f60f9107700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60f4071040 0x7f60f40c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.243+0000 7f60f9107700 1 -- 172.21.15.156:0/613738420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60f4160540 msgr2=0x7f60f415f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.243+0000 7f60f9107700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60f4160540 0x7f60f415f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.243+0000 7f60f9107700 1 -- 172.21.15.156:0/613738420 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec009a70 con 0x7f60f406dee0 2023-11-28T11:10:33.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.243+0000 7f60f9107700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60f406dee0 0x7f60f414b4e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f60e400c550 tx=0x7f60e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:33.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.243+0000 7f60e9ffb700 1 -- 172.21.15.156:0/613738420 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e4004c80 con 0x7f60f406dee0 2023-11-28T11:10:33.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.243+0000 7f60e9ffb700 1 -- 172.21.15.156:0/613738420 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e4004e00 con 0x7f60f406dee0 2023-11-28T11:10:33.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.243+0000 7f60e9ffb700 1 -- 172.21.15.156:0/613738420 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e4007310 con 0x7f60f406dee0 2023-11-28T11:10:33.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.243+0000 7f60fb36b700 1 -- 172.21.15.156:0/613738420 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f415ccd0 con 0x7f60f406dee0 2023-11-28T11:10:33.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.243+0000 7f60fb36b700 1 -- 172.21.15.156:0/613738420 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f406e870 con 0x7f60f406dee0 2023-11-28T11:10:33.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.244+0000 7f60fb36b700 1 -- 172.21.15.156:0/613738420 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f406a270 con 0x7f60f406dee0 2023-11-28T11:10:33.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.244+0000 7f60e9ffb700 1 -- 172.21.15.156:0/613738420 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f60e401a020 con 0x7f60f406dee0 2023-11-28T11:10:33.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.245+0000 7f60e9ffb700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60d80682f0 0x7f60d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.245+0000 7f60e9ffb700 1 -- 172.21.15.156:0/613738420 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f60e4085900 con 0x7f60f406dee0 2023-11-28T11:10:33.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.245+0000 7f60f8906700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60d80682f0 0x7f60d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.245+0000 7f60f8906700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60d80682f0 0x7f60d806a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f60f0006fd0 tx=0x7f60f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:33.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.249+0000 7f60e9ffb700 1 -- 172.21.15.156:0/613738420 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f60e4052170 con 0x7f60f406dee0 2023-11-28T11:10:33.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.471+0000 7f60fb36b700 1 -- 172.21.15.156:0/613738420 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60f40c6d90 con 0x7f60f406dee0 2023-11-28T11:10:33.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.472+0000 7f60e9ffb700 1 -- 172.21.15.156:0/613738420 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f60e4059250 con 0x7f60f406dee0 2023-11-28T11:10:33.473 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:33.473 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/365552946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":365552946},{"type":"v1","addr":"172.21.15.186:6834","nonce":365552946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1918325985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":1918325985},{"type":"v1","addr":"172.21.15.156:6837","nonce":1918325985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/2889641252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2889641252},{"type":"v1","addr":"172.21.15.156:6836","nonce":2889641252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:54.274258+0000","modified":"2023-11-28T11:09:57.285496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3832587641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3832587641},{"type":"v1","addr":"172.21.15.186:6835","nonce":3832587641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:10:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.473+0000 7f60fb36b700 1 -- 172.21.15.156:0/613738420 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60d80682f0 msgr2=0x7f60d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.473+0000 7f60fb36b700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60d80682f0 0x7f60d806a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f60f0006fd0 tx=0x7f60f0008040 comp rx=0 tx=0).stop 2023-11-28T11:10:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.473+0000 7f60fb36b700 1 -- 172.21.15.156:0/613738420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60f406dee0 msgr2=0x7f60f414b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.473+0000 7f60fb36b700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60f406dee0 0x7f60f414b4e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f60e400c550 tx=0x7f60e400c910 comp rx=0 tx=0).stop 2023-11-28T11:10:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.474+0000 7f60fb36b700 1 -- 172.21.15.156:0/613738420 shutdown_connections 2023-11-28T11:10:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.474+0000 7f60fb36b700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60f4160540 0x7f60f415f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.474+0000 7f60fb36b700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60f406dee0 0x7f60f414b4e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.474+0000 7f60fb36b700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60d80682f0 0x7f60d806a7a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.474+0000 7f60fb36b700 1 --2- 172.21.15.156:0/613738420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60f4071040 0x7f60f40c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.474+0000 7f60fb36b700 1 -- 172.21.15.156:0/613738420 >> 172.21.15.156:0/613738420 conn(0x7f60f40d0000 msgr2=0x7f60f406a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:33.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.474+0000 7f60fb36b700 1 -- 172.21.15.156:0/613738420 shutdown_connections 2023-11-28T11:10:33.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.474+0000 7f60fb36b700 1 -- 172.21.15.156:0/613738420 wait complete. 2023-11-28T11:10:33.476 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T11:10:33.487 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:10:33.643 INFO:teuthology.orchestra.run.smithi186.stdout:{"message":"","return_code":0} 2023-11-28T11:10:33.644 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:10:33.644 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:10:33.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.687+0000 7f28a4a44080 1 Processor -- start 2023-11-28T11:10:33.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.687+0000 7f28a4a44080 1 -- start start 2023-11-28T11:10:33.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.687+0000 7f28a4a44080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556eb73a99d0 0x556eb738df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.687+0000 7f28a4a44080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb7437450 0x556eb7435260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.687+0000 7f28a4a44080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556eb743bc10 0x556eb7438820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.687+0000 7f28a4a44080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb743a770 con 0x556eb7437450 2023-11-28T11:10:33.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.687+0000 7f28a4a44080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb738e900 con 0x556eb73a99d0 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.687+0000 7f28a4a44080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x556eb7434e10 con 0x556eb743bc10 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f289a346700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb7437450 0x556eb7435260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f289a346700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb7437450 0x556eb7435260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35036/0 (socket says 172.21.15.156:35036) 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f289a346700 1 -- 172.21.15.156:0/3172234534 learned_addr learned my addr 172.21.15.156:0/3172234534 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f289ab47700 1 --2- 172.21.15.156:0/3172234534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556eb73a99d0 0x556eb738df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f289b348700 1 --2- 172.21.15.156:0/3172234534 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556eb743bc10 0x556eb7438820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f289a346700 1 -- 172.21.15.156:0/3172234534 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556eb743bc10 msgr2=0x556eb7438820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f289a346700 1 --2- 172.21.15.156:0/3172234534 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556eb743bc10 0x556eb7438820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f289a346700 1 -- 172.21.15.156:0/3172234534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556eb73a99d0 msgr2=0x556eb738df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f289a346700 1 --2- 172.21.15.156:0/3172234534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556eb73a99d0 0x556eb738df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f289a346700 1 -- 172.21.15.156:0/3172234534 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289400a1c0 con 0x556eb7437450 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f289a346700 1 --2- 172.21.15.156:0/3172234534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb7437450 0x556eb7435260 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f288c00a770 tx=0x7f288c00aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d5d6d26abe33ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f2899344700 1 -- 172.21.15.156:0/3172234534 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c00f070 con 0x556eb7437450 2023-11-28T11:10:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f2899344700 1 -- 172.21.15.156:0/3172234534 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c007600 con 0x556eb7437450 2023-11-28T11:10:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f2899344700 1 -- 172.21.15.156:0/3172234534 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c007970 con 0x556eb7437450 2023-11-28T11:10:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f28a4a44080 1 -- 172.21.15.156:0/3172234534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb7437450 msgr2=0x556eb7435260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.688+0000 7f28a4a44080 1 --2- 172.21.15.156:0/3172234534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb7437450 0x556eb7435260 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f288c00a770 tx=0x7f288c00aa80 comp rx=0 tx=0).stop 2023-11-28T11:10:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.689+0000 7f28a4a44080 1 -- 172.21.15.156:0/3172234534 shutdown_connections 2023-11-28T11:10:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.689+0000 7f28a4a44080 1 --2- 172.21.15.156:0/3172234534 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556eb743bc10 0x556eb7438820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.689+0000 7f28a4a44080 1 --2- 172.21.15.156:0/3172234534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556eb73a99d0 0x556eb738df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.689+0000 7f28a4a44080 1 --2- 172.21.15.156:0/3172234534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb7437450 0x556eb7435260 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.689+0000 7f28a4a44080 1 -- 172.21.15.156:0/3172234534 >> 172.21.15.156:0/3172234534 conn(0x556eb738bb60 msgr2=0x556eb74367e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.689+0000 7f28a4a44080 1 -- 172.21.15.156:0/3172234534 shutdown_connections 2023-11-28T11:10:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.689+0000 7f28a4a44080 1 -- 172.21.15.156:0/3172234534 wait complete. 2023-11-28T11:10:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.689+0000 7f28a4a44080 1 Processor -- start 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.689+0000 7f28a4a44080 1 -- start start 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f28a4a44080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556eb73a99d0 0x556eb73a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f28a4a44080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556eb7437450 0x556eb742d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f28a4a44080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb743bc10 0x556eb738cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f28a4a44080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb738b120 con 0x556eb743bc10 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f28a4a44080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb742dcb0 con 0x556eb73a99d0 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f28a4a44080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x556eb738e260 con 0x556eb7437450 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f289b348700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb743bc10 0x556eb738cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f289b348700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb743bc10 0x556eb738cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35044/0 (socket says 172.21.15.156:35044) 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f289b348700 1 -- 172.21.15.156:0/1829750209 learned_addr learned my addr 172.21.15.156:0/1829750209 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f289a346700 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556eb7437450 0x556eb742d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f289ab47700 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556eb73a99d0 0x556eb73a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f289b348700 1 -- 172.21.15.156:0/1829750209 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556eb7437450 msgr2=0x556eb742d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f289b348700 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556eb7437450 0x556eb742d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f289b348700 1 -- 172.21.15.156:0/1829750209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556eb73a99d0 msgr2=0x556eb73a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f289b348700 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556eb73a99d0 0x556eb73a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f289b348700 1 -- 172.21.15.156:0/1829750209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c00a420 con 0x556eb743bc10 2023-11-28T11:10:33.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.690+0000 7f289b348700 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb743bc10 0x556eb738cea0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2894003000 tx=0x7f2894003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:33.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.691+0000 7f2893fff700 1 -- 172.21.15.156:0/1829750209 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894010440 con 0x556eb743bc10 2023-11-28T11:10:33.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.691+0000 7f2893fff700 1 -- 172.21.15.156:0/1829750209 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2894009030 con 0x556eb743bc10 2023-11-28T11:10:33.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.691+0000 7f28a4a44080 1 -- 172.21.15.156:0/1829750209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556eb72cc4f0 con 0x556eb743bc10 2023-11-28T11:10:33.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.691+0000 7f2893fff700 1 -- 172.21.15.156:0/1829750209 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894010800 con 0x556eb743bc10 2023-11-28T11:10:33.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.691+0000 7f28a4a44080 1 -- 172.21.15.156:0/1829750209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556eb72b2120 con 0x556eb743bc10 2023-11-28T11:10:33.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.692+0000 7f2893fff700 1 -- 172.21.15.156:0/1829750209 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2894033020 con 0x556eb743bc10 2023-11-28T11:10:33.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.692+0000 7f2893fff700 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f287c066c10 0x7f287c0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.692+0000 7f2893fff700 1 -- 172.21.15.156:0/1829750209 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f289408eb10 con 0x556eb743bc10 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.692+0000 7f289ab47700 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f287c066c10 0x7f287c0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f28a4a44080 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x556eb734d570 0x556eb74cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f28a4a44080 1 -- 172.21.15.156:0/1829750209 --> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51) v8 -- 0x556eb74cdb20 con 0x556eb734d570 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f289a346700 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x556eb734d570 0x556eb74cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f289ab47700 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f287c066c10 0x7f287c0690c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x556eb739dad0 tx=0x7f288400e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f289a346700 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x556eb734d570 0x556eb74cd5e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f289a346700 1 -- 172.21.15.156:0/1829750209 <== osd.1 v2:172.21.15.156:6820/103566 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f288c0079f0 con 0x556eb734d570 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f28a4a44080 1 -- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x556eb734d570 msgr2=0x556eb74cd5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f28a4a44080 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x556eb734d570 0x556eb74cd5e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f28a4a44080 1 -- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f287c066c10 msgr2=0x7f287c0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f28a4a44080 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f287c066c10 0x7f287c0690c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x556eb739dad0 tx=0x7f288400e040 comp rx=0 tx=0).stop 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f28a4a44080 1 -- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb743bc10 msgr2=0x556eb738cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:33.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f28a4a44080 1 --2- 172.21.15.156:0/1829750209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556eb743bc10 0x556eb738cea0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2894003000 tx=0x7f2894003330 comp rx=0 tx=0).stop 2023-11-28T11:10:33.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f289b348700 1 -- 172.21.15.156:0/1829750209 reap_dead start 2023-11-28T11:10:33.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f28a4a44080 1 -- 172.21.15.156:0/1829750209 shutdown_connections 2023-11-28T11:10:33.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f28a4a44080 1 -- 172.21.15.156:0/1829750209 >> 172.21.15.156:0/1829750209 conn(0x556eb738bb60 msgr2=0x556eb73a0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:33.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.693+0000 7f28a4a44080 1 -- 172.21.15.156:0/1829750209 shutdown_connections 2023-11-28T11:10:33.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:33.694+0000 7f28a4a44080 1 -- 172.21.15.156:0/1829750209 wait complete. 2023-11-28T11:10:33.701 INFO:teuthology.orchestra.run.smithi156.stdout:Yalphaparent_a 2023-11-28T11:10:33.701 DEBUG:teuthology.orchestra.run.smithi156:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:10:33.756 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:10:33.756 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 1099511627777, 2023-11-28T11:10:33.756 INFO:teuthology.orchestra.run.smithi156.stdout: "ancestors": [ 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: "dirino": 1099511627776, 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: "dname": "alpha", 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: "version": 2 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: "dirino": 1, 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: "dname": "parent_a", 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: "version": 4 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: "pool": 8, 2023-11-28T11:10:33.757 INFO:teuthology.orchestra.run.smithi156.stdout: "old_pools": [] 2023-11-28T11:10:33.758 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:10:33.758 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:33.759 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:10:33.759 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:10:33.840 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:10:33.886 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:10:33.886 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:10:33.886 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:33.904 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:10:33.904 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:33.963 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:34.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.122+0000 7f5b3bdf7700 1 Processor -- start 2023-11-28T11:10:34.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.123+0000 7f5b3bdf7700 1 -- start start 2023-11-28T11:10:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.123+0000 7f5b3bdf7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b3415cef0 0x7f5b340af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.123+0000 7f5b3bdf7700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5b340cbdf0 0x7f5b3406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.123+0000 7f5b3bdf7700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b340cc5f0 con 0x7f5b3406b8c0 2023-11-28T11:10:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.123+0000 7f5b3bdf7700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3406bd40 con 0x7f5b340cbdf0 2023-11-28T11:10:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.123+0000 7f5b3bdf7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3415d470 con 0x7f5b3415cef0 2023-11-28T11:10:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b3a394700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5b3406b8c0 0x7f5b34071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49930/0 (socket says 172.21.15.156:49930) 2023-11-28T11:10:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b3a394700 1 -- 172.21.15.156:0/2696134131 learned_addr learned my addr 172.21.15.156:0/2696134131 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b39392700 1 --2- 172.21.15.156:0/2696134131 >> v2:172.21.15.186:3300/0 conn(0x7f5b340cbdf0 0x7f5b3406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b39b93700 1 --2- 172.21.15.156:0/2696134131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b3415cef0 0x7f5b340af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955211275 0 0) 0x7f5b340cc5f0 con 0x7f5b3406b8c0 2023-11-28T11:10:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b200035b0 con 0x7f5b3406b8c0 2023-11-28T11:10:34.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3549084680 0 0) 0x7f5b200035b0 con 0x7f5b3406b8c0 2023-11-28T11:10:34.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b3415cef0 msgr2=0x7f5b340af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 --2- 172.21.15.156:0/2696134131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b3415cef0 0x7f5b340af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 >> v2:172.21.15.186:3300/0 conn(0x7f5b340cbdf0 msgr2=0x7f5b3406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 --2- 172.21.15.156:0/2696134131 >> v2:172.21.15.186:3300/0 conn(0x7f5b340cbdf0 0x7f5b3406b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3415d750 con 0x7f5b3406b8c0 2023-11-28T11:10:34.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f5b2c0019f0 con 0x7f5b3406b8c0 2023-11-28T11:10:34.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b39392700 1 --2- 172.21.15.156:0/2696134131 >> v2:172.21.15.186:3300/0 conn(0x7f5b340cbdf0 0x7f5b3406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:34.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 >> v1:172.21.15.156:6789/0 conn(0x7f5b3406b8c0 legacy=0x7f5b34071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 --2- 172.21.15.156:0/2696134131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b3415cef0 0x7f5b20003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 --2- 172.21.15.156:0/2696134131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20004030 0x7f5b20004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 --2- 172.21.15.156:0/2696134131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b20004d90 0x7f5b20005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3406bd40 con 0x7f5b20004030 2023-11-28T11:10:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3415d470 con 0x7f5b20004d90 2023-11-28T11:10:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2000bb10 con 0x7f5b3415cef0 2023-11-28T11:10:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b39392700 1 --2- 172.21.15.156:0/2696134131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20004030 0x7f5b20004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b39b93700 1 --2- 172.21.15.156:0/2696134131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b3415cef0 0x7f5b20003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b3a394700 1 --2- 172.21.15.156:0/2696134131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b20004d90 0x7f5b20005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b39392700 1 -- 172.21.15.156:0/2696134131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b3415cef0 msgr2=0x7f5b20003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.124+0000 7f5b39392700 1 --2- 172.21.15.156:0/2696134131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b3415cef0 0x7f5b20003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b39392700 1 -- 172.21.15.156:0/2696134131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b20004d90 msgr2=0x7f5b20005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b39392700 1 --2- 172.21.15.156:0/2696134131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b20004d90 0x7f5b20005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b39392700 1 -- 172.21.15.156:0/2696134131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b200035b0 con 0x7f5b20004030 2023-11-28T11:10:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b39392700 1 --2- 172.21.15.156:0/2696134131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20004030 0x7f5b20004850 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009950 tx=0x7f5b20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3aa560b99488de6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3000c070 con 0x7f5b20004030 2023-11-28T11:10:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b30013610 con 0x7f5b20004030 2023-11-28T11:10:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b38b91700 1 -- 172.21.15.156:0/2696134131 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30013980 con 0x7f5b20004030 2023-11-28T11:10:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/2696134131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20004030 msgr2=0x7f5b20004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b3bdf7700 1 --2- 172.21.15.156:0/2696134131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20004030 0x7f5b20004850 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009950 tx=0x7f5b20005880 comp rx=0 tx=0).stop 2023-11-28T11:10:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b3a394700 1 -- 172.21.15.156:0/2696134131 reap_dead start 2023-11-28T11:10:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/2696134131 shutdown_connections 2023-11-28T11:10:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/2696134131 >> 172.21.15.156:0/2696134131 conn(0x7f5b340d0000 msgr2=0x7f5b340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/2696134131 shutdown_connections 2023-11-28T11:10:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.125+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/2696134131 wait complete. 2023-11-28T11:10:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.127+0000 7f5b3bdf7700 1 Processor -- start 2023-11-28T11:10:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.127+0000 7f5b3bdf7700 1 -- start start 2023-11-28T11:10:34.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.127+0000 7f5b3bdf7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20004030 0x7f5b340c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.127+0000 7f5b3bdf7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b341598c0 0x7f5b340ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.127+0000 7f5b3bdf7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b340b5040 0x7f5b340b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.127+0000 7f5b3bdf7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b340cafa0 con 0x7f5b20004030 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.127+0000 7f5b3bdf7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b340b59d0 con 0x7f5b340b5040 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.127+0000 7f5b3bdf7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b340c7ba0 con 0x7f5b341598c0 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.127+0000 7f5b39b93700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20004030 0x7f5b340c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.127+0000 7f5b39b93700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20004030 0x7f5b340c6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35060/0 (socket says 172.21.15.156:35060) 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b39b93700 1 -- 172.21.15.156:0/3630188710 learned_addr learned my addr 172.21.15.156:0/3630188710 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b39392700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b341598c0 0x7f5b340ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b3a394700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b340b5040 0x7f5b340b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b39b93700 1 -- 172.21.15.156:0/3630188710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b341598c0 msgr2=0x7f5b340ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b39b93700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b341598c0 0x7f5b340ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b39b93700 1 -- 172.21.15.156:0/3630188710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b340b5040 msgr2=0x7f5b340b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b39b93700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b340b5040 0x7f5b340b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b39b93700 1 -- 172.21.15.156:0/3630188710 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3415dd10 con 0x7f5b20004030 2023-11-28T11:10:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b39392700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b341598c0 0x7f5b340ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b3a394700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b340b5040 0x7f5b340b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b39b93700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20004030 0x7f5b340c6a00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5b240116c0 tx=0x7f5b24011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b2a7fc700 1 -- 172.21.15.156:0/3630188710 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2400c070 con 0x7f5b20004030 2023-11-28T11:10:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b2a7fc700 1 -- 172.21.15.156:0/3630188710 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b24015870 con 0x7f5b20004030 2023-11-28T11:10:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/3630188710 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3415da30 con 0x7f5b20004030 2023-11-28T11:10:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b2a7fc700 1 -- 172.21.15.156:0/3630188710 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24015c20 con 0x7f5b20004030 2023-11-28T11:10:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.128+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/3630188710 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3415dff0 con 0x7f5b20004030 2023-11-28T11:10:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.129+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/3630188710 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b340d0e60 con 0x7f5b20004030 2023-11-28T11:10:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.129+0000 7f5b2a7fc700 1 -- 172.21.15.156:0/3630188710 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5b24025070 con 0x7f5b20004030 2023-11-28T11:10:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.130+0000 7f5b2a7fc700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5b180682f0 0x7f5b1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.130+0000 7f5b2a7fc700 1 -- 172.21.15.156:0/3630188710 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5b24085e40 con 0x7f5b20004030 2023-11-28T11:10:34.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.130+0000 7f5b39392700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5b180682f0 0x7f5b1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.130+0000 7f5b39392700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5b180682f0 0x7f5b1806a7a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5b30016040 tx=0x7f5b3001e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.134+0000 7f5b2a7fc700 1 -- 172.21.15.156:0/3630188710 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5b24052660 con 0x7f5b20004030 2023-11-28T11:10:34.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.323+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/3630188710 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b3415bd40 con 0x7f5b20004030 2023-11-28T11:10:34.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.324+0000 7f5b2a7fc700 1 -- 172.21.15.156:0/3630188710 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18066 (secure 0 0 0) 0x7f5b24023090 con 0x7f5b20004030 2023-11-28T11:10:34.326 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:34.326 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":51,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:10:02.303530+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:52.331853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:53.232328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.2d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:34.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.326+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/3630188710 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5b180682f0 msgr2=0x7f5b1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.326+0000 7f5b3bdf7700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5b180682f0 0x7f5b1806a7a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5b30016040 tx=0x7f5b3001e040 comp rx=0 tx=0).stop 2023-11-28T11:10:34.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.326+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/3630188710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20004030 msgr2=0x7f5b340c6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.326+0000 7f5b3bdf7700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20004030 0x7f5b340c6a00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5b240116c0 tx=0x7f5b24011a80 comp rx=0 tx=0).stop 2023-11-28T11:10:34.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.326+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/3630188710 shutdown_connections 2023-11-28T11:10:34.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.326+0000 7f5b3bdf7700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b340b5040 0x7f5b340b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.326+0000 7f5b3bdf7700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20004030 0x7f5b340c6a00 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.326+0000 7f5b3bdf7700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5b180682f0 0x7f5b1806a7a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.326+0000 7f5b3bdf7700 1 --2- 172.21.15.156:0/3630188710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b341598c0 0x7f5b340ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.326+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/3630188710 >> 172.21.15.156:0/3630188710 conn(0x7f5b340d0000 msgr2=0x7f5b3406e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.326+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/3630188710 shutdown_connections 2023-11-28T11:10:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.326+0000 7f5b3bdf7700 1 -- 172.21.15.156:0/3630188710 wait complete. 2023-11-28T11:10:34.340 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:09:52.331853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:09:53.232328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:10:34.340 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:10:34.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.462+0000 7f8dcb5ed700 1 Processor -- start 2023-11-28T11:10:34.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.463+0000 7f8dcb5ed700 1 -- start start 2023-11-28T11:10:34.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.463+0000 7f8dcb5ed700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8dc4159490 0x7f8dc40b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.463+0000 7f8dcb5ed700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8dc40cdc80 0x7f8dc40b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.463+0000 7f8dcb5ed700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc40ccc00 con 0x7f8dc4159860 2023-11-28T11:10:34.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.463+0000 7f8dcb5ed700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc40ccec0 con 0x7f8dc4159490 2023-11-28T11:10:34.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.463+0000 7f8dcb5ed700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc40cd1a0 con 0x7f8dc40cdc80 2023-11-28T11:10:34.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.463+0000 7f8dc9389700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8dc4159490 0x7f8dc40b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.464+0000 7f8dc9389700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8dc4159490 0x7f8dc40b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35068/0 (socket says 172.21.15.156:35068) 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.464+0000 7f8dc9389700 1 -- 172.21.15.156:0/3965039205 learned_addr learned my addr 172.21.15.156:0/3965039205 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.464+0000 7f8dc9b8a700 1 --2- 172.21.15.156:0/3965039205 >> v2:172.21.15.186:3300/0 conn(0x7f8dc40cdc80 0x7f8dc40b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.464+0000 7f8dc9389700 1 -- 172.21.15.156:0/3965039205 >> v2:172.21.15.186:3300/0 conn(0x7f8dc40cdc80 msgr2=0x7f8dc40b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.464+0000 7f8dc9389700 1 --2- 172.21.15.156:0/3965039205 >> v2:172.21.15.186:3300/0 conn(0x7f8dc40cdc80 0x7f8dc40b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.464+0000 7f8dc9389700 1 -- 172.21.15.156:0/3965039205 >> v1:172.21.15.186:6789/0 conn(0x7f8dc4159860 legacy=0x7f8dc40c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.464+0000 7f8dc9389700 1 -- 172.21.15.156:0/3965039205 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc40cd480 con 0x7f8dc4159490 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.464+0000 7f8dc9389700 1 --2- 172.21.15.156:0/3965039205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc4159490 0x7f8dc40b0670 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40c5be0 tx=0x7f8db4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5afe91dc927285a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.464+0000 7f8dbbfff700 1 -- 172.21.15.156:0/3965039205 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db4004230 con 0x7f8dc4159490 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.464+0000 7f8dbbfff700 1 -- 172.21.15.156:0/3965039205 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db4004390 con 0x7f8dc4159490 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.464+0000 7f8dbbfff700 1 -- 172.21.15.156:0/3965039205 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db40036a0 con 0x7f8dc4159490 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.465+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/3965039205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc4159490 msgr2=0x7f8dc40b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.465+0000 7f8dcb5ed700 1 --2- 172.21.15.156:0/3965039205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc4159490 0x7f8dc40b0670 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40c5be0 tx=0x7f8db4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.465+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/3965039205 shutdown_connections 2023-11-28T11:10:34.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.465+0000 7f8dcb5ed700 1 --2- 172.21.15.156:0/3965039205 >> v2:172.21.15.186:3300/0 conn(0x7f8dc40cdc80 0x7f8dc40b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.465+0000 7f8dcb5ed700 1 --2- 172.21.15.156:0/3965039205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc4159490 0x7f8dc40b0670 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.465+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/3965039205 >> 172.21.15.156:0/3965039205 conn(0x7f8dc40cb670 msgr2=0x7f8dc40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.465+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/3965039205 shutdown_connections 2023-11-28T11:10:34.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.465+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/3965039205 wait complete. 2023-11-28T11:10:34.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.466+0000 7f8dcb5ed700 1 Processor -- start 2023-11-28T11:10:34.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.466+0000 7f8dcb5ed700 1 -- start start 2023-11-28T11:10:34.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dcb5ed700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8dc40cdc80 0x7f8dc40c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dcb5ed700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dc4159490 0x7f8dc40b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dcb5ed700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc40c1060 0x7f8dc40c14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dcb5ed700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc40c78a0 con 0x7f8dc40c1060 2023-11-28T11:10:34.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dcb5ed700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc40c7cc0 con 0x7f8dc40cdc80 2023-11-28T11:10:34.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dcb5ed700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc40c6c20 con 0x7f8dc4159490 2023-11-28T11:10:34.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dc9b8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc40c1060 0x7f8dc40c14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dc9b8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc40c1060 0x7f8dc40c14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35076/0 (socket says 172.21.15.156:35076) 2023-11-28T11:10:34.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dc9b8a700 1 -- 172.21.15.156:0/1451448919 learned_addr learned my addr 172.21.15.156:0/1451448919 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dc9389700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8dc40cdc80 0x7f8dc40c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dc8b88700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dc4159490 0x7f8dc40b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dc9b8a700 1 -- 172.21.15.156:0/1451448919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dc4159490 msgr2=0x7f8dc40b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dc9b8a700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dc4159490 0x7f8dc40b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dc9b8a700 1 -- 172.21.15.156:0/1451448919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8dc40cdc80 msgr2=0x7f8dc40c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dc9b8a700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8dc40cdc80 0x7f8dc40c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dc9b8a700 1 -- 172.21.15.156:0/1451448919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db4009550 con 0x7f8dc40c1060 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8dc9b8a700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc40c1060 0x7f8dc40c14d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc007e40 tx=0x7f8dbc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.467+0000 7f8db9ffb700 1 -- 172.21.15.156:0/1451448919 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00ed30 con 0x7f8dc40c1060 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.468+0000 7f8db9ffb700 1 -- 172.21.15.156:0/1451448919 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc014420 con 0x7f8dc40c1060 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.468+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/1451448919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc40c9bd0 con 0x7f8dc40c1060 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.468+0000 7f8db9ffb700 1 -- 172.21.15.156:0/1451448919 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc0147d0 con 0x7f8dc40c1060 2023-11-28T11:10:34.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.468+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/1451448919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc40cd760 con 0x7f8dc40c1060 2023-11-28T11:10:34.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.469+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/1451448919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc40c9d60 con 0x7f8dc40c1060 2023-11-28T11:10:34.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.469+0000 7f8db9ffb700 1 -- 172.21.15.156:0/1451448919 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8dbc012050 con 0x7f8dc40c1060 2023-11-28T11:10:34.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.469+0000 7f8db9ffb700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8da80682f0 0x7f8da806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.469+0000 7f8db9ffb700 1 -- 172.21.15.156:0/1451448919 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8dbc0861f0 con 0x7f8dc40c1060 2023-11-28T11:10:34.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.469+0000 7f8dc9389700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8da80682f0 0x7f8da806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.470+0000 7f8dc9389700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8da80682f0 0x7f8da806a7a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009130 tx=0x7f8db4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.474+0000 7f8db9ffb700 1 -- 172.21.15.156:0/1451448919 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8dbc052a10 con 0x7f8dc40c1060 2023-11-28T11:10:34.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.656+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/1451448919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8dc40745c0 con 0x7f8dc40c1060 2023-11-28T11:10:34.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.656+0000 7f8db9ffb700 1 -- 172.21.15.156:0/1451448919 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8dbc056030 con 0x7f8dc40c1060 2023-11-28T11:10:34.658 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:10:34.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.659+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/1451448919 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8da80682f0 msgr2=0x7f8da806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.659+0000 7f8dcb5ed700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8da80682f0 0x7f8da806a7a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009130 tx=0x7f8db4008030 comp rx=0 tx=0).stop 2023-11-28T11:10:34.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.659+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/1451448919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc40c1060 msgr2=0x7f8dc40c14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.659+0000 7f8dcb5ed700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc40c1060 0x7f8dc40c14d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc007e40 tx=0x7f8dbc00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.659+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/1451448919 shutdown_connections 2023-11-28T11:10:34.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.659+0000 7f8dcb5ed700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc40c1060 0x7f8dc40c14d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.659+0000 7f8dcb5ed700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8dc40cdc80 0x7f8dc40c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.659+0000 7f8dcb5ed700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f8da80682f0 0x7f8da806a7a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.659+0000 7f8dcb5ed700 1 --2- 172.21.15.156:0/1451448919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dc4159490 0x7f8dc40b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.659+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/1451448919 >> 172.21.15.156:0/1451448919 conn(0x7f8dc40cb670 msgr2=0x7f8dc406fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.659+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/1451448919 shutdown_connections 2023-11-28T11:10:34.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.659+0000 7f8dcb5ed700 1 -- 172.21.15.156:0/1451448919 wait complete. 2023-11-28T11:10:34.672 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:10:34.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.794+0000 7fd234285700 1 Processor -- start 2023-11-28T11:10:34.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.794+0000 7fd234285700 1 -- start start 2023-11-28T11:10:34.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.795+0000 7fd234285700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd22c0cbdf0 0x7fd22c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.795+0000 7fd234285700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c0cc5f0 con 0x7fd22c15cef0 2023-11-28T11:10:34.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.795+0000 7fd234285700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c15d470 con 0x7fd22c06ba90 2023-11-28T11:10:34.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.795+0000 7fd234285700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22c15d750 con 0x7fd22c0cbdf0 2023-11-28T11:10:34.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.795+0000 7fd232822700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd22c0cbdf0 0x7fd22c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.795+0000 7fd232021700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd22c15cef0 0x7fd22c06b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49958/0 (socket says 172.21.15.156:49958) 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.795+0000 7fd232021700 1 -- 172.21.15.156:0/3902604863 learned_addr learned my addr 172.21.15.156:0/3902604863 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.795+0000 7fd232822700 1 -- 172.21.15.156:0/3902604863 >> v1:172.21.15.186:6789/0 conn(0x7fd22c06ba90 legacy=0x7fd22c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.795+0000 7fd232822700 1 -- 172.21.15.156:0/3902604863 >> v1:172.21.15.156:6789/0 conn(0x7fd22c15cef0 legacy=0x7fd22c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.795+0000 7fd232822700 1 -- 172.21.15.156:0/3902604863 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c15da30 con 0x7fd22c0cbdf0 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.795+0000 7fd23101f700 1 -- 172.21.15.156:0/3902604863 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677345212 0 0) 0x7fd22c0cc5f0 con 0x7fd22c15cef0 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.795+0000 7fd232822700 1 --2- 172.21.15.156:0/3902604863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd22c0cbdf0 0x7fd22c0af400 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd224009d90 tx=0x7fd22400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf9271ff50c03c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.796+0000 7fd23101f700 1 -- 172.21.15.156:0/3902604863 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22400e070 con 0x7fd22c0cbdf0 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.796+0000 7fd23101f700 1 -- 172.21.15.156:0/3902604863 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22400bbd0 con 0x7fd22c0cbdf0 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.796+0000 7fd23101f700 1 -- 172.21.15.156:0/3902604863 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd224012430 con 0x7fd22c0cbdf0 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.796+0000 7fd234285700 1 -- 172.21.15.156:0/3902604863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd22c0cbdf0 msgr2=0x7fd22c0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.796+0000 7fd234285700 1 --2- 172.21.15.156:0/3902604863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd22c0cbdf0 0x7fd22c0af400 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd224009d90 tx=0x7fd22400b330 comp rx=0 tx=0).stop 2023-11-28T11:10:34.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.796+0000 7fd234285700 1 -- 172.21.15.156:0/3902604863 shutdown_connections 2023-11-28T11:10:34.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.796+0000 7fd234285700 1 --2- 172.21.15.156:0/3902604863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd22c0cbdf0 0x7fd22c0af400 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.796+0000 7fd234285700 1 -- 172.21.15.156:0/3902604863 >> 172.21.15.156:0/3902604863 conn(0x7fd22c0d0000 msgr2=0x7fd22c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.796+0000 7fd234285700 1 -- 172.21.15.156:0/3902604863 shutdown_connections 2023-11-28T11:10:34.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.796+0000 7fd234285700 1 -- 172.21.15.156:0/3902604863 wait complete. 2023-11-28T11:10:34.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.798+0000 7fd234285700 1 Processor -- start 2023-11-28T11:10:34.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.798+0000 7fd234285700 1 -- start start 2023-11-28T11:10:34.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd234285700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd22c06ba90 0x7fd22c1eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd234285700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd22c0cbdf0 0x7fd22c0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd234285700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd22c15cef0 0x7fd22c0c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd234285700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0c9040 con 0x7fd22c06ba90 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd234285700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0c8430 con 0x7fd22c0cbdf0 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd234285700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0c8710 con 0x7fd22c15cef0 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd232021700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd22c06ba90 0x7fd22c1eeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd232021700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd22c06ba90 0x7fd22c1eeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35084/0 (socket says 172.21.15.156:35084) 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd232021700 1 -- 172.21.15.156:0/102072775 learned_addr learned my addr 172.21.15.156:0/102072775 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd232822700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd22c15cef0 0x7fd22c0c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd231820700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd22c0cbdf0 0x7fd22c0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd232021700 1 -- 172.21.15.156:0/102072775 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd22c15cef0 msgr2=0x7fd22c0c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd232021700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd22c15cef0 0x7fd22c0c3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd232021700 1 -- 172.21.15.156:0/102072775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd22c0cbdf0 msgr2=0x7fd22c0c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd232021700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd22c0cbdf0 0x7fd22c0c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd232021700 1 -- 172.21.15.156:0/102072775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd224009a70 con 0x7fd22c06ba90 2023-11-28T11:10:34.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd231820700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd22c0cbdf0 0x7fd22c0c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:34.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd232822700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd22c15cef0 0x7fd22c0c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:34.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd232021700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd22c06ba90 0x7fd22c1eeaa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd22800c520 tx=0x7fd22800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd222ffd700 1 -- 172.21.15.156:0/102072775 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228004c80 con 0x7fd22c06ba90 2023-11-28T11:10:34.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd222ffd700 1 -- 172.21.15.156:0/102072775 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd228004e20 con 0x7fd22c06ba90 2023-11-28T11:10:34.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd234285700 1 -- 172.21.15.156:0/102072775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22c15dff0 con 0x7fd22c06ba90 2023-11-28T11:10:34.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.799+0000 7fd222ffd700 1 -- 172.21.15.156:0/102072775 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228007310 con 0x7fd22c06ba90 2023-11-28T11:10:34.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.800+0000 7fd234285700 1 -- 172.21.15.156:0/102072775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22c15dd10 con 0x7fd22c06ba90 2023-11-28T11:10:34.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.800+0000 7fd234285700 1 -- 172.21.15.156:0/102072775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22c0cad60 con 0x7fd22c06ba90 2023-11-28T11:10:34.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.801+0000 7fd222ffd700 1 -- 172.21.15.156:0/102072775 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd22801a020 con 0x7fd22c06ba90 2023-11-28T11:10:34.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.802+0000 7fd222ffd700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd2100682f0 0x7fd21006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:34.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.802+0000 7fd222ffd700 1 -- 172.21.15.156:0/102072775 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd228085ee0 con 0x7fd22c06ba90 2023-11-28T11:10:34.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.802+0000 7fd231820700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd2100682f0 0x7fd21006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:34.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.802+0000 7fd231820700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd2100682f0 0x7fd21006a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd22c1f26a0 tx=0x7fd21c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:34.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.805+0000 7fd222ffd700 1 -- 172.21.15.156:0/102072775 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd2280527a0 con 0x7fd22c06ba90 2023-11-28T11:10:34.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.989+0000 7fd234285700 1 -- 172.21.15.156:0/102072775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd22c04e720 con 0x7fd22c06ba90 2023-11-28T11:10:34.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.989+0000 7fd222ffd700 1 -- 172.21.15.156:0/102072775 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd228055dc0 con 0x7fd22c06ba90 2023-11-28T11:10:34.991 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:10:34.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.991+0000 7fd234285700 1 -- 172.21.15.156:0/102072775 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd2100682f0 msgr2=0x7fd21006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.991+0000 7fd234285700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd2100682f0 0x7fd21006a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd22c1f26a0 tx=0x7fd21c009040 comp rx=0 tx=0).stop 2023-11-28T11:10:34.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.991+0000 7fd234285700 1 -- 172.21.15.156:0/102072775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd22c06ba90 msgr2=0x7fd22c1eeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:34.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.991+0000 7fd234285700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd22c06ba90 0x7fd22c1eeaa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd22800c520 tx=0x7fd22800c8e0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.992+0000 7fd234285700 1 -- 172.21.15.156:0/102072775 shutdown_connections 2023-11-28T11:10:34.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.992+0000 7fd234285700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd2100682f0 0x7fd21006a7a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.992+0000 7fd234285700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd22c15cef0 0x7fd22c0c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.992+0000 7fd234285700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd22c06ba90 0x7fd22c1eeaa0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.992+0000 7fd234285700 1 --2- 172.21.15.156:0/102072775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd22c0cbdf0 0x7fd22c0c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:34.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.992+0000 7fd234285700 1 -- 172.21.15.156:0/102072775 >> 172.21.15.156:0/102072775 conn(0x7fd22c0d0000 msgr2=0x7fd22c0ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:34.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.992+0000 7fd234285700 1 -- 172.21.15.156:0/102072775 shutdown_connections 2023-11-28T11:10:34.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:34.992+0000 7fd234285700 1 -- 172.21.15.156:0/102072775 wait complete. 2023-11-28T11:10:35.005 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:10:35.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.128+0000 7fb509d80700 1 Processor -- start 2023-11-28T11:10:35.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.129+0000 7fb509d80700 1 -- start start 2023-11-28T11:10:35.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.129+0000 7fb509d80700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb5040cd8c0 0x7fb504140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.129+0000 7fb509d80700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb5040cdc90 0x7fb5040b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.129+0000 7fb509d80700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50413ff00 con 0x7fb5040b2e70 2023-11-28T11:10:35.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.129+0000 7fb509d80700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5041404c0 con 0x7fb5040cd8c0 2023-11-28T11:10:35.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.129+0000 7fb509d80700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50415aa80 con 0x7fb5040cdc90 2023-11-28T11:10:35.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.129+0000 7fb5037fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb5040cd8c0 0x7fb504140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.129+0000 7fb503fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb5040b2e70 0x7fb50406f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49972/0 (socket says 172.21.15.156:49972) 2023-11-28T11:10:35.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.129+0000 7fb503fff700 1 -- 172.21.15.156:0/375177061 learned_addr learned my addr 172.21.15.156:0/375177061 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:35.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.130+0000 7fb502ffd700 1 --2- 172.21.15.156:0/375177061 >> v2:172.21.15.186:3300/0 conn(0x7fb5040cdc90 0x7fb5040b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.130+0000 7fb5037fe700 1 -- 172.21.15.156:0/375177061 >> v2:172.21.15.186:3300/0 conn(0x7fb5040cdc90 msgr2=0x7fb5040b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:35.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.130+0000 7fb5037fe700 1 --2- 172.21.15.156:0/375177061 >> v2:172.21.15.186:3300/0 conn(0x7fb5040cdc90 0x7fb5040b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.130+0000 7fb5037fe700 1 -- 172.21.15.156:0/375177061 >> v1:172.21.15.156:6789/0 conn(0x7fb5040b2e70 legacy=0x7fb50406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.130+0000 7fb5027fc700 1 -- 172.21.15.156:0/375177061 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131833239 0 0) 0x7fb50413ff00 con 0x7fb5040b2e70 2023-11-28T11:10:35.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.130+0000 7fb5037fe700 1 -- 172.21.15.156:0/375177061 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5040b3310 con 0x7fb5040cd8c0 2023-11-28T11:10:35.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.130+0000 7fb5037fe700 1 --2- 172.21.15.156:0/375177061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5040cd8c0 0x7fb504140950 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb504071c40 tx=0x7fb4f4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c295ae9c57a1a52d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:35.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.130+0000 7fb5027fc700 1 -- 172.21.15.156:0/375177061 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4005e30 con 0x7fb5040cd8c0 2023-11-28T11:10:35.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.130+0000 7fb5027fc700 1 -- 172.21.15.156:0/375177061 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f4006030 con 0x7fb5040cd8c0 2023-11-28T11:10:35.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.130+0000 7fb5027fc700 1 -- 172.21.15.156:0/375177061 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40063c0 con 0x7fb5040cd8c0 2023-11-28T11:10:35.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.131+0000 7fb509d80700 1 -- 172.21.15.156:0/375177061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5040cd8c0 msgr2=0x7fb504140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.131+0000 7fb509d80700 1 --2- 172.21.15.156:0/375177061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5040cd8c0 0x7fb504140950 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb504071c40 tx=0x7fb4f4009b00 comp rx=0 tx=0).stop 2023-11-28T11:10:35.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.131+0000 7fb509d80700 1 -- 172.21.15.156:0/375177061 shutdown_connections 2023-11-28T11:10:35.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.131+0000 7fb509d80700 1 --2- 172.21.15.156:0/375177061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5040cd8c0 0x7fb504140950 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.131+0000 7fb509d80700 1 --2- 172.21.15.156:0/375177061 >> v2:172.21.15.186:3300/0 conn(0x7fb5040cdc90 0x7fb5040b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.131+0000 7fb509d80700 1 -- 172.21.15.156:0/375177061 >> 172.21.15.156:0/375177061 conn(0x7fb50406aed0 msgr2=0x7fb50406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:35.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.131+0000 7fb509d80700 1 -- 172.21.15.156:0/375177061 shutdown_connections 2023-11-28T11:10:35.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.131+0000 7fb509d80700 1 -- 172.21.15.156:0/375177061 wait complete. 2023-11-28T11:10:35.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.132+0000 7fb509d80700 1 Processor -- start 2023-11-28T11:10:35.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.132+0000 7fb509d80700 1 -- start start 2023-11-28T11:10:35.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb509d80700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5040b2e70 0x7fb50406d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb509d80700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5040cd8c0 0x7fb504153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb509d80700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5040cdc90 0x7fb5041586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb509d80700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb504158260 con 0x7fb5040cd8c0 2023-11-28T11:10:35.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb509d80700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5041544c0 con 0x7fb5040b2e70 2023-11-28T11:10:35.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb509d80700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50415d570 con 0x7fb5040cdc90 2023-11-28T11:10:35.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb502ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5040cd8c0 0x7fb504153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb502ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5040cd8c0 0x7fb504153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35098/0 (socket says 172.21.15.156:35098) 2023-11-28T11:10:35.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb502ffd700 1 -- 172.21.15.156:0/4017180782 learned_addr learned my addr 172.21.15.156:0/4017180782 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:35.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb5037fe700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5040b2e70 0x7fb50406d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb503fff700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5040cdc90 0x7fb5041586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb502ffd700 1 -- 172.21.15.156:0/4017180782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5040cdc90 msgr2=0x7fb5041586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb502ffd700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5040cdc90 0x7fb5041586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb502ffd700 1 -- 172.21.15.156:0/4017180782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5040b2e70 msgr2=0x7fb50406d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb502ffd700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5040b2e70 0x7fb50406d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb502ffd700 1 -- 172.21.15.156:0/4017180782 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4009430 con 0x7fb5040cd8c0 2023-11-28T11:10:35.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb502ffd700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5040cd8c0 0x7fb504153f80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00d7d0 tx=0x7fb4ec00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:35.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.133+0000 7fb4fbfff700 1 -- 172.21.15.156:0/4017180782 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec0125f0 con 0x7fb5040cd8c0 2023-11-28T11:10:35.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.134+0000 7fb4fbfff700 1 -- 172.21.15.156:0/4017180782 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec012750 con 0x7fb5040cd8c0 2023-11-28T11:10:35.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.134+0000 7fb509d80700 1 -- 172.21.15.156:0/4017180782 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb504140150 con 0x7fb5040cd8c0 2023-11-28T11:10:35.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.134+0000 7fb4fbfff700 1 -- 172.21.15.156:0/4017180782 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec012ae0 con 0x7fb5040cd8c0 2023-11-28T11:10:35.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.134+0000 7fb509d80700 1 -- 172.21.15.156:0/4017180782 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5040b35f0 con 0x7fb5040cd8c0 2023-11-28T11:10:35.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.135+0000 7fb509d80700 1 -- 172.21.15.156:0/4017180782 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5040ce100 con 0x7fb5040cd8c0 2023-11-28T11:10:35.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.135+0000 7fb4fbfff700 1 -- 172.21.15.156:0/4017180782 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb4ec012c40 con 0x7fb5040cd8c0 2023-11-28T11:10:35.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.135+0000 7fb4fbfff700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb4e40682f0 0x7fb4e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.135+0000 7fb4fbfff700 1 -- 172.21.15.156:0/4017180782 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb4ec085e10 con 0x7fb5040cd8c0 2023-11-28T11:10:35.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.135+0000 7fb5037fe700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb4e40682f0 0x7fb4e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.136+0000 7fb5037fe700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb4e40682f0 0x7fb4e406a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009b40 tx=0x7fb4f4008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:35.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.139+0000 7fb4fbfff700 1 -- 172.21.15.156:0/4017180782 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb4ec052630 con 0x7fb5040cd8c0 2023-11-28T11:10:35.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.321+0000 7fb509d80700 1 -- 172.21.15.156:0/4017180782 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb504068690 con 0x7fb5040cd8c0 2023-11-28T11:10:35.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.321+0000 7fb4fbfff700 1 -- 172.21.15.156:0/4017180782 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4ec012ed0 con 0x7fb5040cd8c0 2023-11-28T11:10:35.323 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:10:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.323+0000 7fb509d80700 1 -- 172.21.15.156:0/4017180782 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb4e40682f0 msgr2=0x7fb4e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.323+0000 7fb509d80700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb4e40682f0 0x7fb4e406a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009b40 tx=0x7fb4f4008070 comp rx=0 tx=0).stop 2023-11-28T11:10:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.323+0000 7fb509d80700 1 -- 172.21.15.156:0/4017180782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5040cd8c0 msgr2=0x7fb504153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.323+0000 7fb509d80700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5040cd8c0 0x7fb504153f80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00d7d0 tx=0x7fb4ec00dae0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.323+0000 7fb509d80700 1 -- 172.21.15.156:0/4017180782 shutdown_connections 2023-11-28T11:10:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.323+0000 7fb509d80700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb4e40682f0 0x7fb4e406a7a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.323+0000 7fb509d80700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5040cdc90 0x7fb5041586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.323+0000 7fb509d80700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5040b2e70 0x7fb50406d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.323+0000 7fb509d80700 1 --2- 172.21.15.156:0/4017180782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5040cd8c0 0x7fb504153f80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.323+0000 7fb509d80700 1 -- 172.21.15.156:0/4017180782 >> 172.21.15.156:0/4017180782 conn(0x7fb50406aed0 msgr2=0x7fb5040cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.323+0000 7fb509d80700 1 -- 172.21.15.156:0/4017180782 shutdown_connections 2023-11-28T11:10:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.323+0000 7fb509d80700 1 -- 172.21.15.156:0/4017180782 wait complete. 2023-11-28T11:10:35.336 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:10:35.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.460+0000 7fddcc8be700 1 Processor -- start 2023-11-28T11:10:35.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.460+0000 7fddcc8be700 1 -- start start 2023-11-28T11:10:35.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.461+0000 7fddcc8be700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fddc81658c0 0x7fddc8167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.461+0000 7fddcc8be700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc815d060 con 0x7fddc806dee0 2023-11-28T11:10:35.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.461+0000 7fddcc8be700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc815d340 con 0x7fddc8071040 2023-11-28T11:10:35.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.461+0000 7fddcc8be700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc8161550 con 0x7fddc81658c0 2023-11-28T11:10:35.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.461+0000 7fddc6d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fddc81658c0 0x7fddc8167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.461+0000 7fddc5d9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fddc806dee0 0x7fddc806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49984/0 (socket says 172.21.15.156:49984) 2023-11-28T11:10:35.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.461+0000 7fddc5d9b700 1 -- 172.21.15.156:0/1671655561 learned_addr learned my addr 172.21.15.156:0/1671655561 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:35.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.461+0000 7fddc6d9d700 1 -- 172.21.15.156:0/1671655561 >> v1:172.21.15.186:6789/0 conn(0x7fddc8071040 legacy=0x7fddc8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:35.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.461+0000 7fddc6d9d700 1 -- 172.21.15.156:0/1671655561 >> v1:172.21.15.156:6789/0 conn(0x7fddc806dee0 legacy=0x7fddc806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.461+0000 7fddc6d9d700 1 -- 172.21.15.156:0/1671655561 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc815d620 con 0x7fddc81658c0 2023-11-28T11:10:35.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.461+0000 7fddc559a700 1 -- 172.21.15.156:0/1671655561 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140919306 0 0) 0x7fddc815d060 con 0x7fddc806dee0 2023-11-28T11:10:35.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.462+0000 7fddc6d9d700 1 --2- 172.21.15.156:0/1671655561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddc81658c0 0x7fddc8167ca0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009d90 tx=0x7fddc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971f646bbf78b1c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:35.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.462+0000 7fddc559a700 1 -- 172.21.15.156:0/1671655561 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc000e070 con 0x7fddc81658c0 2023-11-28T11:10:35.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.462+0000 7fddc559a700 1 -- 172.21.15.156:0/1671655561 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc000bbd0 con 0x7fddc81658c0 2023-11-28T11:10:35.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.462+0000 7fddc559a700 1 -- 172.21.15.156:0/1671655561 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0003a20 con 0x7fddc81658c0 2023-11-28T11:10:35.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.462+0000 7fddcc8be700 1 -- 172.21.15.156:0/1671655561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddc81658c0 msgr2=0x7fddc8167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.462+0000 7fddcc8be700 1 --2- 172.21.15.156:0/1671655561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddc81658c0 0x7fddc8167ca0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009d90 tx=0x7fddc000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:35.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.462+0000 7fddcc8be700 1 -- 172.21.15.156:0/1671655561 shutdown_connections 2023-11-28T11:10:35.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.462+0000 7fddcc8be700 1 --2- 172.21.15.156:0/1671655561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddc81658c0 0x7fddc8167ca0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.462+0000 7fddcc8be700 1 -- 172.21.15.156:0/1671655561 >> 172.21.15.156:0/1671655561 conn(0x7fddc80d0000 msgr2=0x7fddc80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:35.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.462+0000 7fddcc8be700 1 -- 172.21.15.156:0/1671655561 shutdown_connections 2023-11-28T11:10:35.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.462+0000 7fddcc8be700 1 -- 172.21.15.156:0/1671655561 wait complete. 2023-11-28T11:10:35.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.464+0000 7fddcc8be700 1 Processor -- start 2023-11-28T11:10:35.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.464+0000 7fddcc8be700 1 -- start start 2023-11-28T11:10:35.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.465+0000 7fddcc8be700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc806dee0 0x7fddc80ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.465+0000 7fddcc8be700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc8071040 0x7fddc80ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.465+0000 7fddcc8be700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddc81658c0 0x7fddc80cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.465+0000 7fddcc8be700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80c4900 con 0x7fddc81658c0 2023-11-28T11:10:35.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.465+0000 7fddcc8be700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80c6350 con 0x7fddc806dee0 2023-11-28T11:10:35.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.465+0000 7fddcc8be700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80b1170 con 0x7fddc8071040 2023-11-28T11:10:35.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.465+0000 7fddc6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddc81658c0 0x7fddc80cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.465+0000 7fddc6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddc81658c0 0x7fddc80cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35114/0 (socket says 172.21.15.156:35114) 2023-11-28T11:10:35.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.465+0000 7fddc6d9d700 1 -- 172.21.15.156:0/456277581 learned_addr learned my addr 172.21.15.156:0/456277581 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:35.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.465+0000 7fddc659c700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc806dee0 0x7fddc80ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.465+0000 7fddc5d9b700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc8071040 0x7fddc80ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.466+0000 7fddc6d9d700 1 -- 172.21.15.156:0/456277581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc8071040 msgr2=0x7fddc80ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.466+0000 7fddc6d9d700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc8071040 0x7fddc80ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.466+0000 7fddc6d9d700 1 -- 172.21.15.156:0/456277581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc806dee0 msgr2=0x7fddc80ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.466+0000 7fddc6d9d700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc806dee0 0x7fddc80ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.466+0000 7fddc6d9d700 1 -- 172.21.15.156:0/456277581 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0009a70 con 0x7fddc81658c0 2023-11-28T11:10:35.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.466+0000 7fddc6d9d700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddc81658c0 0x7fddc80cd1a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fddc000b370 tx=0x7fddc0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:35.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.466+0000 7fddb6ffd700 1 -- 172.21.15.156:0/456277581 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0012630 con 0x7fddc81658c0 2023-11-28T11:10:35.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.466+0000 7fddb6ffd700 1 -- 172.21.15.156:0/456277581 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc0012790 con 0x7fddc81658c0 2023-11-28T11:10:35.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.466+0000 7fddb6ffd700 1 -- 172.21.15.156:0/456277581 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0012b40 con 0x7fddc81658c0 2023-11-28T11:10:35.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.466+0000 7fddcc8be700 1 -- 172.21.15.156:0/456277581 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc815ccd0 con 0x7fddc81658c0 2023-11-28T11:10:35.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.466+0000 7fddcc8be700 1 -- 172.21.15.156:0/456277581 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc815d900 con 0x7fddc81658c0 2023-11-28T11:10:35.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.467+0000 7fddcc8be700 1 -- 172.21.15.156:0/456277581 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc806a270 con 0x7fddc81658c0 2023-11-28T11:10:35.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.467+0000 7fddb6ffd700 1 -- 172.21.15.156:0/456277581 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fddc002b070 con 0x7fddc81658c0 2023-11-28T11:10:35.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.467+0000 7fddb6ffd700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdda80682f0 0x7fdda806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.468+0000 7fddb6ffd700 1 -- 172.21.15.156:0/456277581 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fddc0087920 con 0x7fddc81658c0 2023-11-28T11:10:35.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.468+0000 7fddc659c700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdda80682f0 0x7fdda806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.468+0000 7fddc659c700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdda80682f0 0x7fdda806a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009960 tx=0x7fddbc0090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:35.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.472+0000 7fddb6ffd700 1 -- 172.21.15.156:0/456277581 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fddc0054140 con 0x7fddc81658c0 2023-11-28T11:10:35.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.654+0000 7fddcc8be700 1 -- 172.21.15.156:0/456277581 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fddc80720a0 con 0x7fddc81658c0 2023-11-28T11:10:35.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.655+0000 7fddb6ffd700 1 -- 172.21.15.156:0/456277581 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7fddc0029020 con 0x7fddc81658c0 2023-11-28T11:10:35.656 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:10:35.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.657+0000 7fddcc8be700 1 -- 172.21.15.156:0/456277581 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdda80682f0 msgr2=0x7fdda806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.657+0000 7fddcc8be700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdda80682f0 0x7fdda806a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009960 tx=0x7fddbc0090e0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.657+0000 7fddcc8be700 1 -- 172.21.15.156:0/456277581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddc81658c0 msgr2=0x7fddc80cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.657+0000 7fddcc8be700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddc81658c0 0x7fddc80cd1a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fddc000b370 tx=0x7fddc0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.657+0000 7fddcc8be700 1 -- 172.21.15.156:0/456277581 shutdown_connections 2023-11-28T11:10:35.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.657+0000 7fddcc8be700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddc81658c0 0x7fddc80cd1a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.657+0000 7fddcc8be700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc806dee0 0x7fddc80ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.657+0000 7fddcc8be700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdda80682f0 0x7fdda806a7a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.657+0000 7fddcc8be700 1 --2- 172.21.15.156:0/456277581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc8071040 0x7fddc80ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.657+0000 7fddcc8be700 1 -- 172.21.15.156:0/456277581 >> 172.21.15.156:0/456277581 conn(0x7fddc80d0000 msgr2=0x7fddc80cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:35.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.657+0000 7fddcc8be700 1 -- 172.21.15.156:0/456277581 shutdown_connections 2023-11-28T11:10:35.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.657+0000 7fddcc8be700 1 -- 172.21.15.156:0/456277581 wait complete. 2023-11-28T11:10:35.671 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:10:35.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.792+0000 7f144ac7b700 1 Processor -- start 2023-11-28T11:10:35.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.792+0000 7f144ac7b700 1 -- start start 2023-11-28T11:10:35.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f144ac7b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f144406ba90 0x7f14440af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f144ac7b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14440cc5f0 con 0x7f14440cbdf0 2023-11-28T11:10:35.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f144ac7b700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144415d470 con 0x7f144415cef0 2023-11-28T11:10:35.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f144ac7b700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144415d750 con 0x7f144406ba90 2023-11-28T11:10:35.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f1449218700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f14440cbdf0 0x7f14440ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50000/0 (socket says 172.21.15.156:50000) 2023-11-28T11:10:35.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f1449218700 1 -- 172.21.15.156:0/2659162429 learned_addr learned my addr 172.21.15.156:0/2659162429 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:35.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f1443fff700 1 --2- 172.21.15.156:0/2659162429 >> v2:172.21.15.186:3300/0 conn(0x7f144406ba90 0x7f14440af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f14437fe700 1 -- 172.21.15.156:0/2659162429 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987690545 0 0) 0x7f14440cc5f0 con 0x7f14440cbdf0 2023-11-28T11:10:35.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f14437fe700 1 -- 172.21.15.156:0/2659162429 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14340035b0 con 0x7f14440cbdf0 2023-11-28T11:10:35.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f14437fe700 1 -- 172.21.15.156:0/2659162429 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4185190082 0 0) 0x7f14340035b0 con 0x7f14440cbdf0 2023-11-28T11:10:35.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f1443fff700 1 -- 172.21.15.156:0/2659162429 >> v1:172.21.15.186:6789/0 conn(0x7f144415cef0 legacy=0x7f144406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f1443fff700 1 -- 172.21.15.156:0/2659162429 >> v1:172.21.15.156:6789/0 conn(0x7f14440cbdf0 legacy=0x7f14440ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.793+0000 7f1443fff700 1 -- 172.21.15.156:0/2659162429 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144415da30 con 0x7f144406ba90 2023-11-28T11:10:35.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.794+0000 7f1443fff700 1 --2- 172.21.15.156:0/2659162429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f144406ba90 0x7f14440af400 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1430009840 tx=0x7f1430009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1183d12684217c3f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:35.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.794+0000 7f14437fe700 1 -- 172.21.15.156:0/2659162429 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c0019f0 con 0x7f144406ba90 2023-11-28T11:10:35.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.794+0000 7f14437fe700 1 -- 172.21.15.156:0/2659162429 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1430004030 con 0x7f144406ba90 2023-11-28T11:10:35.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.794+0000 7f14437fe700 1 -- 172.21.15.156:0/2659162429 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14300043a0 con 0x7f144406ba90 2023-11-28T11:10:35.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.794+0000 7f144ac7b700 1 -- 172.21.15.156:0/2659162429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f144406ba90 msgr2=0x7f14440af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.794+0000 7f144ac7b700 1 --2- 172.21.15.156:0/2659162429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f144406ba90 0x7f14440af400 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1430009840 tx=0x7f1430009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:35.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.794+0000 7f144ac7b700 1 -- 172.21.15.156:0/2659162429 shutdown_connections 2023-11-28T11:10:35.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.794+0000 7f144ac7b700 1 --2- 172.21.15.156:0/2659162429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f144406ba90 0x7f14440af400 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.794+0000 7f144ac7b700 1 -- 172.21.15.156:0/2659162429 >> 172.21.15.156:0/2659162429 conn(0x7f14440d0000 msgr2=0x7f14440d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:35.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.794+0000 7f144ac7b700 1 -- 172.21.15.156:0/2659162429 shutdown_connections 2023-11-28T11:10:35.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.794+0000 7f144ac7b700 1 -- 172.21.15.156:0/2659162429 wait complete. 2023-11-28T11:10:35.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.796+0000 7f144ac7b700 1 Processor -- start 2023-11-28T11:10:35.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.796+0000 7f144ac7b700 1 -- start start 2023-11-28T11:10:35.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.796+0000 7f144ac7b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f144406ba90 0x7f14441f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.796+0000 7f144ac7b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14440cbdf0 0x7f144415a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.796+0000 7f144ac7b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144415cef0 0x7f14441f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.796+0000 7f144ac7b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14441f0440 con 0x7f144406ba90 2023-11-28T11:10:35.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.796+0000 7f144ac7b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14441f09c0 con 0x7f14440cbdf0 2023-11-28T11:10:35.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.796+0000 7f144ac7b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14441f1390 con 0x7f144415cef0 2023-11-28T11:10:35.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1448a17700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f144406ba90 0x7f14441f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1448a17700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f144406ba90 0x7f14441f0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35124/0 (socket says 172.21.15.156:35124) 2023-11-28T11:10:35.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1448a17700 1 -- 172.21.15.156:0/3482818240 learned_addr learned my addr 172.21.15.156:0/3482818240 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:35.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1449218700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144415cef0 0x7f14441f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1443fff700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14440cbdf0 0x7f144415a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1448a17700 1 -- 172.21.15.156:0/3482818240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144415cef0 msgr2=0x7f14441f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1448a17700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144415cef0 0x7f14441f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1448a17700 1 -- 172.21.15.156:0/3482818240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14440cbdf0 msgr2=0x7f144415a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:35.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1448a17700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14440cbdf0 0x7f144415a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:35.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1448a17700 1 -- 172.21.15.156:0/3482818240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14300094a0 con 0x7f144406ba90 2023-11-28T11:10:35.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1443fff700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14440cbdf0 0x7f144415a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:35.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1449218700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144415cef0 0x7f14441f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:35.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f1448a17700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f144406ba90 0x7f14441f0e50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f143800c480 tx=0x7f143800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:35.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f14417fa700 1 -- 172.21.15.156:0/3482818240 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14380052d0 con 0x7f144406ba90 2023-11-28T11:10:35.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f14417fa700 1 -- 172.21.15.156:0/3482818240 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1438005430 con 0x7f144406ba90 2023-11-28T11:10:35.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f144ac7b700 1 -- 172.21.15.156:0/3482818240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144415dff0 con 0x7f144406ba90 2023-11-28T11:10:35.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f14417fa700 1 -- 172.21.15.156:0/3482818240 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14380038c0 con 0x7f144406ba90 2023-11-28T11:10:35.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.797+0000 7f144ac7b700 1 -- 172.21.15.156:0/3482818240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144415dd10 con 0x7f144406ba90 2023-11-28T11:10:35.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.798+0000 7f144ac7b700 1 -- 172.21.15.156:0/3482818240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14440cad60 con 0x7f144406ba90 2023-11-28T11:10:35.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.799+0000 7f14417fa700 1 -- 172.21.15.156:0/3482818240 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1438019030 con 0x7f144406ba90 2023-11-28T11:10:35.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.799+0000 7f14417fa700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14280682f0 0x7f142806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:35.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.799+0000 7f14417fa700 1 -- 172.21.15.156:0/3482818240 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1438085080 con 0x7f144406ba90 2023-11-28T11:10:35.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.799+0000 7f1443fff700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14280682f0 0x7f142806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:35.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.799+0000 7f1443fff700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14280682f0 0x7f142806a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14440cdca0 tx=0x7f1430008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:35.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:35.803+0000 7f14417fa700 1 -- 172.21.15.156:0/3482818240 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1438051820 con 0x7f144406ba90 2023-11-28T11:10:36.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.000+0000 7f144ac7b700 1 -- 172.21.15.156:0/3482818240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f14441f0ca0 con 0x7f144406ba90 2023-11-28T11:10:36.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.000+0000 7f14417fa700 1 -- 172.21.15.156:0/3482818240 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1438054e40 con 0x7f144406ba90 2023-11-28T11:10:36.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.003+0000 7f144ac7b700 1 -- 172.21.15.156:0/3482818240 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14280682f0 msgr2=0x7f142806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.003+0000 7f144ac7b700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14280682f0 0x7f142806a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14440cdca0 tx=0x7f1430008030 comp rx=0 tx=0).stop 2023-11-28T11:10:36.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.003+0000 7f144ac7b700 1 -- 172.21.15.156:0/3482818240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f144406ba90 msgr2=0x7f14441f0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.003+0000 7f144ac7b700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f144406ba90 0x7f14441f0e50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f143800c480 tx=0x7f143800c840 comp rx=0 tx=0).stop 2023-11-28T11:10:36.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.003+0000 7f144ac7b700 1 -- 172.21.15.156:0/3482818240 shutdown_connections 2023-11-28T11:10:36.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.003+0000 7f144ac7b700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f14280682f0 0x7f142806a7a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.003+0000 7f144ac7b700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144415cef0 0x7f14441f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.003+0000 7f144ac7b700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f144406ba90 0x7f14441f0e50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.003+0000 7f144ac7b700 1 --2- 172.21.15.156:0/3482818240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14440cbdf0 0x7f144415a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.003+0000 7f144ac7b700 1 -- 172.21.15.156:0/3482818240 >> 172.21.15.156:0/3482818240 conn(0x7f14440d0000 msgr2=0x7f144406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:36.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.003+0000 7f144ac7b700 1 -- 172.21.15.156:0/3482818240 shutdown_connections 2023-11-28T11:10:36.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.003+0000 7f144ac7b700 1 -- 172.21.15.156:0/3482818240 wait complete. 2023-11-28T11:10:36.005 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T11:10:36.016 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:10:36.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:36.017 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:36.071 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:10:36.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:36.071 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:10:36.103 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:10:36.103 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:36.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:10:36.123 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:10:36.123 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:36.161 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2550 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:10:36.161 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:10:36.162 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:36.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:10:36.182 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:36.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.321+0000 7f5642b6f700 1 Processor -- start 2023-11-28T11:10:36.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.321+0000 7f5642b6f700 1 -- start start 2023-11-28T11:10:36.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f5642b6f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f563c15cef0 0x7f563c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:36.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f5642b6f700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f563c06ba90 0x7f563c0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:36.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f5642b6f700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c0cc5f0 con 0x7f563c0cbdf0 2023-11-28T11:10:36.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f5642b6f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c15d470 con 0x7f563c15cef0 2023-11-28T11:10:36.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f5642b6f700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c15d750 con 0x7f563c06ba90 2023-11-28T11:10:36.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f564090b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f563c15cef0 0x7f563c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:36.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f564090b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f563c15cef0 0x7f563c0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35134/0 (socket says 172.21.15.156:35134) 2023-11-28T11:10:36.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f564090b700 1 -- 172.21.15.156:0/4159814174 learned_addr learned my addr 172.21.15.156:0/4159814174 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:36.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f564110c700 1 --2- 172.21.15.156:0/4159814174 >> v2:172.21.15.186:3300/0 conn(0x7f563c06ba90 0x7f563c0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:36.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f564090b700 1 -- 172.21.15.156:0/4159814174 >> v2:172.21.15.186:3300/0 conn(0x7f563c06ba90 msgr2=0x7f563c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:36.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f564090b700 1 --2- 172.21.15.156:0/4159814174 >> v2:172.21.15.186:3300/0 conn(0x7f563c06ba90 0x7f563c0ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f564090b700 1 -- 172.21.15.156:0/4159814174 >> v1:172.21.15.186:6789/0 conn(0x7f563c0cbdf0 legacy=0x7f563c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f564090b700 1 -- 172.21.15.156:0/4159814174 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c15da30 con 0x7f563c15cef0 2023-11-28T11:10:36.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.322+0000 7f564090b700 1 --2- 172.21.15.156:0/4159814174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c15cef0 0x7f563c0af400 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f563000bb90 tx=0x7f563000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f908e68e8a916246 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:36.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.323+0000 7f563b7fe700 1 -- 172.21.15.156:0/4159814174 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563000dd10 con 0x7f563c15cef0 2023-11-28T11:10:36.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.323+0000 7f563b7fe700 1 -- 172.21.15.156:0/4159814174 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563000de70 con 0x7f563c15cef0 2023-11-28T11:10:36.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.323+0000 7f563b7fe700 1 -- 172.21.15.156:0/4159814174 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630010670 con 0x7f563c15cef0 2023-11-28T11:10:36.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.323+0000 7f5642b6f700 1 -- 172.21.15.156:0/4159814174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c15cef0 msgr2=0x7f563c0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.323+0000 7f5642b6f700 1 --2- 172.21.15.156:0/4159814174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c15cef0 0x7f563c0af400 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f563000bb90 tx=0x7f563000bea0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.323+0000 7f5642b6f700 1 -- 172.21.15.156:0/4159814174 shutdown_connections 2023-11-28T11:10:36.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.323+0000 7f5642b6f700 1 --2- 172.21.15.156:0/4159814174 >> v2:172.21.15.186:3300/0 conn(0x7f563c06ba90 0x7f563c0ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.323+0000 7f5642b6f700 1 --2- 172.21.15.156:0/4159814174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c15cef0 0x7f563c0af400 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.323+0000 7f5642b6f700 1 -- 172.21.15.156:0/4159814174 >> 172.21.15.156:0/4159814174 conn(0x7f563c0d0000 msgr2=0x7f563c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:36.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.323+0000 7f5642b6f700 1 -- 172.21.15.156:0/4159814174 shutdown_connections 2023-11-28T11:10:36.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.323+0000 7f5642b6f700 1 -- 172.21.15.156:0/4159814174 wait complete. 2023-11-28T11:10:36.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.325+0000 7f5642b6f700 1 Processor -- start 2023-11-28T11:10:36.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.325+0000 7f5642b6f700 1 -- start start 2023-11-28T11:10:36.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f5642b6f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f563c06ba90 0x7f563c1494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:36.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f5642b6f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c0cbdf0 0x7f563c0b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:36.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f5642b6f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c15cef0 0x7f563c06a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:36.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f5642b6f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1499e0 con 0x7f563c15cef0 2023-11-28T11:10:36.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f5642b6f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1e4e20 con 0x7f563c06ba90 2023-11-28T11:10:36.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f5642b6f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1e5100 con 0x7f563c0cbdf0 2023-11-28T11:10:36.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f564110c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c15cef0 0x7f563c06a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:36.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f564110c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c15cef0 0x7f563c06a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35142/0 (socket says 172.21.15.156:35142) 2023-11-28T11:10:36.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f564110c700 1 -- 172.21.15.156:0/3800025995 learned_addr learned my addr 172.21.15.156:0/3800025995 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:36.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f564090b700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f563c06ba90 0x7f563c1494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:36.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f563bfff700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c0cbdf0 0x7f563c0b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:36.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f564110c700 1 -- 172.21.15.156:0/3800025995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c0cbdf0 msgr2=0x7f563c0b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f564110c700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c0cbdf0 0x7f563c0b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f564110c700 1 -- 172.21.15.156:0/3800025995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f563c06ba90 msgr2=0x7f563c1494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f564110c700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f563c06ba90 0x7f563c1494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.326+0000 7f564110c700 1 -- 172.21.15.156:0/3800025995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563000b7f0 con 0x7f563c15cef0 2023-11-28T11:10:36.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.327+0000 7f564110c700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c15cef0 0x7f563c06a970 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5634007de0 tx=0x7f563400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:36.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.327+0000 7f56397fa700 1 -- 172.21.15.156:0/3800025995 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563400ed30 con 0x7f563c15cef0 2023-11-28T11:10:36.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.327+0000 7f56397fa700 1 -- 172.21.15.156:0/3800025995 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5634014420 con 0x7f563c15cef0 2023-11-28T11:10:36.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.327+0000 7f56397fa700 1 -- 172.21.15.156:0/3800025995 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56340147d0 con 0x7f563c15cef0 2023-11-28T11:10:36.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.327+0000 7f5642b6f700 1 -- 172.21.15.156:0/3800025995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c15dff0 con 0x7f563c15cef0 2023-11-28T11:10:36.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.327+0000 7f5642b6f700 1 -- 172.21.15.156:0/3800025995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c15dd10 con 0x7f563c15cef0 2023-11-28T11:10:36.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.328+0000 7f5642b6f700 1 -- 172.21.15.156:0/3800025995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c0cab90 con 0x7f563c15cef0 2023-11-28T11:10:36.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.328+0000 7f56397fa700 1 -- 172.21.15.156:0/3800025995 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5634012050 con 0x7f563c15cef0 2023-11-28T11:10:36.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.328+0000 7f56397fa700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f56200682f0 0x7f562006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:36.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.328+0000 7f56397fa700 1 -- 172.21.15.156:0/3800025995 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5634086340 con 0x7f563c15cef0 2023-11-28T11:10:36.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.328+0000 7f564090b700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f56200682f0 0x7f562006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:36.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.329+0000 7f564090b700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f56200682f0 0x7f562006a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5630000c00 tx=0x7f5630013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:36.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.333+0000 7f56397fa700 1 -- 172.21.15.156:0/3800025995 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5634052ae0 con 0x7f563c15cef0 2023-11-28T11:10:36.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.559+0000 7f5642b6f700 1 -- 172.21.15.156:0/3800025995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f563c0d0e60 con 0x7f563c15cef0 2023-11-28T11:10:36.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.559+0000 7f56397fa700 1 -- 172.21.15.156:0/3800025995 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f5634056100 con 0x7f563c15cef0 2023-11-28T11:10:36.561 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:36.561 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/365552946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":365552946},{"type":"v1","addr":"172.21.15.186:6834","nonce":365552946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1918325985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":1918325985},{"type":"v1","addr":"172.21.15.156:6837","nonce":1918325985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/2889641252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2889641252},{"type":"v1","addr":"172.21.15.156:6836","nonce":2889641252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:54.274258+0000","modified":"2023-11-28T11:09:57.285496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3832587641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3832587641},{"type":"v1","addr":"172.21.15.186:6835","nonce":3832587641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:10:36.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.561+0000 7f5642b6f700 1 -- 172.21.15.156:0/3800025995 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f56200682f0 msgr2=0x7f562006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.561+0000 7f5642b6f700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f56200682f0 0x7f562006a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5630000c00 tx=0x7f5630013000 comp rx=0 tx=0).stop 2023-11-28T11:10:36.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.561+0000 7f5642b6f700 1 -- 172.21.15.156:0/3800025995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c15cef0 msgr2=0x7f563c06a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.561+0000 7f5642b6f700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c15cef0 0x7f563c06a970 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5634007de0 tx=0x7f563400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.562+0000 7f5642b6f700 1 -- 172.21.15.156:0/3800025995 shutdown_connections 2023-11-28T11:10:36.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.562+0000 7f5642b6f700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c15cef0 0x7f563c06a970 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.562+0000 7f5642b6f700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f563c06ba90 0x7f563c1494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.562+0000 7f5642b6f700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f56200682f0 0x7f562006a7a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.562+0000 7f5642b6f700 1 --2- 172.21.15.156:0/3800025995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c0cbdf0 0x7f563c0b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.562+0000 7f5642b6f700 1 -- 172.21.15.156:0/3800025995 >> 172.21.15.156:0/3800025995 conn(0x7f563c0d0000 msgr2=0x7f563c15c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:36.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.562+0000 7f5642b6f700 1 -- 172.21.15.156:0/3800025995 shutdown_connections 2023-11-28T11:10:36.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.562+0000 7f5642b6f700 1 -- 172.21.15.156:0/3800025995 wait complete. 2023-11-28T11:10:36.564 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T11:10:36.575 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:36.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.693+0000 7f60d829a700 1 Processor -- start 2023-11-28T11:10:36.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.693+0000 7f60d829a700 1 -- start start 2023-11-28T11:10:36.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.694+0000 7f60d829a700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f60d006dee0 0x7f60d006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:36.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.694+0000 7f60d829a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f60d0160540 0x7f60d0160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:36.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.694+0000 7f60d829a700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d015d060 con 0x7f60d0071040 2023-11-28T11:10:36.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.694+0000 7f60d829a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d015d340 con 0x7f60d0160540 2023-11-28T11:10:36.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.694+0000 7f60d829a700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d015d620 con 0x7f60d006dee0 2023-11-28T11:10:36.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.695+0000 7f60d6837700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f60d0160540 0x7f60d0160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:36.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.695+0000 7f60d6837700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f60d0160540 0x7f60d0160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35150/0 (socket says 172.21.15.156:35150) 2023-11-28T11:10:36.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.695+0000 7f60d6837700 1 -- 172.21.15.156:0/121267294 learned_addr learned my addr 172.21.15.156:0/121267294 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:36.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.695+0000 7f60d5835700 1 --2- 172.21.15.156:0/121267294 >> v2:172.21.15.186:3300/0 conn(0x7f60d006dee0 0x7f60d006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:36.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.695+0000 7f60d6837700 1 -- 172.21.15.156:0/121267294 >> v2:172.21.15.186:3300/0 conn(0x7f60d006dee0 msgr2=0x7f60d006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:36.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.695+0000 7f60d6837700 1 --2- 172.21.15.156:0/121267294 >> v2:172.21.15.186:3300/0 conn(0x7f60d006dee0 0x7f60d006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.695+0000 7f60d6837700 1 -- 172.21.15.156:0/121267294 >> v1:172.21.15.186:6789/0 conn(0x7f60d0071040 legacy=0x7f60d0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.695+0000 7f60d6837700 1 -- 172.21.15.156:0/121267294 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d015d900 con 0x7f60d0160540 2023-11-28T11:10:36.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.695+0000 7f60d6837700 1 --2- 172.21.15.156:0/121267294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d0160540 0x7f60d0160980 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f60d006fb00 tx=0x7f60c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5fa2a7f9e4c022 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:36.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.695+0000 7f60d5034700 1 -- 172.21.15.156:0/121267294 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c800e070 con 0x7f60d0160540 2023-11-28T11:10:36.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.695+0000 7f60d5034700 1 -- 172.21.15.156:0/121267294 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c800b910 con 0x7f60d0160540 2023-11-28T11:10:36.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.695+0000 7f60d5034700 1 -- 172.21.15.156:0/121267294 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c800bcc0 con 0x7f60d0160540 2023-11-28T11:10:36.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.696+0000 7f60d829a700 1 -- 172.21.15.156:0/121267294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d0160540 msgr2=0x7f60d0160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.696+0000 7f60d829a700 1 --2- 172.21.15.156:0/121267294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d0160540 0x7f60d0160980 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f60d006fb00 tx=0x7f60c800b330 comp rx=0 tx=0).stop 2023-11-28T11:10:36.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.696+0000 7f60d829a700 1 -- 172.21.15.156:0/121267294 shutdown_connections 2023-11-28T11:10:36.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.696+0000 7f60d829a700 1 --2- 172.21.15.156:0/121267294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d0160540 0x7f60d0160980 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.696+0000 7f60d829a700 1 --2- 172.21.15.156:0/121267294 >> v2:172.21.15.186:3300/0 conn(0x7f60d006dee0 0x7f60d006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.696+0000 7f60d829a700 1 -- 172.21.15.156:0/121267294 >> 172.21.15.156:0/121267294 conn(0x7f60d00d0000 msgr2=0x7f60d00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:36.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.696+0000 7f60d829a700 1 -- 172.21.15.156:0/121267294 shutdown_connections 2023-11-28T11:10:36.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.696+0000 7f60d829a700 1 -- 172.21.15.156:0/121267294 wait complete. 2023-11-28T11:10:36.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.697+0000 7f60d829a700 1 Processor -- start 2023-11-28T11:10:36.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.697+0000 7f60d829a700 1 -- start start 2023-11-28T11:10:36.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d829a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60d006dee0 0x7f60d0150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:36.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d829a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d0071040 0x7f60d00c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:36.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d829a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60d0160540 0x7f60d00b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:36.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d829a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d00c3e20 con 0x7f60d0071040 2023-11-28T11:10:36.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d829a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d00c4100 con 0x7f60d0160540 2023-11-28T11:10:36.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d829a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d00c43e0 con 0x7f60d006dee0 2023-11-28T11:10:36.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d5835700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d0071040 0x7f60d00c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:36.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d5835700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d0071040 0x7f60d00c4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35154/0 (socket says 172.21.15.156:35154) 2023-11-28T11:10:36.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d5835700 1 -- 172.21.15.156:0/2621987106 learned_addr learned my addr 172.21.15.156:0/2621987106 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:36.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d6036700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60d006dee0 0x7f60d0150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:36.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d6837700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60d0160540 0x7f60d00b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:36.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d5835700 1 -- 172.21.15.156:0/2621987106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60d006dee0 msgr2=0x7f60d0150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d5835700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60d006dee0 0x7f60d0150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d5835700 1 -- 172.21.15.156:0/2621987106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60d0160540 msgr2=0x7f60d00b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d5835700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60d0160540 0x7f60d00b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d5835700 1 -- 172.21.15.156:0/2621987106 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c8009b30 con 0x7f60d0071040 2023-11-28T11:10:36.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60d5835700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d0071040 0x7f60d00c4c60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00e870 tx=0x7f60cc00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:36.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.698+0000 7f60c6ffd700 1 -- 172.21.15.156:0/2621987106 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc009590 con 0x7f60d0071040 2023-11-28T11:10:36.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.699+0000 7f60c6ffd700 1 -- 172.21.15.156:0/2621987106 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60cc013680 con 0x7f60d0071040 2023-11-28T11:10:36.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.699+0000 7f60d829a700 1 -- 172.21.15.156:0/2621987106 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d015ccd0 con 0x7f60d0071040 2023-11-28T11:10:36.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.699+0000 7f60c6ffd700 1 -- 172.21.15.156:0/2621987106 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0139f0 con 0x7f60d0071040 2023-11-28T11:10:36.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.699+0000 7f60d829a700 1 -- 172.21.15.156:0/2621987106 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d006e870 con 0x7f60d0071040 2023-11-28T11:10:36.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.699+0000 7f60d829a700 1 -- 172.21.15.156:0/2621987106 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d006a270 con 0x7f60d0071040 2023-11-28T11:10:36.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.700+0000 7f60c6ffd700 1 -- 172.21.15.156:0/2621987106 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f60cc019030 con 0x7f60d0071040 2023-11-28T11:10:36.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.700+0000 7f60c6ffd700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60b40682f0 0x7f60b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:36.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.700+0000 7f60c6ffd700 1 -- 172.21.15.156:0/2621987106 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f60cc085990 con 0x7f60d0071040 2023-11-28T11:10:36.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.700+0000 7f60d6036700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60b40682f0 0x7f60b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:36.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.701+0000 7f60d6036700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60b40682f0 0x7f60b406a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f60c0007fd0 tx=0x7f60c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:36.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.704+0000 7f60c6ffd700 1 -- 172.21.15.156:0/2621987106 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f60cc0521b0 con 0x7f60d0071040 2023-11-28T11:10:36.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.888+0000 7f60d829a700 1 -- 172.21.15.156:0/2621987106 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60d00720a0 con 0x7f60d0071040 2023-11-28T11:10:36.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.889+0000 7f60c6ffd700 1 -- 172.21.15.156:0/2621987106 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18066 (secure 0 0 0) 0x7f60cc0557d0 con 0x7f60d0071040 2023-11-28T11:10:36.891 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:36.891 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":51,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:10:02.303530+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:52.331853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:53.232328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.2d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.891+0000 7f60d829a700 1 -- 172.21.15.156:0/2621987106 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60b40682f0 msgr2=0x7f60b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.891+0000 7f60d829a700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60b40682f0 0x7f60b406a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f60c0007fd0 tx=0x7f60c0009040 comp rx=0 tx=0).stop 2023-11-28T11:10:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.891+0000 7f60d829a700 1 -- 172.21.15.156:0/2621987106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d0071040 msgr2=0x7f60d00c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.891+0000 7f60d829a700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d0071040 0x7f60d00c4c60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00e870 tx=0x7f60cc00eb80 comp rx=0 tx=0).stop 2023-11-28T11:10:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.891+0000 7f60d829a700 1 -- 172.21.15.156:0/2621987106 shutdown_connections 2023-11-28T11:10:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.891+0000 7f60d829a700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60d0160540 0x7f60d00b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.891+0000 7f60d829a700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f60b40682f0 0x7f60b406a7a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.891+0000 7f60d829a700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60d006dee0 0x7f60d0150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.891+0000 7f60d829a700 1 --2- 172.21.15.156:0/2621987106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d0071040 0x7f60d00c4c60 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.891+0000 7f60d829a700 1 -- 172.21.15.156:0/2621987106 >> 172.21.15.156:0/2621987106 conn(0x7f60d00d0000 msgr2=0x7f60d015fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.891+0000 7f60d829a700 1 -- 172.21.15.156:0/2621987106 shutdown_connections 2023-11-28T11:10:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:36.891+0000 7f60d829a700 1 -- 172.21.15.156:0/2621987106 wait complete. 2023-11-28T11:10:36.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:09:52.331853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:09:53.232328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:10:36.905 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:10:37.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.028+0000 7efc2e7bc700 1 Processor -- start 2023-11-28T11:10:37.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.028+0000 7efc2e7bc700 1 -- start start 2023-11-28T11:10:37.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc2e7bc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc2815cef0 0x7efc280af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc2e7bc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efc280cbdf0 0x7efc2806b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc2e7bc700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc280cc5f0 con 0x7efc2806b8c0 2023-11-28T11:10:37.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc2e7bc700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2806bd40 con 0x7efc280cbdf0 2023-11-28T11:10:37.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc2e7bc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2815d470 con 0x7efc2815cef0 2023-11-28T11:10:37.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc277fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efc280cbdf0 0x7efc2806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc2cd59700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7efc2806b8c0 0x7efc28071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50042/0 (socket says 172.21.15.156:50042) 2023-11-28T11:10:37.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc2cd59700 1 -- 172.21.15.156:0/2760036008 learned_addr learned my addr 172.21.15.156:0/2760036008 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:37.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc27fff700 1 --2- 172.21.15.156:0/2760036008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc2815cef0 0x7efc280af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc277fe700 1 -- 172.21.15.156:0/2760036008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc2815cef0 msgr2=0x7efc280af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:37.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc277fe700 1 --2- 172.21.15.156:0/2760036008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc2815cef0 0x7efc280af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc277fe700 1 -- 172.21.15.156:0/2760036008 >> v1:172.21.15.156:6789/0 conn(0x7efc2806b8c0 legacy=0x7efc28071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc277fe700 1 -- 172.21.15.156:0/2760036008 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2815d750 con 0x7efc280cbdf0 2023-11-28T11:10:37.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc26ffd700 1 -- 172.21.15.156:0/2760036008 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258061230 0 0) 0x7efc280cc5f0 con 0x7efc2806b8c0 2023-11-28T11:10:37.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc27fff700 1 --2- 172.21.15.156:0/2760036008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc2815cef0 0x7efc280af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:37.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.029+0000 7efc277fe700 1 --2- 172.21.15.156:0/2760036008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc280cbdf0 0x7efc2806b380 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009840 tx=0x7efc1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e48e1740c71307d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:37.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.030+0000 7efc26ffd700 1 -- 172.21.15.156:0/2760036008 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c011830 con 0x7efc280cbdf0 2023-11-28T11:10:37.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.030+0000 7efc26ffd700 1 -- 172.21.15.156:0/2760036008 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c011990 con 0x7efc280cbdf0 2023-11-28T11:10:37.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.030+0000 7efc26ffd700 1 -- 172.21.15.156:0/2760036008 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c011d20 con 0x7efc280cbdf0 2023-11-28T11:10:37.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.030+0000 7efc2e7bc700 1 -- 172.21.15.156:0/2760036008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc280cbdf0 msgr2=0x7efc2806b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.030+0000 7efc2e7bc700 1 --2- 172.21.15.156:0/2760036008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc280cbdf0 0x7efc2806b380 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009840 tx=0x7efc1c009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:37.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.030+0000 7efc2e7bc700 1 -- 172.21.15.156:0/2760036008 shutdown_connections 2023-11-28T11:10:37.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.030+0000 7efc2e7bc700 1 --2- 172.21.15.156:0/2760036008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc2815cef0 0x7efc280af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.030+0000 7efc2e7bc700 1 --2- 172.21.15.156:0/2760036008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc280cbdf0 0x7efc2806b380 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.030+0000 7efc2e7bc700 1 -- 172.21.15.156:0/2760036008 >> 172.21.15.156:0/2760036008 conn(0x7efc280d0000 msgr2=0x7efc280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:37.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.030+0000 7efc2e7bc700 1 -- 172.21.15.156:0/2760036008 shutdown_connections 2023-11-28T11:10:37.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.030+0000 7efc2e7bc700 1 -- 172.21.15.156:0/2760036008 wait complete. 2023-11-28T11:10:37.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.031+0000 7efc2e7bc700 1 Processor -- start 2023-11-28T11:10:37.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.032+0000 7efc2e7bc700 1 -- start start 2023-11-28T11:10:37.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.032+0000 7efc2e7bc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc2806b8c0 0x7efc2806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.032+0000 7efc2e7bc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc280cbdf0 0x7efc2806dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.032+0000 7efc2e7bc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc2815cef0 0x7efc280ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.032+0000 7efc2e7bc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2809c1b0 con 0x7efc280cbdf0 2023-11-28T11:10:37.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.032+0000 7efc2e7bc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc28147dc0 con 0x7efc2815cef0 2023-11-28T11:10:37.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.032+0000 7efc2e7bc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc281480a0 con 0x7efc2806b8c0 2023-11-28T11:10:37.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.032+0000 7efc277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc280cbdf0 0x7efc2806dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.032+0000 7efc277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc280cbdf0 0x7efc2806dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35166/0 (socket says 172.21.15.156:35166) 2023-11-28T11:10:37.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.032+0000 7efc277fe700 1 -- 172.21.15.156:0/1745043960 learned_addr learned my addr 172.21.15.156:0/1745043960 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:37.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc27fff700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc2806b8c0 0x7efc2806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc2cd59700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc2815cef0 0x7efc280ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc277fe700 1 -- 172.21.15.156:0/1745043960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc2806b8c0 msgr2=0x7efc2806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc277fe700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc2806b8c0 0x7efc2806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc277fe700 1 -- 172.21.15.156:0/1745043960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc2815cef0 msgr2=0x7efc280ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc277fe700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc2815cef0 0x7efc280ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc277fe700 1 -- 172.21.15.156:0/1745043960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0094a0 con 0x7efc280cbdf0 2023-11-28T11:10:37.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc2cd59700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc2815cef0 0x7efc280ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:37.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc27fff700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc2806b8c0 0x7efc2806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:37.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc277fe700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc280cbdf0 0x7efc2806dfa0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0090b0 tx=0x7efc1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:37.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc24ff9700 1 -- 172.21.15.156:0/1745043960 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0038a0 con 0x7efc280cbdf0 2023-11-28T11:10:37.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc24ff9700 1 -- 172.21.15.156:0/1745043960 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c01f070 con 0x7efc280cbdf0 2023-11-28T11:10:37.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc24ff9700 1 -- 172.21.15.156:0/1745043960 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c003c40 con 0x7efc280cbdf0 2023-11-28T11:10:37.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc2e7bc700 1 -- 172.21.15.156:0/1745043960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2815dd10 con 0x7efc280cbdf0 2023-11-28T11:10:37.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.033+0000 7efc2e7bc700 1 -- 172.21.15.156:0/1745043960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2815da30 con 0x7efc280cbdf0 2023-11-28T11:10:37.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.034+0000 7efc2e7bc700 1 -- 172.21.15.156:0/1745043960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2815dff0 con 0x7efc280cbdf0 2023-11-28T11:10:37.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.034+0000 7efc24ff9700 1 -- 172.21.15.156:0/1745043960 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efc1c00f050 con 0x7efc280cbdf0 2023-11-28T11:10:37.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.035+0000 7efc24ff9700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7efc140682f0 0x7efc1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.035+0000 7efc27fff700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7efc140682f0 0x7efc1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.035+0000 7efc24ff9700 1 -- 172.21.15.156:0/1745043960 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7efc1c021070 con 0x7efc280cbdf0 2023-11-28T11:10:37.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.035+0000 7efc27fff700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7efc140682f0 0x7efc1406a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efc18008910 tx=0x7efc18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:37.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.039+0000 7efc24ff9700 1 -- 172.21.15.156:0/1745043960 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efc1c0540a0 con 0x7efc280cbdf0 2023-11-28T11:10:37.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.219+0000 7efc2e7bc700 1 -- 172.21.15.156:0/1745043960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc2809c2f0 con 0x7efc280cbdf0 2023-11-28T11:10:37.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.220+0000 7efc24ff9700 1 -- 172.21.15.156:0/1745043960 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc2809c2f0 con 0x7efc280cbdf0 2023-11-28T11:10:37.222 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:10:37.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.222+0000 7efc2e7bc700 1 -- 172.21.15.156:0/1745043960 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7efc140682f0 msgr2=0x7efc1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.222+0000 7efc2e7bc700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7efc140682f0 0x7efc1406a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efc18008910 tx=0x7efc18008040 comp rx=0 tx=0).stop 2023-11-28T11:10:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.222+0000 7efc2e7bc700 1 -- 172.21.15.156:0/1745043960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc280cbdf0 msgr2=0x7efc2806dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.222+0000 7efc2e7bc700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc280cbdf0 0x7efc2806dfa0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0090b0 tx=0x7efc1c004750 comp rx=0 tx=0).stop 2023-11-28T11:10:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.222+0000 7efc2e7bc700 1 -- 172.21.15.156:0/1745043960 shutdown_connections 2023-11-28T11:10:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.222+0000 7efc2e7bc700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc2815cef0 0x7efc280ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.222+0000 7efc2e7bc700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7efc140682f0 0x7efc1406a7a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.222+0000 7efc2e7bc700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc2806b8c0 0x7efc2806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.222+0000 7efc2e7bc700 1 --2- 172.21.15.156:0/1745043960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc280cbdf0 0x7efc2806dfa0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.222+0000 7efc2e7bc700 1 -- 172.21.15.156:0/1745043960 >> 172.21.15.156:0/1745043960 conn(0x7efc280d0000 msgr2=0x7efc280afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.222+0000 7efc2e7bc700 1 -- 172.21.15.156:0/1745043960 shutdown_connections 2023-11-28T11:10:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.222+0000 7efc2e7bc700 1 -- 172.21.15.156:0/1745043960 wait complete. 2023-11-28T11:10:37.235 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:10:37.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.357+0000 7f5da46ef700 1 Processor -- start 2023-11-28T11:10:37.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.357+0000 7f5da46ef700 1 -- start start 2023-11-28T11:10:37.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.358+0000 7f5da46ef700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c0cd7c0 0x7f5d9c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.358+0000 7f5da46ef700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c0d1950 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.358+0000 7f5da46ef700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c0ce140 con 0x7f5d9c161340 2023-11-28T11:10:37.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.358+0000 7f5da46ef700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0b5800 con 0x7f5d9c0cd7c0 2023-11-28T11:10:37.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da248b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5d9c15c9b0 0x7f5d9c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50052/0 (socket says 172.21.15.156:50052) 2023-11-28T11:10:37.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da248b700 1 -- 172.21.15.156:0/3470856022 learned_addr learned my addr 172.21.15.156:0/3470856022 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:37.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da2c8c700 1 --2- 172.21.15.156:0/3470856022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c0cd7c0 0x7f5d9c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da1489700 1 -- 172.21.15.156:0/3470856022 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769956729 0 0) 0x7f5d9c0d1950 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da1489700 1 -- 172.21.15.156:0/3470856022 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d880035b0 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da1489700 1 -- 172.21.15.156:0/3470856022 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804603542 0 0) 0x7f5d9c0ce140 con 0x7f5d9c161340 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da1489700 1 -- 172.21.15.156:0/3470856022 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d9c0d1950 con 0x7f5d9c161340 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da1489700 1 -- 172.21.15.156:0/3470856022 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 100534316 0 0) 0x7f5d880035b0 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da2c8c700 1 -- 172.21.15.156:0/3470856022 >> v1:172.21.15.186:6789/0 conn(0x7f5d9c161340 legacy=0x7f5d9c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da2c8c700 1 -- 172.21.15.156:0/3470856022 >> v1:172.21.15.156:6789/0 conn(0x7f5d9c15c9b0 legacy=0x7f5d9c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da2c8c700 1 -- 172.21.15.156:0/3470856022 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c0d24f0 con 0x7f5d9c0cd7c0 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da2c8c700 1 --2- 172.21.15.156:0/3470856022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c0cd7c0 0x7f5d9c0cdc00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009dc0 tx=0x7f5d9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e72916385842d36f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.359+0000 7f5da1489700 1 -- 172.21.15.156:0/3470856022 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400e070 con 0x7f5d9c0cd7c0 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.360+0000 7f5da1489700 1 -- 172.21.15.156:0/3470856022 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d94003a20 con 0x7f5d9c0cd7c0 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.360+0000 7f5da1489700 1 -- 172.21.15.156:0/3470856022 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c001670 con 0x7f5d9c0cd7c0 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.360+0000 7f5da46ef700 1 -- 172.21.15.156:0/3470856022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c0cd7c0 msgr2=0x7f5d9c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.360+0000 7f5da46ef700 1 --2- 172.21.15.156:0/3470856022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c0cd7c0 0x7f5d9c0cdc00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009dc0 tx=0x7f5d9400b330 comp rx=0 tx=0).stop 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.360+0000 7f5da46ef700 1 -- 172.21.15.156:0/3470856022 shutdown_connections 2023-11-28T11:10:37.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.360+0000 7f5da46ef700 1 --2- 172.21.15.156:0/3470856022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c0cd7c0 0x7f5d9c0cdc00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.360+0000 7f5da46ef700 1 -- 172.21.15.156:0/3470856022 >> 172.21.15.156:0/3470856022 conn(0x7f5d9c072600 msgr2=0x7f5d9c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:37.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.360+0000 7f5da46ef700 1 -- 172.21.15.156:0/3470856022 shutdown_connections 2023-11-28T11:10:37.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.360+0000 7f5da46ef700 1 -- 172.21.15.156:0/3470856022 wait complete. 2023-11-28T11:10:37.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.362+0000 7f5da46ef700 1 Processor -- start 2023-11-28T11:10:37.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.362+0000 7f5da46ef700 1 -- start start 2023-11-28T11:10:37.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da46ef700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d9c0cd7c0 0x7f5d9c0cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da46ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d9c15c9b0 0x7f5d9c1f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da46ef700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c161340 0x7f5d9c14d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da46ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c1f2660 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da46ef700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c1f6190 con 0x7f5d9c0cd7c0 2023-11-28T11:10:37.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da46ef700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c1f6410 con 0x7f5d9c161340 2023-11-28T11:10:37.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da1c8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d9c15c9b0 0x7f5d9c1f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da1c8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d9c15c9b0 0x7f5d9c1f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35182/0 (socket says 172.21.15.156:35182) 2023-11-28T11:10:37.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da1c8a700 1 -- 172.21.15.156:0/563176312 learned_addr learned my addr 172.21.15.156:0/563176312 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:37.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da1c8a700 1 -- 172.21.15.156:0/563176312 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c161340 msgr2=0x7f5d9c14d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da2c8c700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c161340 0x7f5d9c14d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da248b700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d9c0cd7c0 0x7f5d9c0cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da1c8a700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c161340 0x7f5d9c14d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da1c8a700 1 -- 172.21.15.156:0/563176312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d9c0cd7c0 msgr2=0x7f5d9c0cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da1c8a700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d9c0cd7c0 0x7f5d9c0cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da1c8a700 1 -- 172.21.15.156:0/563176312 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d94009a70 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da1c8a700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d9c15c9b0 0x7f5d9c1f5c50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c530 tx=0x7f5d9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:37.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5d92ffd700 1 -- 172.21.15.156:0/563176312 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d980052d0 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5d92ffd700 1 -- 172.21.15.156:0/563176312 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d98005430 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5da46ef700 1 -- 172.21.15.156:0/563176312 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c0d1300 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.363+0000 7f5d92ffd700 1 -- 172.21.15.156:0/563176312 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d980038c0 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.364+0000 7f5da46ef700 1 -- 172.21.15.156:0/563176312 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c0b5520 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.364+0000 7f5da46ef700 1 -- 172.21.15.156:0/563176312 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c0d15e0 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.365+0000 7f5d92ffd700 1 -- 172.21.15.156:0/563176312 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5d98019030 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.365+0000 7f5d92ffd700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5d800682c0 0x7f5d8006a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.365+0000 7f5d92ffd700 1 -- 172.21.15.156:0/563176312 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5d980850b0 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.365+0000 7f5da248b700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5d800682c0 0x7f5d8006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.366+0000 7f5da248b700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5d800682c0 0x7f5d8006a770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0016a0 tx=0x7f5d8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:37.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.370+0000 7f5d92ffd700 1 -- 172.21.15.156:0/563176312 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5d980518d0 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.550+0000 7f5da46ef700 1 -- 172.21.15.156:0/563176312 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d9c0745c0 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.551+0000 7f5d92ffd700 1 -- 172.21.15.156:0/563176312 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d98054ef0 con 0x7f5d9c15c9b0 2023-11-28T11:10:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:10:37.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.553+0000 7f5da46ef700 1 -- 172.21.15.156:0/563176312 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5d800682c0 msgr2=0x7f5d8006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.553+0000 7f5da46ef700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5d800682c0 0x7f5d8006a770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0016a0 tx=0x7f5d8c009040 comp rx=0 tx=0).stop 2023-11-28T11:10:37.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.553+0000 7f5da46ef700 1 -- 172.21.15.156:0/563176312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d9c15c9b0 msgr2=0x7f5d9c1f5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.553+0000 7f5da46ef700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d9c15c9b0 0x7f5d9c1f5c50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c530 tx=0x7f5d9800c8f0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.553+0000 7f5da46ef700 1 -- 172.21.15.156:0/563176312 shutdown_connections 2023-11-28T11:10:37.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.553+0000 7f5da46ef700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5d800682c0 0x7f5d8006a770 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.553+0000 7f5da46ef700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c161340 0x7f5d9c14d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.553+0000 7f5da46ef700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d9c0cd7c0 0x7f5d9c0cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.553+0000 7f5da46ef700 1 --2- 172.21.15.156:0/563176312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d9c15c9b0 0x7f5d9c1f5c50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.553+0000 7f5da46ef700 1 -- 172.21.15.156:0/563176312 >> 172.21.15.156:0/563176312 conn(0x7f5d9c072600 msgr2=0x7f5d9c0b89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:37.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.553+0000 7f5da46ef700 1 -- 172.21.15.156:0/563176312 shutdown_connections 2023-11-28T11:10:37.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.553+0000 7f5da46ef700 1 -- 172.21.15.156:0/563176312 wait complete. 2023-11-28T11:10:37.566 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:10:37.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.687+0000 7f932cee8700 1 Processor -- start 2023-11-28T11:10:37.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.687+0000 7f932cee8700 1 -- start start 2023-11-28T11:10:37.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.688+0000 7f932cee8700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93280d01b0 0x7f93280d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.688+0000 7f932cee8700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93280b0230 con 0x7f93280b16d0 2023-11-28T11:10:37.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.688+0000 7f932cee8700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93280b37d0 con 0x7f93280b20b0 2023-11-28T11:10:37.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.688+0000 7f932cee8700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932815cec0 con 0x7f93280d01b0 2023-11-28T11:10:37.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.688+0000 7f9326d9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f93280b16d0 0x7f93280b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50070/0 (socket says 172.21.15.156:50070) 2023-11-28T11:10:37.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.688+0000 7f9326d9d700 1 -- 172.21.15.156:0/1952644345 learned_addr learned my addr 172.21.15.156:0/1952644345 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:37.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.689+0000 7f932659c700 1 --2- 172.21.15.156:0/1952644345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93280d01b0 0x7f93280d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.689+0000 7f932559a700 1 -- 172.21.15.156:0/1952644345 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702993502 0 0) 0x7f93280b0230 con 0x7f93280b16d0 2023-11-28T11:10:37.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.689+0000 7f932559a700 1 -- 172.21.15.156:0/1952644345 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93100035b0 con 0x7f93280b16d0 2023-11-28T11:10:37.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.689+0000 7f932659c700 1 -- 172.21.15.156:0/1952644345 >> v1:172.21.15.186:6789/0 conn(0x7f93280b20b0 legacy=0x7f93280b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.689+0000 7f932559a700 1 -- 172.21.15.156:0/1952644345 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835301200 0 0) 0x7f93280b37d0 con 0x7f93280b20b0 2023-11-28T11:10:37.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.689+0000 7f932659c700 1 -- 172.21.15.156:0/1952644345 >> v1:172.21.15.156:6789/0 conn(0x7f93280b16d0 legacy=0x7f93280b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.689+0000 7f932659c700 1 -- 172.21.15.156:0/1952644345 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280b4570 con 0x7f93280d01b0 2023-11-28T11:10:37.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.689+0000 7f932659c700 1 --2- 172.21.15.156:0/1952644345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93280d01b0 0x7f93280d0c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f931c0098a0 tx=0x7f931c009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee6911312e2ed1d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:37.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.689+0000 7f932559a700 1 -- 172.21.15.156:0/1952644345 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93200019f0 con 0x7f93280d01b0 2023-11-28T11:10:37.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.689+0000 7f932559a700 1 -- 172.21.15.156:0/1952644345 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c004030 con 0x7f93280d01b0 2023-11-28T11:10:37.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.689+0000 7f932559a700 1 -- 172.21.15.156:0/1952644345 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c0043c0 con 0x7f93280d01b0 2023-11-28T11:10:37.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.690+0000 7f932cee8700 1 -- 172.21.15.156:0/1952644345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93280d01b0 msgr2=0x7f93280d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.690+0000 7f932cee8700 1 --2- 172.21.15.156:0/1952644345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93280d01b0 0x7f93280d0c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f931c0098a0 tx=0x7f931c009bb0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.690+0000 7f932cee8700 1 -- 172.21.15.156:0/1952644345 shutdown_connections 2023-11-28T11:10:37.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.690+0000 7f932cee8700 1 --2- 172.21.15.156:0/1952644345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93280d01b0 0x7f93280d0c20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.690+0000 7f932cee8700 1 -- 172.21.15.156:0/1952644345 >> 172.21.15.156:0/1952644345 conn(0x7f932809c3f0 msgr2=0x7f93280d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:37.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.690+0000 7f932cee8700 1 -- 172.21.15.156:0/1952644345 shutdown_connections 2023-11-28T11:10:37.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.690+0000 7f932cee8700 1 -- 172.21.15.156:0/1952644345 wait complete. 2023-11-28T11:10:37.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.691+0000 7f932cee8700 1 Processor -- start 2023-11-28T11:10:37.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.691+0000 7f932cee8700 1 -- start start 2023-11-28T11:10:37.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932cee8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93280b16d0 0x7f932814c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932cee8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93280b20b0 0x7f932815f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932cee8700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93280d01b0 0x7f932815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932cee8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932814c8a0 con 0x7f93280b16d0 2023-11-28T11:10:37.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932cee8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280b8010 con 0x7f93280b20b0 2023-11-28T11:10:37.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932cee8700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93280b8260 con 0x7f93280d01b0 2023-11-28T11:10:37.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93280b16d0 0x7f932814c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93280b16d0 0x7f932814c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35194/0 (socket says 172.21.15.156:35194) 2023-11-28T11:10:37.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932659c700 1 -- 172.21.15.156:0/267095590 learned_addr learned my addr 172.21.15.156:0/267095590 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:37.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f9326d9d700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93280d01b0 0x7f932815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f9325d9b700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93280b20b0 0x7f932815f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932659c700 1 -- 172.21.15.156:0/267095590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93280d01b0 msgr2=0x7f932815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932659c700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93280d01b0 0x7f932815b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932659c700 1 -- 172.21.15.156:0/267095590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93280b20b0 msgr2=0x7f932815f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932659c700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93280b20b0 0x7f932815f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.692+0000 7f932659c700 1 -- 172.21.15.156:0/267095590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c0094a0 con 0x7f93280b16d0 2023-11-28T11:10:37.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.693+0000 7f9325d9b700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93280b20b0 0x7f932815f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:37.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.693+0000 7f932659c700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93280b16d0 0x7f932814c360 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f931c0097c0 tx=0x7f931c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:37.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.693+0000 7f9316ffd700 1 -- 172.21.15.156:0/267095590 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c010790 con 0x7f93280b16d0 2023-11-28T11:10:37.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.693+0000 7f9316ffd700 1 -- 172.21.15.156:0/267095590 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c0108f0 con 0x7f93280b16d0 2023-11-28T11:10:37.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.693+0000 7f9316ffd700 1 -- 172.21.15.156:0/267095590 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c010c80 con 0x7f93280b16d0 2023-11-28T11:10:37.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.693+0000 7f932cee8700 1 -- 172.21.15.156:0/267095590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93280afbe0 con 0x7f93280b16d0 2023-11-28T11:10:37.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.693+0000 7f932cee8700 1 -- 172.21.15.156:0/267095590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93280b4850 con 0x7f93280b16d0 2023-11-28T11:10:37.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.694+0000 7f932cee8700 1 -- 172.21.15.156:0/267095590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93280afec0 con 0x7f93280b16d0 2023-11-28T11:10:37.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.694+0000 7f9316ffd700 1 -- 172.21.15.156:0/267095590 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f931c00f050 con 0x7f93280b16d0 2023-11-28T11:10:37.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.694+0000 7f9316ffd700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f93080682f0 0x7f930806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:37.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.695+0000 7f9316ffd700 1 -- 172.21.15.156:0/267095590 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f931c087550 con 0x7f93280b16d0 2023-11-28T11:10:37.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.695+0000 7f9325d9b700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f93080682f0 0x7f930806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:37.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.695+0000 7f9325d9b700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f93080682f0 0x7f930806a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f93280c9db0 tx=0x7f9318009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:37.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.699+0000 7f9316ffd700 1 -- 172.21.15.156:0/267095590 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f931c053c80 con 0x7f93280b16d0 2023-11-28T11:10:37.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.882+0000 7f932cee8700 1 -- 172.21.15.156:0/267095590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f932815d000 con 0x7f93280b16d0 2023-11-28T11:10:37.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.883+0000 7f9316ffd700 1 -- 172.21.15.156:0/267095590 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f931c0045b0 con 0x7f93280b16d0 2023-11-28T11:10:37.884 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:10:37.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.885+0000 7f932cee8700 1 -- 172.21.15.156:0/267095590 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f93080682f0 msgr2=0x7f930806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.885+0000 7f932cee8700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f93080682f0 0x7f930806a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f93280c9db0 tx=0x7f9318009040 comp rx=0 tx=0).stop 2023-11-28T11:10:37.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.885+0000 7f932cee8700 1 -- 172.21.15.156:0/267095590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93280b16d0 msgr2=0x7f932814c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:37.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.885+0000 7f932cee8700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93280b16d0 0x7f932814c360 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f931c0097c0 tx=0x7f931c004750 comp rx=0 tx=0).stop 2023-11-28T11:10:37.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.885+0000 7f932cee8700 1 -- 172.21.15.156:0/267095590 shutdown_connections 2023-11-28T11:10:37.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.885+0000 7f932cee8700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f93080682f0 0x7f930806a7a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.885+0000 7f932cee8700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93280d01b0 0x7f932815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.885+0000 7f932cee8700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93280b16d0 0x7f932814c360 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.885+0000 7f932cee8700 1 --2- 172.21.15.156:0/267095590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93280b20b0 0x7f932815f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:37.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.885+0000 7f932cee8700 1 -- 172.21.15.156:0/267095590 >> 172.21.15.156:0/267095590 conn(0x7f932809c3f0 msgr2=0x7f932806b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:37.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.885+0000 7f932cee8700 1 -- 172.21.15.156:0/267095590 shutdown_connections 2023-11-28T11:10:37.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:37.885+0000 7f932cee8700 1 -- 172.21.15.156:0/267095590 wait complete. 2023-11-28T11:10:37.898 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:10:38.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.021+0000 7f367d8b9700 1 Processor -- start 2023-11-28T11:10:38.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.021+0000 7f367d8b9700 1 -- start start 2023-11-28T11:10:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.021+0000 7f367d8b9700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3678159860 0x7f36780c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f367d8b9700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780b0d80 0x7f36780c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f367d8b9700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36780ccc00 con 0x7f3678159490 2023-11-28T11:10:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f367d8b9700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36780ccec0 con 0x7f3678159860 2023-11-28T11:10:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f367d8b9700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36780cd1a0 con 0x7f36780b0d80 2023-11-28T11:10:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3676ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3678159490 0x7f36780b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50080/0 (socket says 172.21.15.156:50080) 2023-11-28T11:10:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3676ffd700 1 -- 172.21.15.156:0/3930720001 learned_addr learned my addr 172.21.15.156:0/3930720001 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:38.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f36777fe700 1 --2- 172.21.15.156:0/3930720001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780b0d80 0x7f36780c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f36767fc700 1 --2- 172.21.15.156:0/3930720001 >> v2:172.21.15.186:3300/0 conn(0x7f3678159860 0x7f36780c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537496817 0 0) 0x7f36780ccc00 con 0x7f3678159490 2023-11-28T11:10:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36600035b0 con 0x7f3678159490 2023-11-28T11:10:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 884433200 0 0) 0x7f36600035b0 con 0x7f3678159490 2023-11-28T11:10:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780b0d80 msgr2=0x7f36780c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 --2- 172.21.15.156:0/3930720001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780b0d80 0x7f36780c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 >> v2:172.21.15.186:3300/0 conn(0x7f3678159860 msgr2=0x7f36780c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 --2- 172.21.15.156:0/3930720001 >> v2:172.21.15.186:3300/0 conn(0x7f3678159860 0x7f36780c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36780cd480 con 0x7f3678159490 2023-11-28T11:10:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f3668001670 con 0x7f3678159490 2023-11-28T11:10:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 >> v1:172.21.15.156:6789/0 conn(0x7f3678159490 legacy=0x7f36780b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 --2- 172.21.15.156:0/3930720001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3660003a00 0x7f3660003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 --2- 172.21.15.156:0/3930720001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780b0d80 0x7f3660004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 --2- 172.21.15.156:0/3930720001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3660004d90 0x7f3660005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36780ccec0 con 0x7f3660004d90 2023-11-28T11:10:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36780cd1a0 con 0x7f3660003a00 2023-11-28T11:10:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366000ba10 con 0x7f36780b0d80 2023-11-28T11:10:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f36777fe700 1 --2- 172.21.15.156:0/3930720001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3660004d90 0x7f3660005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f3676ffd700 1 --2- 172.21.15.156:0/3930720001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3660003a00 0x7f3660003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.022+0000 7f36767fc700 1 --2- 172.21.15.156:0/3930720001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780b0d80 0x7f3660004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f36777fe700 1 -- 172.21.15.156:0/3930720001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780b0d80 msgr2=0x7f3660004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f36777fe700 1 --2- 172.21.15.156:0/3930720001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780b0d80 0x7f3660004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f36777fe700 1 -- 172.21.15.156:0/3930720001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3660003a00 msgr2=0x7f3660003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:38.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f36777fe700 1 --2- 172.21.15.156:0/3930720001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3660003a00 0x7f3660003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f36777fe700 1 -- 172.21.15.156:0/3930720001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36600035b0 con 0x7f3660004d90 2023-11-28T11:10:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f36777fe700 1 --2- 172.21.15.156:0/3930720001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3660004d90 0x7f3660005200 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3670009fc0 tx=0x7f3660005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ce9c8af5d66f33 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367000ebd0 con 0x7f3660004d90 2023-11-28T11:10:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367000f040 con 0x7f3660004d90 2023-11-28T11:10:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f3675ffb700 1 -- 172.21.15.156:0/3930720001 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670014480 con 0x7f3660004d90 2023-11-28T11:10:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f367d8b9700 1 -- 172.21.15.156:0/3930720001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3660004d90 msgr2=0x7f3660005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f367d8b9700 1 --2- 172.21.15.156:0/3930720001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3660004d90 0x7f3660005200 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3670009fc0 tx=0x7f3660005880 comp rx=0 tx=0).stop 2023-11-28T11:10:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f36777fe700 1 -- 172.21.15.156:0/3930720001 reap_dead start 2023-11-28T11:10:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f367d8b9700 1 -- 172.21.15.156:0/3930720001 shutdown_connections 2023-11-28T11:10:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f367d8b9700 1 -- 172.21.15.156:0/3930720001 >> 172.21.15.156:0/3930720001 conn(0x7f36780cb670 msgr2=0x7f36780cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.023+0000 7f367d8b9700 1 -- 172.21.15.156:0/3930720001 shutdown_connections 2023-11-28T11:10:38.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.024+0000 7f367d8b9700 1 -- 172.21.15.156:0/3930720001 wait complete. 2023-11-28T11:10:38.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.025+0000 7f367d8b9700 1 Processor -- start 2023-11-28T11:10:38.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.025+0000 7f367d8b9700 1 -- start start 2023-11-28T11:10:38.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f367d8b9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36780c38c0 0x7f36781486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f367d8b9700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780c3c90 0x7f367806c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f367d8b9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f367814e8f0 0x7f367814ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f367d8b9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367814f2d0 con 0x7f36780c38c0 2023-11-28T11:10:38.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f367d8b9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3678147c50 con 0x7f367814e8f0 2023-11-28T11:10:38.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f367d8b9700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3678148210 con 0x7f36780c3c90 2023-11-28T11:10:38.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f3676ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36780c38c0 0x7f36781486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f3676ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36780c38c0 0x7f36781486a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35214/0 (socket says 172.21.15.156:35214) 2023-11-28T11:10:38.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f3676ffd700 1 -- 172.21.15.156:0/964763991 learned_addr learned my addr 172.21.15.156:0/964763991 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:38.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f36777fe700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f367814e8f0 0x7f367814ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f36767fc700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780c3c90 0x7f367806c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f3676ffd700 1 -- 172.21.15.156:0/964763991 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780c3c90 msgr2=0x7f367806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f3676ffd700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780c3c90 0x7f367806c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f3676ffd700 1 -- 172.21.15.156:0/964763991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f367814e8f0 msgr2=0x7f367814ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f3676ffd700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f367814e8f0 0x7f367814ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f3676ffd700 1 -- 172.21.15.156:0/964763991 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36780cdc80 con 0x7f36780c38c0 2023-11-28T11:10:38.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f36767fc700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780c3c90 0x7f367806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:38.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f36777fe700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f367814e8f0 0x7f367814ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:38.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f3676ffd700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36780c38c0 0x7f36781486a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3668004fd0 tx=0x7f36680076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:38.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f365ffff700 1 -- 172.21.15.156:0/964763991 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366800f8a0 con 0x7f36780c38c0 2023-11-28T11:10:38.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f365ffff700 1 -- 172.21.15.156:0/964763991 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366800fa00 con 0x7f36780c38c0 2023-11-28T11:10:38.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f367d8b9700 1 -- 172.21.15.156:0/964763991 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36780cd760 con 0x7f36780c38c0 2023-11-28T11:10:38.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f365ffff700 1 -- 172.21.15.156:0/964763991 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366800fdb0 con 0x7f36780c38c0 2023-11-28T11:10:38.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.026+0000 7f367d8b9700 1 -- 172.21.15.156:0/964763991 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36780cde10 con 0x7f36780c38c0 2023-11-28T11:10:38.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.027+0000 7f367d8b9700 1 -- 172.21.15.156:0/964763991 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36780745c0 con 0x7f36780c38c0 2023-11-28T11:10:38.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.028+0000 7f365ffff700 1 -- 172.21.15.156:0/964763991 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f366801d460 con 0x7f36780c38c0 2023-11-28T11:10:38.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.028+0000 7f365ffff700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f36640682f0 0x7f366406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.028+0000 7f365ffff700 1 -- 172.21.15.156:0/964763991 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3668085fb0 con 0x7f36780c38c0 2023-11-28T11:10:38.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.028+0000 7f36767fc700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f36640682f0 0x7f366406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.028+0000 7f36767fc700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f36640682f0 0x7f366406a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f36780c4600 tx=0x7f366c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:38.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.032+0000 7f365ffff700 1 -- 172.21.15.156:0/964763991 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3668052770 con 0x7f36780c38c0 2023-11-28T11:10:38.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.213+0000 7f367d8b9700 1 -- 172.21.15.156:0/964763991 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36780b4b70 con 0x7f36780c38c0 2023-11-28T11:10:38.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.213+0000 7f365ffff700 1 -- 172.21.15.156:0/964763991 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3668055d90 con 0x7f36780c38c0 2023-11-28T11:10:38.215 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:10:38.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.216+0000 7f367d8b9700 1 -- 172.21.15.156:0/964763991 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f36640682f0 msgr2=0x7f366406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.216+0000 7f367d8b9700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f36640682f0 0x7f366406a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f36780c4600 tx=0x7f366c008040 comp rx=0 tx=0).stop 2023-11-28T11:10:38.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.216+0000 7f367d8b9700 1 -- 172.21.15.156:0/964763991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36780c38c0 msgr2=0x7f36781486a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.216+0000 7f367d8b9700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36780c38c0 0x7f36781486a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3668004fd0 tx=0x7f36680076a0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.216+0000 7f367d8b9700 1 -- 172.21.15.156:0/964763991 shutdown_connections 2023-11-28T11:10:38.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.216+0000 7f367d8b9700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f367814e8f0 0x7f367814ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.216+0000 7f367d8b9700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36780c38c0 0x7f36781486a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.216+0000 7f367d8b9700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f36640682f0 0x7f366406a7a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.216+0000 7f367d8b9700 1 --2- 172.21.15.156:0/964763991 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36780c3c90 0x7f367806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.216+0000 7f367d8b9700 1 -- 172.21.15.156:0/964763991 >> 172.21.15.156:0/964763991 conn(0x7f36780cb670 msgr2=0x7f367815c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:38.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.216+0000 7f367d8b9700 1 -- 172.21.15.156:0/964763991 shutdown_connections 2023-11-28T11:10:38.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.216+0000 7f367d8b9700 1 -- 172.21.15.156:0/964763991 wait complete. 2023-11-28T11:10:38.229 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:38.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.348+0000 7f6e5fc53700 1 Processor -- start 2023-11-28T11:10:38.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.349+0000 7f6e5fc53700 1 -- start start 2023-11-28T11:10:38.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.349+0000 7f6e5fc53700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580cd8c0 0x7f6e58140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.349+0000 7f6e5fc53700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6e580cdc90 0x7f6e580b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.349+0000 7f6e5fc53700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e5813ff00 con 0x7f6e580b2e70 2023-11-28T11:10:38.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.349+0000 7f6e5fc53700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e581404c0 con 0x7f6e580cdc90 2023-11-28T11:10:38.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.349+0000 7f6e5fc53700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5815aa80 con 0x7f6e580cd8c0 2023-11-28T11:10:38.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.349+0000 7f6e5d1ee700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6e580cdc90 0x7f6e580b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.350+0000 7f6e5d1ee700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6e580cdc90 0x7f6e580b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35222/0 (socket says 172.21.15.156:35222) 2023-11-28T11:10:38.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.350+0000 7f6e5d1ee700 1 -- 172.21.15.156:0/1759306380 learned_addr learned my addr 172.21.15.156:0/1759306380 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:38.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.350+0000 7f6e5d9ef700 1 --2- 172.21.15.156:0/1759306380 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580cd8c0 0x7f6e58140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.350+0000 7f6e5d1ee700 1 -- 172.21.15.156:0/1759306380 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580cd8c0 msgr2=0x7f6e58140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:38.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.350+0000 7f6e5d1ee700 1 --2- 172.21.15.156:0/1759306380 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580cd8c0 0x7f6e58140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.350+0000 7f6e5d1ee700 1 -- 172.21.15.156:0/1759306380 >> v1:172.21.15.186:6789/0 conn(0x7f6e580b2e70 legacy=0x7f6e5806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.350+0000 7f6e5d1ee700 1 -- 172.21.15.156:0/1759306380 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580b3310 con 0x7f6e580cdc90 2023-11-28T11:10:38.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.350+0000 7f6e5d1ee700 1 --2- 172.21.15.156:0/1759306380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580cdc90 0x7f6e580b2930 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6e480098f0 tx=0x7f6e48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1475615dfaf3975 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:38.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.350+0000 7f6e5c9ed700 1 -- 172.21.15.156:0/1759306380 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800b8a0 con 0x7f6e580cdc90 2023-11-28T11:10:38.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.350+0000 7f6e5c9ed700 1 -- 172.21.15.156:0/1759306380 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4800ba00 con 0x7f6e580cdc90 2023-11-28T11:10:38.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.350+0000 7f6e5c9ed700 1 -- 172.21.15.156:0/1759306380 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800bd90 con 0x7f6e580cdc90 2023-11-28T11:10:38.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.351+0000 7f6e5fc53700 1 -- 172.21.15.156:0/1759306380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580cdc90 msgr2=0x7f6e580b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.351+0000 7f6e5fc53700 1 --2- 172.21.15.156:0/1759306380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580cdc90 0x7f6e580b2930 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6e480098f0 tx=0x7f6e48009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:38.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.351+0000 7f6e5fc53700 1 -- 172.21.15.156:0/1759306380 shutdown_connections 2023-11-28T11:10:38.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.351+0000 7f6e5fc53700 1 --2- 172.21.15.156:0/1759306380 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580cd8c0 0x7f6e58140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.351+0000 7f6e5fc53700 1 --2- 172.21.15.156:0/1759306380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580cdc90 0x7f6e580b2930 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.351+0000 7f6e5fc53700 1 -- 172.21.15.156:0/1759306380 >> 172.21.15.156:0/1759306380 conn(0x7f6e5806aed0 msgr2=0x7f6e5806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:38.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.351+0000 7f6e5fc53700 1 -- 172.21.15.156:0/1759306380 shutdown_connections 2023-11-28T11:10:38.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.351+0000 7f6e5fc53700 1 -- 172.21.15.156:0/1759306380 wait complete. 2023-11-28T11:10:38.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.352+0000 7f6e5fc53700 1 Processor -- start 2023-11-28T11:10:38.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.352+0000 7f6e5fc53700 1 -- start start 2023-11-28T11:10:38.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5fc53700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580b2e70 0x7f6e580ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5fc53700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580cd8c0 0x7f6e58155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5fc53700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e580cdc90 0x7f6e580b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5fc53700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580cae80 con 0x7f6e580cd8c0 2023-11-28T11:10:38.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5fc53700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580b4e60 con 0x7f6e580cdc90 2023-11-28T11:10:38.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5fc53700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580b5140 con 0x7f6e580b2e70 2023-11-28T11:10:38.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5d1ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580cd8c0 0x7f6e58155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5d1ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580cd8c0 0x7f6e58155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35230/0 (socket says 172.21.15.156:35230) 2023-11-28T11:10:38.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5d1ee700 1 -- 172.21.15.156:0/3314908036 learned_addr learned my addr 172.21.15.156:0/3314908036 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:38.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5d9ef700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580b2e70 0x7f6e580ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5e1f0700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e580cdc90 0x7f6e580b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5d1ee700 1 -- 172.21.15.156:0/3314908036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580b2e70 msgr2=0x7f6e580ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5d1ee700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580b2e70 0x7f6e580ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5d1ee700 1 -- 172.21.15.156:0/3314908036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e580cdc90 msgr2=0x7f6e580b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5d1ee700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e580cdc90 0x7f6e580b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5d1ee700 1 -- 172.21.15.156:0/3314908036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48009550 con 0x7f6e580cd8c0 2023-11-28T11:10:38.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5e1f0700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e580cdc90 0x7f6e580b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:38.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5d1ee700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580cd8c0 0x7f6e58155da0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009c40 tx=0x7f6e48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:38.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e4e7fc700 1 -- 172.21.15.156:0/3314908036 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48004590 con 0x7f6e580cd8c0 2023-11-28T11:10:38.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e4e7fc700 1 -- 172.21.15.156:0/3314908036 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4801e070 con 0x7f6e580cd8c0 2023-11-28T11:10:38.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e5fc53700 1 -- 172.21.15.156:0/3314908036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e58140150 con 0x7f6e580cd8c0 2023-11-28T11:10:38.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.353+0000 7f6e4e7fc700 1 -- 172.21.15.156:0/3314908036 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e480038e0 con 0x7f6e580cd8c0 2023-11-28T11:10:38.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.354+0000 7f6e5fc53700 1 -- 172.21.15.156:0/3314908036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e580b35f0 con 0x7f6e580cd8c0 2023-11-28T11:10:38.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.355+0000 7f6e5fc53700 1 -- 172.21.15.156:0/3314908036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e580ce100 con 0x7f6e580cd8c0 2023-11-28T11:10:38.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.355+0000 7f6e4e7fc700 1 -- 172.21.15.156:0/3314908036 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6e4800f050 con 0x7f6e580cd8c0 2023-11-28T11:10:38.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.355+0000 7f6e4e7fc700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6e3c0682f0 0x7f6e3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.355+0000 7f6e4e7fc700 1 -- 172.21.15.156:0/3314908036 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6e48020070 con 0x7f6e580cd8c0 2023-11-28T11:10:38.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.355+0000 7f6e5d9ef700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6e3c0682f0 0x7f6e3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.356+0000 7f6e5d9ef700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6e3c0682f0 0x7f6e3c06a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6e54006fd0 tx=0x7f6e54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:38.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.360+0000 7f6e4e7fc700 1 -- 172.21.15.156:0/3314908036 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6e480538e0 con 0x7f6e580cd8c0 2023-11-28T11:10:38.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.586+0000 7f6e5fc53700 1 -- 172.21.15.156:0/3314908036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e58068690 con 0x7f6e580cd8c0 2023-11-28T11:10:38.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.587+0000 7f6e4e7fc700 1 -- 172.21.15.156:0/3314908036 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f6e48056f00 con 0x7f6e580cd8c0 2023-11-28T11:10:38.588 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:38.588 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/365552946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":365552946},{"type":"v1","addr":"172.21.15.186:6834","nonce":365552946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1918325985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":1918325985},{"type":"v1","addr":"172.21.15.156:6837","nonce":1918325985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/2889641252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2889641252},{"type":"v1","addr":"172.21.15.156:6836","nonce":2889641252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:54.274258+0000","modified":"2023-11-28T11:09:57.285496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3832587641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3832587641},{"type":"v1","addr":"172.21.15.186:6835","nonce":3832587641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:10:38.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.589+0000 7f6e5fc53700 1 -- 172.21.15.156:0/3314908036 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6e3c0682f0 msgr2=0x7f6e3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.589+0000 7f6e5fc53700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6e3c0682f0 0x7f6e3c06a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6e54006fd0 tx=0x7f6e54008040 comp rx=0 tx=0).stop 2023-11-28T11:10:38.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.589+0000 7f6e5fc53700 1 -- 172.21.15.156:0/3314908036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580cd8c0 msgr2=0x7f6e58155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.589+0000 7f6e5fc53700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580cd8c0 0x7f6e58155da0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009c40 tx=0x7f6e48004750 comp rx=0 tx=0).stop 2023-11-28T11:10:38.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.589+0000 7f6e5fc53700 1 -- 172.21.15.156:0/3314908036 shutdown_connections 2023-11-28T11:10:38.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.589+0000 7f6e5fc53700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e580cdc90 0x7f6e580b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.589+0000 7f6e5fc53700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6e3c0682f0 0x7f6e3c06a7a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.589+0000 7f6e5fc53700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580b2e70 0x7f6e580ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.589+0000 7f6e5fc53700 1 --2- 172.21.15.156:0/3314908036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580cd8c0 0x7f6e58155da0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.589+0000 7f6e5fc53700 1 -- 172.21.15.156:0/3314908036 >> 172.21.15.156:0/3314908036 conn(0x7f6e5806aed0 msgr2=0x7f6e5813ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:38.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.589+0000 7f6e5fc53700 1 -- 172.21.15.156:0/3314908036 shutdown_connections 2023-11-28T11:10:38.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.589+0000 7f6e5fc53700 1 -- 172.21.15.156:0/3314908036 wait complete. 2023-11-28T11:10:38.591 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T11:10:38.602 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:38.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.721+0000 7fafb3886700 1 Processor -- start 2023-11-28T11:10:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.721+0000 7fafb3886700 1 -- start start 2023-11-28T11:10:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.722+0000 7fafb3886700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fafac071040 0x7fafac071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.722+0000 7fafb3886700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fafac06dee0 0x7fafac06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.722+0000 7fafb3886700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafac071950 con 0x7fafac06cdc0 2023-11-28T11:10:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.722+0000 7fafb3886700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafac161550 con 0x7fafac06dee0 2023-11-28T11:10:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.722+0000 7fafb3886700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafac15d060 con 0x7fafac071040 2023-11-28T11:10:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.722+0000 7fafb0e21700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fafac06dee0 0x7fafac06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.722+0000 7fafb1e23700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fafac06cdc0 0x7fafac06d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50118/0 (socket says 172.21.15.156:50118) 2023-11-28T11:10:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.722+0000 7fafb1e23700 1 -- 172.21.15.156:0/3877896482 learned_addr learned my addr 172.21.15.156:0/3877896482 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:38.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.723+0000 7fafb1622700 1 --2- 172.21.15.156:0/3877896482 >> v2:172.21.15.186:3300/0 conn(0x7fafac071040 0x7fafac071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.723+0000 7fafb0e21700 1 -- 172.21.15.156:0/3877896482 >> v2:172.21.15.186:3300/0 conn(0x7fafac071040 msgr2=0x7fafac071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:38.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.723+0000 7fafb0e21700 1 --2- 172.21.15.156:0/3877896482 >> v2:172.21.15.186:3300/0 conn(0x7fafac071040 0x7fafac071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.723+0000 7fafb0e21700 1 -- 172.21.15.156:0/3877896482 >> v1:172.21.15.156:6789/0 conn(0x7fafac06cdc0 legacy=0x7fafac06d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.723+0000 7fafa3fff700 1 -- 172.21.15.156:0/3877896482 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39742837 0 0) 0x7fafac071950 con 0x7fafac06cdc0 2023-11-28T11:10:38.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.723+0000 7fafb0e21700 1 -- 172.21.15.156:0/3877896482 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac15d340 con 0x7fafac06dee0 2023-11-28T11:10:38.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.723+0000 7fafb0e21700 1 --2- 172.21.15.156:0/3877896482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac06dee0 0x7fafac06e330 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fafac0d0f50 tx=0x7fafa8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6741cf450617eb1b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:38.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.723+0000 7fafa3fff700 1 -- 172.21.15.156:0/3877896482 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa8011830 con 0x7fafac06dee0 2023-11-28T11:10:38.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.723+0000 7fafa3fff700 1 -- 172.21.15.156:0/3877896482 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa8011990 con 0x7fafac06dee0 2023-11-28T11:10:38.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.723+0000 7fafa3fff700 1 -- 172.21.15.156:0/3877896482 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa8011d20 con 0x7fafac06dee0 2023-11-28T11:10:38.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.724+0000 7fafb3886700 1 -- 172.21.15.156:0/3877896482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac06dee0 msgr2=0x7fafac06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.724+0000 7fafb3886700 1 --2- 172.21.15.156:0/3877896482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac06dee0 0x7fafac06e330 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fafac0d0f50 tx=0x7fafa8009b20 comp rx=0 tx=0).stop 2023-11-28T11:10:38.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.724+0000 7fafb3886700 1 -- 172.21.15.156:0/3877896482 shutdown_connections 2023-11-28T11:10:38.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.724+0000 7fafb3886700 1 --2- 172.21.15.156:0/3877896482 >> v2:172.21.15.186:3300/0 conn(0x7fafac071040 0x7fafac071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.724+0000 7fafb3886700 1 --2- 172.21.15.156:0/3877896482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac06dee0 0x7fafac06e330 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.724+0000 7fafb3886700 1 -- 172.21.15.156:0/3877896482 >> 172.21.15.156:0/3877896482 conn(0x7fafac0d0000 msgr2=0x7fafac0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:38.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.724+0000 7fafb3886700 1 -- 172.21.15.156:0/3877896482 shutdown_connections 2023-11-28T11:10:38.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.724+0000 7fafb3886700 1 -- 172.21.15.156:0/3877896482 wait complete. 2023-11-28T11:10:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.725+0000 7fafb3886700 1 Processor -- start 2023-11-28T11:10:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.725+0000 7fafb3886700 1 -- start start 2023-11-28T11:10:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.726+0000 7fafb3886700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac06cdc0 0x7fafac1524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.726+0000 7fafb3886700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafac06dee0 0x7fafac1f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.726+0000 7fafb3886700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafac071040 0x7fafac153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.726+0000 7fafb3886700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac1f6580 con 0x7fafac06cdc0 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.726+0000 7fafb3886700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac1f6860 con 0x7fafac06dee0 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.726+0000 7fafb3886700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0b2fb0 con 0x7fafac071040 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.726+0000 7fafb1622700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac06cdc0 0x7fafac1524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.726+0000 7fafb1622700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac06cdc0 0x7fafac1524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35242/0 (socket says 172.21.15.156:35242) 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.726+0000 7fafb1622700 1 -- 172.21.15.156:0/3919308409 learned_addr learned my addr 172.21.15.156:0/3919308409 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.727+0000 7fafb1622700 1 -- 172.21.15.156:0/3919308409 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafac071040 msgr2=0x7fafac153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.727+0000 7fafb1622700 1 --2- 172.21.15.156:0/3919308409 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafac071040 0x7fafac153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.727+0000 7fafb1622700 1 -- 172.21.15.156:0/3919308409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafac06dee0 msgr2=0x7fafac1f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.727+0000 7fafb1622700 1 --2- 172.21.15.156:0/3919308409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafac06dee0 0x7fafac1f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.727+0000 7fafb1622700 1 -- 172.21.15.156:0/3919308409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa80094a0 con 0x7fafac06cdc0 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.727+0000 7fafb1622700 1 --2- 172.21.15.156:0/3919308409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac06cdc0 0x7fafac1524a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00e870 tx=0x7faf9c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.727+0000 7fafa1ffb700 1 -- 172.21.15.156:0/3919308409 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c00ee70 con 0x7fafac06cdc0 2023-11-28T11:10:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.727+0000 7fafa1ffb700 1 -- 172.21.15.156:0/3919308409 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c00c850 con 0x7fafac06cdc0 2023-11-28T11:10:38.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.727+0000 7fafb3886700 1 -- 172.21.15.156:0/3919308409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafac15d900 con 0x7fafac06cdc0 2023-11-28T11:10:38.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.727+0000 7fafa1ffb700 1 -- 172.21.15.156:0/3919308409 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c00cbc0 con 0x7fafac06cdc0 2023-11-28T11:10:38.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.727+0000 7fafb3886700 1 -- 172.21.15.156:0/3919308409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafac15d620 con 0x7fafac06cdc0 2023-11-28T11:10:38.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.728+0000 7fafb3886700 1 -- 172.21.15.156:0/3919308409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafac06e870 con 0x7fafac06cdc0 2023-11-28T11:10:38.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.729+0000 7fafa1ffb700 1 -- 172.21.15.156:0/3919308409 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faf9c023070 con 0x7fafac06cdc0 2023-11-28T11:10:38.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.729+0000 7fafa1ffb700 1 --2- 172.21.15.156:0/3919308409 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faf900682f0 0x7faf9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:38.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.729+0000 7fafa1ffb700 1 -- 172.21.15.156:0/3919308409 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7faf9c085f10 con 0x7fafac06cdc0 2023-11-28T11:10:38.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.729+0000 7fafb0e21700 1 --2- 172.21.15.156:0/3919308409 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faf900682f0 0x7faf9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:38.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.729+0000 7fafb0e21700 1 --2- 172.21.15.156:0/3919308409 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faf900682f0 0x7faf9006a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fafac06c3b0 tx=0x7fafa8010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:38.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.733+0000 7fafa1ffb700 1 -- 172.21.15.156:0/3919308409 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faf9c052730 con 0x7fafac06cdc0 2023-11-28T11:10:38.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.918+0000 7fafb3886700 1 -- 172.21.15.156:0/3919308409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafac0720a0 con 0x7fafac06cdc0 2023-11-28T11:10:38.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.919+0000 7fafa1ffb700 1 -- 172.21.15.156:0/3919308409 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18066 (secure 0 0 0) 0x7faf9c055d50 con 0x7fafac06cdc0 2023-11-28T11:10:38.921 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:38.921 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":51,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:10:02.303530+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:52.331853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:53.232328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.2d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:38.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.921+0000 7fafb3886700 1 -- 172.21.15.156:0/3919308409 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faf900682f0 msgr2=0x7faf9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.921+0000 7fafb3886700 1 --2- 172.21.15.156:0/3919308409 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faf900682f0 0x7faf9006a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fafac06c3b0 tx=0x7fafa8010000 comp rx=0 tx=0).stop 2023-11-28T11:10:38.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.921+0000 7fafb3886700 1 -- 172.21.15.156:0/3919308409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac06cdc0 msgr2=0x7fafac1524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:38.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.921+0000 7fafb3886700 1 --2- 172.21.15.156:0/3919308409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac06cdc0 0x7fafac1524a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00e870 tx=0x7faf9c00ec30 comp rx=0 tx=0).stop 2023-11-28T11:10:38.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.921+0000 7fafb3886700 1 -- 172.21.15.156:0/3919308409 shutdown_connections 2023-11-28T11:10:38.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.921+0000 7fafb3886700 1 --2- 172.21.15.156:0/3919308409 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7faf900682f0 0x7faf9006a7a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.921+0000 7fafb3886700 1 --2- 172.21.15.156:0/3919308409 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafac071040 0x7fafac153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.921+0000 7fafb3886700 1 --2- 172.21.15.156:0/3919308409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac06cdc0 0x7fafac1524a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.921+0000 7fafb3886700 1 --2- 172.21.15.156:0/3919308409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafac06dee0 0x7fafac1f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:38.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.921+0000 7fafb3886700 1 -- 172.21.15.156:0/3919308409 >> 172.21.15.156:0/3919308409 conn(0x7fafac0d0000 msgr2=0x7fafac06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:38.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.921+0000 7fafb3886700 1 -- 172.21.15.156:0/3919308409 shutdown_connections 2023-11-28T11:10:38.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:38.921+0000 7fafb3886700 1 -- 172.21.15.156:0/3919308409 wait complete. 2023-11-28T11:10:38.937 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:10:38.937 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:39.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.057+0000 7fedcc616700 1 Processor -- start 2023-11-28T11:10:39.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.057+0000 7fedcc616700 1 -- start start 2023-11-28T11:10:39.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.058+0000 7fedcc616700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fedc40cdbd0 0x7fedc4071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.058+0000 7fedcc616700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedc4071580 0x7fedc415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.058+0000 7fedcc616700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc4071980 con 0x7fedc4154e70 2023-11-28T11:10:39.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.058+0000 7fedcc616700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc40ce050 con 0x7fedc40cdbd0 2023-11-28T11:10:39.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.058+0000 7fedcc616700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4155240 con 0x7fedc4071580 2023-11-28T11:10:39.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.058+0000 7fedca3b2700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fedc4154e70 0x7fedc40cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50128/0 (socket says 172.21.15.156:50128) 2023-11-28T11:10:39.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.058+0000 7fedca3b2700 1 -- 172.21.15.156:0/1904095001 learned_addr learned my addr 172.21.15.156:0/1904095001 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:39.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657989306 0 0) 0x7fedc4071980 con 0x7fedc4154e70 2023-11-28T11:10:39.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc9bb1700 1 --2- 172.21.15.156:0/1904095001 >> v2:172.21.15.186:3300/0 conn(0x7fedc40cdbd0 0x7fedc4071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedcabb3700 1 --2- 172.21.15.156:0/1904095001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedc4071580 0x7fedc415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb00035b0 con 0x7fedc4154e70 2023-11-28T11:10:39.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4264811746 0 0) 0x7fedb00035b0 con 0x7fedc4154e70 2023-11-28T11:10:39.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedc4071580 msgr2=0x7fedc415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:39.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 --2- 172.21.15.156:0/1904095001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedc4071580 0x7fedc415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 >> v2:172.21.15.186:3300/0 conn(0x7fedc40cdbd0 msgr2=0x7fedc4071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:39.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 --2- 172.21.15.156:0/1904095001 >> v2:172.21.15.186:3300/0 conn(0x7fedc40cdbd0 0x7fedc4071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc415b100 con 0x7fedc4154e70 2023-11-28T11:10:39.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fedb4001670 con 0x7fedc4154e70 2023-11-28T11:10:39.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 >> v1:172.21.15.156:6789/0 conn(0x7fedc4154e70 legacy=0x7fedc40cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 --2- 172.21.15.156:0/1904095001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb0003a00 0x7fedb0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 --2- 172.21.15.156:0/1904095001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedc4071580 0x7fedb0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 --2- 172.21.15.156:0/1904095001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb0004d90 0x7fedb0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4155240 con 0x7fedb0003a00 2023-11-28T11:10:39.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb000ba10 con 0x7fedb0004d90 2023-11-28T11:10:39.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb000bb50 con 0x7fedc4071580 2023-11-28T11:10:39.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedca3b2700 1 --2- 172.21.15.156:0/1904095001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb0003a00 0x7fedb0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedcabb3700 1 --2- 172.21.15.156:0/1904095001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb0004d90 0x7fedb0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedc9bb1700 1 --2- 172.21.15.156:0/1904095001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedc4071580 0x7fedb0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedca3b2700 1 -- 172.21.15.156:0/1904095001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedc4071580 msgr2=0x7fedb0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:39.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedca3b2700 1 --2- 172.21.15.156:0/1904095001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedc4071580 0x7fedb0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedca3b2700 1 -- 172.21.15.156:0/1904095001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb0004d90 msgr2=0x7fedb0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:39.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedca3b2700 1 --2- 172.21.15.156:0/1904095001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb0004d90 0x7fedb0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.059+0000 7fedca3b2700 1 -- 172.21.15.156:0/1904095001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb00035b0 con 0x7fedb0003a00 2023-11-28T11:10:39.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.060+0000 7fedca3b2700 1 --2- 172.21.15.156:0/1904095001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb0003a00 0x7fedb0003e10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fedb400c370 tx=0x7fedb0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbafe4edc300c482 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:39.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.060+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4007c60 con 0x7fedb0003a00 2023-11-28T11:10:39.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.060+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb400ce50 con 0x7fedb0003a00 2023-11-28T11:10:39.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.060+0000 7fedc93b0700 1 -- 172.21.15.156:0/1904095001 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4005ba0 con 0x7fedb0003a00 2023-11-28T11:10:39.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.060+0000 7fedcc616700 1 -- 172.21.15.156:0/1904095001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb0003a00 msgr2=0x7fedb0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.060+0000 7fedcc616700 1 --2- 172.21.15.156:0/1904095001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb0003a00 0x7fedb0003e10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fedb400c370 tx=0x7fedb0005880 comp rx=0 tx=0).stop 2023-11-28T11:10:39.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.060+0000 7fedcabb3700 1 -- 172.21.15.156:0/1904095001 reap_dead start 2023-11-28T11:10:39.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.060+0000 7fedcc616700 1 -- 172.21.15.156:0/1904095001 shutdown_connections 2023-11-28T11:10:39.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.060+0000 7fedcc616700 1 -- 172.21.15.156:0/1904095001 >> 172.21.15.156:0/1904095001 conn(0x7fedc40d0000 msgr2=0x7fedc40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:39.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.060+0000 7fedcc616700 1 -- 172.21.15.156:0/1904095001 shutdown_connections 2023-11-28T11:10:39.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.060+0000 7fedcc616700 1 -- 172.21.15.156:0/1904095001 wait complete. 2023-11-28T11:10:39.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.062+0000 7fedcc616700 1 Processor -- start 2023-11-28T11:10:39.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.062+0000 7fedcc616700 1 -- start start 2023-11-28T11:10:39.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.062+0000 7fedcc616700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb0003a00 0x7fedc40b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.062+0000 7fedcc616700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedc41598b0 0x7fedc40ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.062+0000 7fedcc616700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedc4159d20 0x7fedc40ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.062+0000 7fedcc616700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc406e6d0 con 0x7fedb0003a00 2023-11-28T11:10:39.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.062+0000 7fedcc616700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4158e60 con 0x7fedc4159d20 2023-11-28T11:10:39.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.062+0000 7fedcc616700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4159420 con 0x7fedc41598b0 2023-11-28T11:10:39.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.062+0000 7fedca3b2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb0003a00 0x7fedc40b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.062+0000 7fedca3b2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb0003a00 0x7fedc40b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35262/0 (socket says 172.21.15.156:35262) 2023-11-28T11:10:39.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.062+0000 7fedca3b2700 1 -- 172.21.15.156:0/3025586657 learned_addr learned my addr 172.21.15.156:0/3025586657 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:39.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.063+0000 7fedca3b2700 1 -- 172.21.15.156:0/3025586657 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedc41598b0 msgr2=0x7fedc40ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.063+0000 7fedca3b2700 1 --2- 172.21.15.156:0/3025586657 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedc41598b0 0x7fedc40ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.063+0000 7fedca3b2700 1 -- 172.21.15.156:0/3025586657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedc4159d20 msgr2=0x7fedc40ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.063+0000 7fedca3b2700 1 --2- 172.21.15.156:0/3025586657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedc4159d20 0x7fedc40ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.063+0000 7fedca3b2700 1 -- 172.21.15.156:0/3025586657 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc415b6c0 con 0x7fedb0003a00 2023-11-28T11:10:39.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.063+0000 7fedca3b2700 1 --2- 172.21.15.156:0/3025586657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb0003a00 0x7fedc40b2e30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fedb400c960 tx=0x7fedb4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:39.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.063+0000 7fedbaffd700 1 -- 172.21.15.156:0/3025586657 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4004e90 con 0x7fedb0003a00 2023-11-28T11:10:39.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.063+0000 7fedbaffd700 1 -- 172.21.15.156:0/3025586657 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb40070d0 con 0x7fedb0003a00 2023-11-28T11:10:39.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.063+0000 7fedcc616700 1 -- 172.21.15.156:0/3025586657 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc415b3e0 con 0x7fedb0003a00 2023-11-28T11:10:39.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.063+0000 7fedbaffd700 1 -- 172.21.15.156:0/3025586657 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb40074a0 con 0x7fedb0003a00 2023-11-28T11:10:39.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.063+0000 7fedcc616700 1 -- 172.21.15.156:0/3025586657 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc415b9a0 con 0x7fedb0003a00 2023-11-28T11:10:39.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.064+0000 7fedcc616700 1 -- 172.21.15.156:0/3025586657 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc4159700 con 0x7fedb0003a00 2023-11-28T11:10:39.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.064+0000 7fedbaffd700 1 -- 172.21.15.156:0/3025586657 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fedb4024030 con 0x7fedb0003a00 2023-11-28T11:10:39.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.065+0000 7fedbaffd700 1 --2- 172.21.15.156:0/3025586657 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7feda80682f0 0x7feda806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.065+0000 7fedbaffd700 1 -- 172.21.15.156:0/3025586657 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fedb4087210 con 0x7fedb0003a00 2023-11-28T11:10:39.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.065+0000 7fedc9bb1700 1 --2- 172.21.15.156:0/3025586657 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7feda80682f0 0x7feda806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.065+0000 7fedc9bb1700 1 --2- 172.21.15.156:0/3025586657 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7feda80682f0 0x7feda806a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fedc0006ba0 tx=0x7fedc0006b30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:39.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.069+0000 7fedbaffd700 1 -- 172.21.15.156:0/3025586657 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fedb4053a30 con 0x7fedb0003a00 2023-11-28T11:10:39.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.297+0000 7fedcc616700 1 -- 172.21.15.156:0/3025586657 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedc406e6d0 con 0x7fedb0003a00 2023-11-28T11:10:39.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.298+0000 7fedbaffd700 1 -- 172.21.15.156:0/3025586657 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fedb4007dc0 con 0x7fedb0003a00 2023-11-28T11:10:39.299 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:39.299 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/365552946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":365552946},{"type":"v1","addr":"172.21.15.186:6834","nonce":365552946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1918325985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":1918325985},{"type":"v1","addr":"172.21.15.156:6837","nonce":1918325985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/2889641252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2889641252},{"type":"v1","addr":"172.21.15.156:6836","nonce":2889641252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:54.274258+0000","modified":"2023-11-28T11:09:57.285496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3832587641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3832587641},{"type":"v1","addr":"172.21.15.186:6835","nonce":3832587641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:10:39.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.300+0000 7fedcc616700 1 -- 172.21.15.156:0/3025586657 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7feda80682f0 msgr2=0x7feda806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.300+0000 7fedcc616700 1 --2- 172.21.15.156:0/3025586657 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7feda80682f0 0x7feda806a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fedc0006ba0 tx=0x7fedc0006b30 comp rx=0 tx=0).stop 2023-11-28T11:10:39.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.300+0000 7fedcc616700 1 -- 172.21.15.156:0/3025586657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb0003a00 msgr2=0x7fedc40b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.300+0000 7fedcc616700 1 --2- 172.21.15.156:0/3025586657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb0003a00 0x7fedc40b2e30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fedb400c960 tx=0x7fedb4000fe0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.300+0000 7fedcc616700 1 -- 172.21.15.156:0/3025586657 shutdown_connections 2023-11-28T11:10:39.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.300+0000 7fedcc616700 1 --2- 172.21.15.156:0/3025586657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedc4159d20 0x7fedc40ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.300+0000 7fedcc616700 1 --2- 172.21.15.156:0/3025586657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb0003a00 0x7fedc40b2e30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.300+0000 7fedcc616700 1 --2- 172.21.15.156:0/3025586657 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7feda80682f0 0x7feda806a7a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.300+0000 7fedcc616700 1 --2- 172.21.15.156:0/3025586657 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedc41598b0 0x7fedc40ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.300+0000 7fedcc616700 1 -- 172.21.15.156:0/3025586657 >> 172.21.15.156:0/3025586657 conn(0x7fedc40d0000 msgr2=0x7fedc40cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:39.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.300+0000 7fedcc616700 1 -- 172.21.15.156:0/3025586657 shutdown_connections 2023-11-28T11:10:39.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.300+0000 7fedcc616700 1 -- 172.21.15.156:0/3025586657 wait complete. 2023-11-28T11:10:39.303 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T11:10:39.313 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:39.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.432+0000 7f996b564700 1 Processor -- start 2023-11-28T11:10:39.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.433+0000 7f996b564700 1 -- start start 2023-11-28T11:10:39.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.433+0000 7f996b564700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f99640b3040 0x7f996415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.433+0000 7f996b564700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996406b420 con 0x7f99640ce040 2023-11-28T11:10:39.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.433+0000 7f996b564700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996406b700 con 0x7f99640d0a00 2023-11-28T11:10:39.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.433+0000 7f996b564700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996406ff60 con 0x7f99640b3040 2023-11-28T11:10:39.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f9969300700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f99640ce040 0x7f996406f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50148/0 (socket says 172.21.15.156:50148) 2023-11-28T11:10:39.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f9969300700 1 -- 172.21.15.156:0/226298901 learned_addr learned my addr 172.21.15.156:0/226298901 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:39.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f9969b01700 1 --2- 172.21.15.156:0/226298901 >> v2:172.21.15.186:3300/0 conn(0x7f99640b3040 0x7f996415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565623285 0 0) 0x7f996406b420 con 0x7f99640ce040 2023-11-28T11:10:39.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99500035b0 con 0x7f99640ce040 2023-11-28T11:10:39.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3223312922 0 0) 0x7f99500035b0 con 0x7f99640ce040 2023-11-28T11:10:39.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 >> v2:172.21.15.186:3300/0 conn(0x7f99640b3040 msgr2=0x7f996415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:39.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 --2- 172.21.15.156:0/226298901 >> v2:172.21.15.186:3300/0 conn(0x7f99640b3040 0x7f996415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 >> v1:172.21.15.186:6789/0 conn(0x7f99640d0a00 legacy=0x7f99640b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996406b9e0 con 0x7f99640ce040 2023-11-28T11:10:39.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f99540017f0 con 0x7f99640ce040 2023-11-28T11:10:39.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 >> v1:172.21.15.156:6789/0 conn(0x7f99640ce040 legacy=0x7f996406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 --2- 172.21.15.156:0/226298901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9950003a00 0x7f9950003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 --2- 172.21.15.156:0/226298901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9950004850 0x7f9950004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 --2- 172.21.15.156:0/226298901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9950005200 0x7f995000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996406ff60 con 0x7f9950003a00 2023-11-28T11:10:39.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995000c080 con 0x7f9950004850 2023-11-28T11:10:39.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995000c240 con 0x7f9950005200 2023-11-28T11:10:39.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f9969300700 1 --2- 172.21.15.156:0/226298901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9950003a00 0x7f9950003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f9968aff700 1 --2- 172.21.15.156:0/226298901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9950004850 0x7f9950004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.434+0000 7f9969b01700 1 --2- 172.21.15.156:0/226298901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9950005200 0x7f995000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f9969300700 1 -- 172.21.15.156:0/226298901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9950005200 msgr2=0x7f995000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:39.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f9969300700 1 --2- 172.21.15.156:0/226298901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9950005200 0x7f995000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f9969300700 1 -- 172.21.15.156:0/226298901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9950004850 msgr2=0x7f9950004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:39.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f9969300700 1 --2- 172.21.15.156:0/226298901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9950004850 0x7f9950004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f9969b01700 1 -- 172.21.15.156:0/226298901 reap_dead start 2023-11-28T11:10:39.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f9969300700 1 -- 172.21.15.156:0/226298901 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99500035b0 con 0x7f9950003a00 2023-11-28T11:10:39.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f9969300700 1 --2- 172.21.15.156:0/226298901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9950003a00 0x7f9950003e10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f995400c210 tx=0x7f9950005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd68e88d6f577779 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:39.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995400cd30 con 0x7f9950003a00 2023-11-28T11:10:39.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9954007b80 con 0x7f9950003a00 2023-11-28T11:10:39.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f995bfff700 1 -- 172.21.15.156:0/226298901 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954007310 con 0x7f9950003a00 2023-11-28T11:10:39.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f996b564700 1 -- 172.21.15.156:0/226298901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9950003a00 msgr2=0x7f9950003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f996b564700 1 --2- 172.21.15.156:0/226298901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9950003a00 0x7f9950003e10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f995400c210 tx=0x7f9950005880 comp rx=0 tx=0).stop 2023-11-28T11:10:39.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f996b564700 1 -- 172.21.15.156:0/226298901 shutdown_connections 2023-11-28T11:10:39.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.435+0000 7f996b564700 1 --2- 172.21.15.156:0/226298901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9950003a00 0x7f9950003e10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.436+0000 7f996b564700 1 -- 172.21.15.156:0/226298901 >> 172.21.15.156:0/226298901 conn(0x7f996406aed0 msgr2=0x7f996406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:39.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.436+0000 7f996b564700 1 -- 172.21.15.156:0/226298901 shutdown_connections 2023-11-28T11:10:39.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.436+0000 7f996b564700 1 -- 172.21.15.156:0/226298901 wait complete. 2023-11-28T11:10:39.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.437+0000 7f996b564700 1 Processor -- start 2023-11-28T11:10:39.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.437+0000 7f996b564700 1 -- start start 2023-11-28T11:10:39.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.437+0000 7f996b564700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9950003a00 0x7f99641520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.437+0000 7f996b564700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99640ca3f0 0x7f9964151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.437+0000 7f996b564700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9964154270 0x7f99641546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.437+0000 7f996b564700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9964151f70 con 0x7f9964154270 2023-11-28T11:10:39.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.437+0000 7f996b564700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99640ca9b0 con 0x7f9950003a00 2023-11-28T11:10:39.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.437+0000 7f996b564700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9964154f40 con 0x7f99640ca3f0 2023-11-28T11:10:39.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9969b01700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9964154270 0x7f99641546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9969b01700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9964154270 0x7f99641546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35286/0 (socket says 172.21.15.156:35286) 2023-11-28T11:10:39.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9969b01700 1 -- 172.21.15.156:0/3058043392 learned_addr learned my addr 172.21.15.156:0/3058043392 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:39.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9968aff700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99640ca3f0 0x7f9964151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9969300700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9950003a00 0x7f99641520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9969b01700 1 -- 172.21.15.156:0/3058043392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99640ca3f0 msgr2=0x7f9964151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9969b01700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99640ca3f0 0x7f9964151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9969b01700 1 -- 172.21.15.156:0/3058043392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9950003a00 msgr2=0x7f99641520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9969b01700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9950003a00 0x7f99641520d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9969b01700 1 -- 172.21.15.156:0/3058043392 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996406bfa0 con 0x7f9964154270 2023-11-28T11:10:39.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9969b01700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9964154270 0x7f99641546f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9964073e20 tx=0x7f995c011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:39.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9959ffb700 1 -- 172.21.15.156:0/3058043392 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c00e040 con 0x7f9964154270 2023-11-28T11:10:39.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9959ffb700 1 -- 172.21.15.156:0/3058043392 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c00f040 con 0x7f9964154270 2023-11-28T11:10:39.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f9959ffb700 1 -- 172.21.15.156:0/3058043392 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c00fdb0 con 0x7f9964154270 2023-11-28T11:10:39.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f996b564700 1 -- 172.21.15.156:0/3058043392 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996406bcc0 con 0x7f9964154270 2023-11-28T11:10:39.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.438+0000 7f996b564700 1 -- 172.21.15.156:0/3058043392 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996415df10 con 0x7f9964154270 2023-11-28T11:10:39.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.439+0000 7f996b564700 1 -- 172.21.15.156:0/3058043392 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9964068690 con 0x7f9964154270 2023-11-28T11:10:39.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.440+0000 7f9959ffb700 1 -- 172.21.15.156:0/3058043392 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f995c022460 con 0x7f9964154270 2023-11-28T11:10:39.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.440+0000 7f9959ffb700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f99480682f0 0x7f994806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.440+0000 7f9959ffb700 1 -- 172.21.15.156:0/3058043392 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f995c086350 con 0x7f9964154270 2023-11-28T11:10:39.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.440+0000 7f9969300700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f99480682f0 0x7f994806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.441+0000 7f9969300700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f99480682f0 0x7f994806a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f99640b2d00 tx=0x7f995400a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:39.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.444+0000 7f9959ffb700 1 -- 172.21.15.156:0/3058043392 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f995c052b10 con 0x7f9964154270 2023-11-28T11:10:39.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.667+0000 7f996b564700 1 -- 172.21.15.156:0/3058043392 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99640d0160 con 0x7f9964154270 2023-11-28T11:10:39.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.668+0000 7f9959ffb700 1 -- 172.21.15.156:0/3058043392 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f995c056130 con 0x7f9964154270 2023-11-28T11:10:39.669 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:39.669 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/365552946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":365552946},{"type":"v1","addr":"172.21.15.186:6834","nonce":365552946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1918325985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":1918325985},{"type":"v1","addr":"172.21.15.156:6837","nonce":1918325985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/2889641252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2889641252},{"type":"v1","addr":"172.21.15.156:6836","nonce":2889641252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:09:54.274258+0000","modified":"2023-11-28T11:09:57.285496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3832587641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3832587641},{"type":"v1","addr":"172.21.15.186:6835","nonce":3832587641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:10:39.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.670+0000 7f996b564700 1 -- 172.21.15.156:0/3058043392 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f99480682f0 msgr2=0x7f994806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.670+0000 7f996b564700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f99480682f0 0x7f994806a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f99640b2d00 tx=0x7f995400a000 comp rx=0 tx=0).stop 2023-11-28T11:10:39.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.670+0000 7f996b564700 1 -- 172.21.15.156:0/3058043392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9964154270 msgr2=0x7f99641546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.670+0000 7f996b564700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9964154270 0x7f99641546f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9964073e20 tx=0x7f995c011f80 comp rx=0 tx=0).stop 2023-11-28T11:10:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.670+0000 7f996b564700 1 -- 172.21.15.156:0/3058043392 shutdown_connections 2023-11-28T11:10:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.670+0000 7f996b564700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9964154270 0x7f99641546f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.670+0000 7f996b564700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9950003a00 0x7f99641520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.670+0000 7f996b564700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f99480682f0 0x7f994806a7a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.670+0000 7f996b564700 1 --2- 172.21.15.156:0/3058043392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99640ca3f0 0x7f9964151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.670+0000 7f996b564700 1 -- 172.21.15.156:0/3058043392 >> 172.21.15.156:0/3058043392 conn(0x7f996406aed0 msgr2=0x7f99640cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.670+0000 7f996b564700 1 -- 172.21.15.156:0/3058043392 shutdown_connections 2023-11-28T11:10:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.670+0000 7f996b564700 1 -- 172.21.15.156:0/3058043392 wait complete. 2023-11-28T11:10:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T11:10:39.683 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:39.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.803+0000 7fc38a200700 1 Processor -- start 2023-11-28T11:10:39.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.803+0000 7fc38a200700 1 -- start start 2023-11-28T11:10:39.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc38a200700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc38406f850 0x7fc38406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc38a200700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38413ff00 con 0x7fc384140950 2023-11-28T11:10:39.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc38a200700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3841404c0 con 0x7fc3840cd8c0 2023-11-28T11:10:39.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc38a200700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc384140dc0 con 0x7fc38406f850 2023-11-28T11:10:39.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc37bfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc384140950 0x7fc3840b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50170/0 (socket says 172.21.15.156:50170) 2023-11-28T11:10:39.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc37bfff700 1 -- 172.21.15.156:0/3499899416 learned_addr learned my addr 172.21.15.156:0/3499899416 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:39.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc383fff700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc38406f850 0x7fc38406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218464372 0 0) 0x7fc38413ff00 con 0x7fc384140950 2023-11-28T11:10:39.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3700035b0 con 0x7fc384140950 2023-11-28T11:10:39.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3054739697 0 0) 0x7fc3700035b0 con 0x7fc384140950 2023-11-28T11:10:39.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc38406f850 msgr2=0x7fc38406fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:39.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc38406f850 0x7fc38406fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 >> v1:172.21.15.186:6789/0 conn(0x7fc3840cd8c0 legacy=0x7fc3840cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38415aa80 con 0x7fc384140950 2023-11-28T11:10:39.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fc36c001670 con 0x7fc384140950 2023-11-28T11:10:39.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc383fff700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc38406f850 0x7fc38406fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:39.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 >> v1:172.21.15.156:6789/0 conn(0x7fc384140950 legacy=0x7fc3840b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc38406f850 0x7fc370003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc370003f70 0x7fc370004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc370004d90 0x7fc370005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc384140dc0 con 0x7fc370003f70 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37000bb50 con 0x7fc370004d90 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37000bd10 con 0x7fc38406f850 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc37bfff700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc370003f70 0x7fc370004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc383fff700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc370004d90 0x7fc370005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.804+0000 7fc3837fe700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc38406f850 0x7fc370003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.805+0000 7fc37bfff700 1 -- 172.21.15.156:0/3499899416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc38406f850 msgr2=0x7fc370003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.805+0000 7fc37bfff700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc38406f850 0x7fc370003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.805+0000 7fc37bfff700 1 -- 172.21.15.156:0/3499899416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc370004d90 msgr2=0x7fc370005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:39.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.805+0000 7fc37bfff700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc370004d90 0x7fc370005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.805+0000 7fc37bfff700 1 -- 172.21.15.156:0/3499899416 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c009a70 con 0x7fc370003f70 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.805+0000 7fc37bfff700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc370003f70 0x7fc370004850 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc3840cea30 tx=0x7fc370005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd5d766e30aaa10 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.805+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c007b70 con 0x7fc370003f70 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.805+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c00cc50 con 0x7fc370003f70 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.805+0000 7fc382ffd700 1 -- 172.21.15.156:0/3499899416 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c005960 con 0x7fc370003f70 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.805+0000 7fc38a200700 1 -- 172.21.15.156:0/3499899416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc370003f70 msgr2=0x7fc370004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.805+0000 7fc38a200700 1 --2- 172.21.15.156:0/3499899416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc370003f70 0x7fc370004850 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc3840cea30 tx=0x7fc370005880 comp rx=0 tx=0).stop 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.805+0000 7fc383fff700 1 -- 172.21.15.156:0/3499899416 reap_dead start 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.806+0000 7fc38a200700 1 -- 172.21.15.156:0/3499899416 shutdown_connections 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.806+0000 7fc38a200700 1 -- 172.21.15.156:0/3499899416 >> 172.21.15.156:0/3499899416 conn(0x7fc38406aed0 msgr2=0x7fc38406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.806+0000 7fc38a200700 1 -- 172.21.15.156:0/3499899416 shutdown_connections 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.806+0000 7fc38a200700 1 -- 172.21.15.156:0/3499899416 wait complete. 2023-11-28T11:10:39.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.807+0000 7fc38a200700 1 Processor -- start 2023-11-28T11:10:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.807+0000 7fc38a200700 1 -- start start 2023-11-28T11:10:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc38a200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc370003f70 0x7fc3840b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc38a200700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841540f0 0x7fc38406b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc38a200700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3840b4e00 0x7fc38413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc38a200700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3840b2b90 con 0x7fc3841540f0 2023-11-28T11:10:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc38a200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38406bc70 con 0x7fc370003f70 2023-11-28T11:10:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc38a200700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3840b4040 con 0x7fc3840b4e00 2023-11-28T11:10:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc37bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841540f0 0x7fc38406b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc37bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841540f0 0x7fc38406b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35304/0 (socket says 172.21.15.156:35304) 2023-11-28T11:10:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc37bfff700 1 -- 172.21.15.156:0/1165673026 learned_addr learned my addr 172.21.15.156:0/1165673026 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc37bfff700 1 -- 172.21.15.156:0/1165673026 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3840b4e00 msgr2=0x7fc38413ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc37bfff700 1 --2- 172.21.15.156:0/1165673026 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3840b4e00 0x7fc38413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc37bfff700 1 -- 172.21.15.156:0/1165673026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc370003f70 msgr2=0x7fc3840b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:39.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc37bfff700 1 --2- 172.21.15.156:0/1165673026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc370003f70 0x7fc3840b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:39.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc37bfff700 1 -- 172.21.15.156:0/1165673026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3840b3310 con 0x7fc3841540f0 2023-11-28T11:10:39.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc37bfff700 1 --2- 172.21.15.156:0/1165673026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841540f0 0x7fc38406b730 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc36c011000 tx=0x7fc36c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:39.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc380ff9700 1 -- 172.21.15.156:0/1165673026 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c004c80 con 0x7fc3841540f0 2023-11-28T11:10:39.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc380ff9700 1 -- 172.21.15.156:0/1165673026 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c0019e0 con 0x7fc3841540f0 2023-11-28T11:10:39.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc38a200700 1 -- 172.21.15.156:0/1165673026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3840b3040 con 0x7fc3841540f0 2023-11-28T11:10:39.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc380ff9700 1 -- 172.21.15.156:0/1165673026 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0070d0 con 0x7fc3841540f0 2023-11-28T11:10:39.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.808+0000 7fc38a200700 1 -- 172.21.15.156:0/1165673026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3840b35f0 con 0x7fc3841540f0 2023-11-28T11:10:39.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.809+0000 7fc38a200700 1 -- 172.21.15.156:0/1165673026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3840b4c50 con 0x7fc3841540f0 2023-11-28T11:10:39.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.810+0000 7fc380ff9700 1 -- 172.21.15.156:0/1165673026 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc36c010070 con 0x7fc3841540f0 2023-11-28T11:10:39.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.810+0000 7fc380ff9700 1 --2- 172.21.15.156:0/1165673026 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc3640682f0 0x7fc36406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:39.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.810+0000 7fc380ff9700 1 -- 172.21.15.156:0/1165673026 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc36c087070 con 0x7fc3841540f0 2023-11-28T11:10:39.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.810+0000 7fc3837fe700 1 --2- 172.21.15.156:0/1165673026 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc3640682f0 0x7fc36406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:39.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.811+0000 7fc3837fe700 1 --2- 172.21.15.156:0/1165673026 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc3640682f0 0x7fc36406a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc374000ef0 tx=0x7fc374009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:39.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.814+0000 7fc380ff9700 1 -- 172.21.15.156:0/1165673026 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc36c053830 con 0x7fc3841540f0 2023-11-28T11:10:39.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.998+0000 7fc38a200700 1 -- 172.21.15.156:0/1165673026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc384068690 con 0x7fc3841540f0 2023-11-28T11:10:40.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:39.999+0000 7fc380ff9700 1 -- 172.21.15.156:0/1165673026 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18066 (secure 0 0 0) 0x7fc36c017090 con 0x7fc3841540f0 2023-11-28T11:10:40.000 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:40.000 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":51,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:10:02.303530+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:09:52.331853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:09:53.232328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.2d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:40.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.001+0000 7fc38a200700 1 -- 172.21.15.156:0/1165673026 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc3640682f0 msgr2=0x7fc36406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:40.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.001+0000 7fc38a200700 1 --2- 172.21.15.156:0/1165673026 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc3640682f0 0x7fc36406a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc374000ef0 tx=0x7fc374009040 comp rx=0 tx=0).stop 2023-11-28T11:10:40.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.001+0000 7fc38a200700 1 -- 172.21.15.156:0/1165673026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841540f0 msgr2=0x7fc38406b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:40.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.001+0000 7fc38a200700 1 --2- 172.21.15.156:0/1165673026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841540f0 0x7fc38406b730 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc36c011000 tx=0x7fc36c001930 comp rx=0 tx=0).stop 2023-11-28T11:10:40.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.001+0000 7fc38a200700 1 -- 172.21.15.156:0/1165673026 shutdown_connections 2023-11-28T11:10:40.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.001+0000 7fc38a200700 1 --2- 172.21.15.156:0/1165673026 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc3640682f0 0x7fc36406a7a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.001+0000 7fc38a200700 1 --2- 172.21.15.156:0/1165673026 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3840b4e00 0x7fc38413ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.001+0000 7fc38a200700 1 --2- 172.21.15.156:0/1165673026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc370003f70 0x7fc3840b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.001+0000 7fc38a200700 1 --2- 172.21.15.156:0/1165673026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841540f0 0x7fc38406b730 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.001+0000 7fc38a200700 1 -- 172.21.15.156:0/1165673026 >> 172.21.15.156:0/1165673026 conn(0x7fc38406aed0 msgr2=0x7fc3840cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:40.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.001+0000 7fc38a200700 1 -- 172.21.15.156:0/1165673026 shutdown_connections 2023-11-28T11:10:40.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.001+0000 7fc38a200700 1 -- 172.21.15.156:0/1165673026 wait complete. 2023-11-28T11:10:40.015 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:10:40.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.137+0000 7ff585682700 1 Processor -- start 2023-11-28T11:10:40.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.137+0000 7ff585682700 1 -- start start 2023-11-28T11:10:40.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff585682700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff5800ce4b0 0x7ff58006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:40.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff585682700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58006bf10 con 0x7ff58006fb30 2023-11-28T11:10:40.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff585682700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5801590b0 con 0x7ff580158ce0 2023-11-28T11:10:40.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff585682700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5800cb0d0 con 0x7ff5800ce4b0 2023-11-28T11:10:40.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff57e7fc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff58006fb30 0x7ff5800cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50190/0 (socket says 172.21.15.156:50190) 2023-11-28T11:10:40.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff57e7fc700 1 -- 172.21.15.156:0/4185823929 learned_addr learned my addr 172.21.15.156:0/4185823929 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:40.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff57f7fe700 1 --2- 172.21.15.156:0/4185823929 >> v2:172.21.15.186:3300/0 conn(0x7ff5800ce4b0 0x7ff58006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:40.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff57dffb700 1 -- 172.21.15.156:0/4185823929 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275575706 0 0) 0x7ff58006bf10 con 0x7ff58006fb30 2023-11-28T11:10:40.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff57dffb700 1 -- 172.21.15.156:0/4185823929 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5680035b0 con 0x7ff58006fb30 2023-11-28T11:10:40.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff57dffb700 1 -- 172.21.15.156:0/4185823929 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589306171 0 0) 0x7ff5801590b0 con 0x7ff580158ce0 2023-11-28T11:10:40.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff57dffb700 1 -- 172.21.15.156:0/4185823929 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff58006bf10 con 0x7ff580158ce0 2023-11-28T11:10:40.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff57dffb700 1 -- 172.21.15.156:0/4185823929 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1350133136 0 0) 0x7ff5680035b0 con 0x7ff58006fb30 2023-11-28T11:10:40.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff57f7fe700 1 -- 172.21.15.156:0/4185823929 >> v1:172.21.15.186:6789/0 conn(0x7ff580158ce0 legacy=0x7ff58015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:40.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff57f7fe700 1 -- 172.21.15.156:0/4185823929 >> v1:172.21.15.156:6789/0 conn(0x7ff58006fb30 legacy=0x7ff5800cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:40.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff57f7fe700 1 -- 172.21.15.156:0/4185823929 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5800cb3b0 con 0x7ff5800ce4b0 2023-11-28T11:10:40.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.138+0000 7ff57f7fe700 1 --2- 172.21.15.156:0/4185823929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5800ce4b0 0x7ff58006c3a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff578009d90 tx=0x7ff57800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ac69b235ece6d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:40.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.139+0000 7ff57dffb700 1 -- 172.21.15.156:0/4185823929 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57800e070 con 0x7ff5800ce4b0 2023-11-28T11:10:40.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.139+0000 7ff57dffb700 1 -- 172.21.15.156:0/4185823929 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff578003a20 con 0x7ff5800ce4b0 2023-11-28T11:10:40.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.139+0000 7ff57dffb700 1 -- 172.21.15.156:0/4185823929 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff574001670 con 0x7ff5800ce4b0 2023-11-28T11:10:40.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.139+0000 7ff585682700 1 -- 172.21.15.156:0/4185823929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5800ce4b0 msgr2=0x7ff58006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:40.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.139+0000 7ff585682700 1 --2- 172.21.15.156:0/4185823929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5800ce4b0 0x7ff58006c3a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff578009d90 tx=0x7ff57800b330 comp rx=0 tx=0).stop 2023-11-28T11:10:40.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.139+0000 7ff585682700 1 -- 172.21.15.156:0/4185823929 shutdown_connections 2023-11-28T11:10:40.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.139+0000 7ff585682700 1 --2- 172.21.15.156:0/4185823929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5800ce4b0 0x7ff58006c3a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.139+0000 7ff585682700 1 -- 172.21.15.156:0/4185823929 >> 172.21.15.156:0/4185823929 conn(0x7ff5800cb670 msgr2=0x7ff5800cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:40.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.139+0000 7ff585682700 1 -- 172.21.15.156:0/4185823929 shutdown_connections 2023-11-28T11:10:40.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.139+0000 7ff585682700 1 -- 172.21.15.156:0/4185823929 wait complete. 2023-11-28T11:10:40.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.140+0000 7ff585682700 1 Processor -- start 2023-11-28T11:10:40.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.140+0000 7ff585682700 1 -- start start 2023-11-28T11:10:40.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.141+0000 7ff585682700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff58006fb30 0x7ff580149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:40.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.141+0000 7ff585682700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5800ce4b0 0x7ff58014f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:40.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.141+0000 7ff585682700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580158ce0 0x7ff5801471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:40.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.141+0000 7ff585682700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58014ebc0 con 0x7ff580158ce0 2023-11-28T11:10:40.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.141+0000 7ff585682700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff580150000 con 0x7ff58006fb30 2023-11-28T11:10:40.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.141+0000 7ff585682700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58014e570 con 0x7ff5800ce4b0 2023-11-28T11:10:40.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.141+0000 7ff57f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580158ce0 0x7ff5801471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:40.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.141+0000 7ff57f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580158ce0 0x7ff5801471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35320/0 (socket says 172.21.15.156:35320) 2023-11-28T11:10:40.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.141+0000 7ff57f7fe700 1 -- 172.21.15.156:0/2554743303 learned_addr learned my addr 172.21.15.156:0/2554743303 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff57e7fc700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5800ce4b0 0x7ff58014f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff57effd700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff58006fb30 0x7ff580149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff57f7fe700 1 -- 172.21.15.156:0/2554743303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5800ce4b0 msgr2=0x7ff58014f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff57f7fe700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5800ce4b0 0x7ff58014f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff57f7fe700 1 -- 172.21.15.156:0/2554743303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff58006fb30 msgr2=0x7ff580149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff57f7fe700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff58006fb30 0x7ff580149f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff57f7fe700 1 -- 172.21.15.156:0/2554743303 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff578009a70 con 0x7ff580158ce0 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff57effd700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff58006fb30 0x7ff580149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff57f7fe700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580158ce0 0x7ff5801471a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff578004000 tx=0x7ff578004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff5677fe700 1 -- 172.21.15.156:0/2554743303 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5780045c0 con 0x7ff580158ce0 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff5677fe700 1 -- 172.21.15.156:0/2554743303 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff578004720 con 0x7ff580158ce0 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff585682700 1 -- 172.21.15.156:0/2554743303 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58006bba0 con 0x7ff580158ce0 2023-11-28T11:10:40.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff5677fe700 1 -- 172.21.15.156:0/2554743303 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57801d470 con 0x7ff580158ce0 2023-11-28T11:10:40.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.142+0000 7ff585682700 1 -- 172.21.15.156:0/2554743303 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58006b8c0 con 0x7ff580158ce0 2023-11-28T11:10:40.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.143+0000 7ff585682700 1 -- 172.21.15.156:0/2554743303 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58014ed30 con 0x7ff580158ce0 2023-11-28T11:10:40.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.143+0000 7ff5677fe700 1 -- 172.21.15.156:0/2554743303 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff578018030 con 0x7ff580158ce0 2023-11-28T11:10:40.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.144+0000 7ff5677fe700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff56c0682f0 0x7ff56c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:40.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.144+0000 7ff5677fe700 1 -- 172.21.15.156:0/2554743303 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff5780876c0 con 0x7ff580158ce0 2023-11-28T11:10:40.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.144+0000 7ff57effd700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff56c0682f0 0x7ff56c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:40.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.144+0000 7ff57effd700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff56c0682f0 0x7ff56c06a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff5800b4b40 tx=0x7ff570009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:40.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.148+0000 7ff5677fe700 1 -- 172.21.15.156:0/2554743303 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff578053df0 con 0x7ff580158ce0 2023-11-28T11:10:40.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.374+0000 7ff585682700 1 -- 172.21.15.156:0/2554743303 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff5800745c0 con 0x7ff580158ce0 2023-11-28T11:10:40.833 INFO:tasks.ceph.mon.a.smithi156.stderr:2023-11-28T11:10:40.831+0000 7f6a4eb7c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:10:40.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.840+0000 7ff5677fe700 1 -- 172.21.15.156:0/2554743303 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff578057410 con 0x7ff580158ce0 2023-11-28T11:10:40.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.842+0000 7ff585682700 1 -- 172.21.15.156:0/2554743303 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff56c0682f0 msgr2=0x7ff56c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:40.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.842+0000 7ff585682700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff56c0682f0 0x7ff56c06a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff5800b4b40 tx=0x7ff570009040 comp rx=0 tx=0).stop 2023-11-28T11:10:40.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.842+0000 7ff585682700 1 -- 172.21.15.156:0/2554743303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580158ce0 msgr2=0x7ff5801471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:40.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.842+0000 7ff585682700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580158ce0 0x7ff5801471a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff578004000 tx=0x7ff578004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.842+0000 7ff585682700 1 -- 172.21.15.156:0/2554743303 shutdown_connections 2023-11-28T11:10:40.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.842+0000 7ff585682700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580158ce0 0x7ff5801471a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.842+0000 7ff585682700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff58006fb30 0x7ff580149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.842+0000 7ff585682700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff56c0682f0 0x7ff56c06a7a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.842+0000 7ff585682700 1 --2- 172.21.15.156:0/2554743303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5800ce4b0 0x7ff58014f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.842+0000 7ff585682700 1 -- 172.21.15.156:0/2554743303 >> 172.21.15.156:0/2554743303 conn(0x7ff5800cb670 msgr2=0x7ff5801505f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:40.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.842+0000 7ff585682700 1 -- 172.21.15.156:0/2554743303 shutdown_connections 2023-11-28T11:10:40.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.843+0000 7ff585682700 1 -- 172.21.15.156:0/2554743303 wait complete. 2023-11-28T11:10:40.846 INFO:teuthology.orchestra.run.smithi156.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:10:40.846 INFO:tasks.ceph.mds.d.smithi186.stderr: -6613> 2023-11-28T11:09:54.863+0000 7f78f8657700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:10:40.850 INFO:tasks.ceph.mds.d.smithi186.stderr: -2943> 2023-11-28T11:10:02.642+0000 7f78f4e50700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5703=0-4194304@1} caps={5703=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5703 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55cb5f27d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:10:40.857 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:10:40.881 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-11-28T11:10:40.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.981+0000 7fe91fe1c700 1 Processor -- start 2023-11-28T11:10:40.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.982+0000 7fe91fe1c700 1 -- start start 2023-11-28T11:10:40.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.982+0000 7fe91fe1c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe918071040 0x7fe918071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:40.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.982+0000 7fe91fe1c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe91806dee0 0x7fe91806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:40.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.982+0000 7fe91fe1c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe91806cdc0 0x7fe91806d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:40.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.982+0000 7fe91fe1c700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe918071950 con 0x7fe918071040 2023-11-28T11:10:40.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.982+0000 7fe91fe1c700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91815d060 con 0x7fe91806cdc0 2023-11-28T11:10:40.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.982+0000 7fe91fe1c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91815d340 con 0x7fe91806dee0 2023-11-28T11:10:40.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.982+0000 7fe91dbb8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe918071040 0x7fe918071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:40.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.982+0000 7fe91dbb8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe918071040 0x7fe918071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35324/0 (socket says 172.21.15.156:35324) 2023-11-28T11:10:40.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.982+0000 7fe91dbb8700 1 -- 172.21.15.156:0/1262155378 learned_addr learned my addr 172.21.15.156:0/1262155378 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:40.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.983+0000 7fe91d3b7700 1 --2- 172.21.15.156:0/1262155378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe91806dee0 0x7fe91806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:40.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.983+0000 7fe91e3b9700 1 --2- 172.21.15.156:0/1262155378 >> v2:172.21.15.186:3300/0 conn(0x7fe91806cdc0 0x7fe91806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:40.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.983+0000 7fe91dbb8700 1 -- 172.21.15.156:0/1262155378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe91806dee0 msgr2=0x7fe91806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:40.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.983+0000 7fe91dbb8700 1 --2- 172.21.15.156:0/1262155378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe91806dee0 0x7fe91806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.983+0000 7fe91dbb8700 1 -- 172.21.15.156:0/1262155378 >> v2:172.21.15.186:3300/0 conn(0x7fe91806cdc0 msgr2=0x7fe91806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:40.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.983+0000 7fe91dbb8700 1 --2- 172.21.15.156:0/1262155378 >> v2:172.21.15.186:3300/0 conn(0x7fe91806cdc0 0x7fe91806d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.983+0000 7fe91dbb8700 1 -- 172.21.15.156:0/1262155378 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91815d620 con 0x7fe918071040 2023-11-28T11:10:40.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.983+0000 7fe91dbb8700 1 --2- 172.21.15.156:0/1262155378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe918071040 0x7fe918071410 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe91809e0f0 tx=0x7fe908009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad93cc58ce4d5bf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:40.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.983+0000 7fe91cbb6700 1 -- 172.21.15.156:0/1262155378 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800b870 con 0x7fe918071040 2023-11-28T11:10:40.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.983+0000 7fe91cbb6700 1 -- 172.21.15.156:0/1262155378 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90800b9d0 con 0x7fe918071040 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.983+0000 7fe91cbb6700 1 -- 172.21.15.156:0/1262155378 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800bd60 con 0x7fe918071040 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.984+0000 7fe91fe1c700 1 -- 172.21.15.156:0/1262155378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe918071040 msgr2=0x7fe918071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.984+0000 7fe91fe1c700 1 --2- 172.21.15.156:0/1262155378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe918071040 0x7fe918071410 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe91809e0f0 tx=0x7fe908009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.984+0000 7fe91fe1c700 1 -- 172.21.15.156:0/1262155378 shutdown_connections 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.984+0000 7fe91fe1c700 1 --2- 172.21.15.156:0/1262155378 >> v2:172.21.15.186:3300/0 conn(0x7fe91806cdc0 0x7fe91806d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.984+0000 7fe91fe1c700 1 --2- 172.21.15.156:0/1262155378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe918071040 0x7fe918071410 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.984+0000 7fe91fe1c700 1 --2- 172.21.15.156:0/1262155378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe91806dee0 0x7fe91806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.984+0000 7fe91fe1c700 1 -- 172.21.15.156:0/1262155378 >> 172.21.15.156:0/1262155378 conn(0x7fe9180d0000 msgr2=0x7fe9180d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.984+0000 7fe91fe1c700 1 -- 172.21.15.156:0/1262155378 shutdown_connections 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.984+0000 7fe91fe1c700 1 -- 172.21.15.156:0/1262155378 wait complete. 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.985+0000 7fe91fe1c700 1 Processor -- start 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.985+0000 7fe91fe1c700 1 -- start start 2023-11-28T11:10:40.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91fe1c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe91806cdc0 0x7fe9180c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:40.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91fe1c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe91806dee0 0x7fe918157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:40.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91fe1c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe918071040 0x7fe9180b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:40.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91fe1c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180c4f00 con 0x7fe918071040 2023-11-28T11:10:40.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91fe1c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180c42f0 con 0x7fe91806dee0 2023-11-28T11:10:40.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91fe1c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180c45d0 con 0x7fe91806cdc0 2023-11-28T11:10:40.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91e3b9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe918071040 0x7fe9180b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:40.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91e3b9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe918071040 0x7fe9180b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35332/0 (socket says 172.21.15.156:35332) 2023-11-28T11:10:40.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91e3b9700 1 -- 172.21.15.156:0/3741400585 learned_addr learned my addr 172.21.15.156:0/3741400585 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:40.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91dbb8700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe91806cdc0 0x7fe9180c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:40.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91d3b7700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe91806dee0 0x7fe918157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:40.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91e3b9700 1 -- 172.21.15.156:0/3741400585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe91806cdc0 msgr2=0x7fe9180c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:40.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91e3b9700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe91806cdc0 0x7fe9180c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91e3b9700 1 -- 172.21.15.156:0/3741400585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe91806dee0 msgr2=0x7fe918157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:40.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91e3b9700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe91806dee0 0x7fe918157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:40.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91e3b9700 1 -- 172.21.15.156:0/3741400585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe908009550 con 0x7fe918071040 2023-11-28T11:10:40.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91d3b7700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe91806dee0 0x7fe918157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:40.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91e3b9700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe918071040 0x7fe9180b4fd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe910007d20 tx=0x7fe91000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:40.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe90e7fc700 1 -- 172.21.15.156:0/3741400585 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91000ede0 con 0x7fe918071040 2023-11-28T11:10:40.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe90e7fc700 1 -- 172.21.15.156:0/3741400585 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe910007e70 con 0x7fe918071040 2023-11-28T11:10:40.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91fe1c700 1 -- 172.21.15.156:0/3741400585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91806e870 con 0x7fe918071040 2023-11-28T11:10:40.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe90e7fc700 1 -- 172.21.15.156:0/3741400585 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910014640 con 0x7fe918071040 2023-11-28T11:10:40.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.986+0000 7fe91fe1c700 1 -- 172.21.15.156:0/3741400585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91815d900 con 0x7fe918071040 2023-11-28T11:10:40.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.987+0000 7fe91fe1c700 1 -- 172.21.15.156:0/3741400585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91815ccd0 con 0x7fe918071040 2023-11-28T11:10:40.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.988+0000 7fe90e7fc700 1 -- 172.21.15.156:0/3741400585 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe910012050 con 0x7fe918071040 2023-11-28T11:10:40.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.988+0000 7fe90e7fc700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe8fc0682f0 0x7fe8fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:40.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.988+0000 7fe90e7fc700 1 -- 172.21.15.156:0/3741400585 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fe910086210 con 0x7fe918071040 2023-11-28T11:10:40.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.988+0000 7fe91dbb8700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe8fc0682f0 0x7fe8fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:40.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.988+0000 7fe91dbb8700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe8fc0682f0 0x7fe8fc06a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe908014040 tx=0x7fe908008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:40.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:40.992+0000 7fe90e7fc700 1 -- 172.21.15.156:0/3741400585 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe910052980 con 0x7fe918071040 2023-11-28T11:10:41.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.221+0000 7fe91fe1c700 1 -- 172.21.15.156:0/3741400585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe9180c5040 con 0x7fe918071040 2023-11-28T11:10:41.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.842+0000 7fe90e7fc700 1 -- 172.21.15.156:0/3741400585 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe910055fa0 con 0x7fe918071040 2023-11-28T11:10:41.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.844+0000 7fe91fe1c700 1 -- 172.21.15.156:0/3741400585 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe8fc0682f0 msgr2=0x7fe8fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:41.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.844+0000 7fe91fe1c700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe8fc0682f0 0x7fe8fc06a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe908014040 tx=0x7fe908008030 comp rx=0 tx=0).stop 2023-11-28T11:10:41.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.844+0000 7fe91fe1c700 1 -- 172.21.15.156:0/3741400585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe918071040 msgr2=0x7fe9180b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:41.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.844+0000 7fe91fe1c700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe918071040 0x7fe9180b4fd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe910007d20 tx=0x7fe91000e3c0 comp rx=0 tx=0).stop 2023-11-28T11:10:41.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.844+0000 7fe91fe1c700 1 -- 172.21.15.156:0/3741400585 shutdown_connections 2023-11-28T11:10:41.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.844+0000 7fe91fe1c700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe918071040 0x7fe9180b4fd0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:41.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.844+0000 7fe91fe1c700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe8fc0682f0 0x7fe8fc06a7a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:41.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.844+0000 7fe91fe1c700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe91806cdc0 0x7fe9180c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:41.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.844+0000 7fe91fe1c700 1 --2- 172.21.15.156:0/3741400585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe91806dee0 0x7fe918157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:41.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.844+0000 7fe91fe1c700 1 -- 172.21.15.156:0/3741400585 >> 172.21.15.156:0/3741400585 conn(0x7fe9180d0000 msgr2=0x7fe9180cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:41.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.844+0000 7fe91fe1c700 1 -- 172.21.15.156:0/3741400585 shutdown_connections 2023-11-28T11:10:41.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.844+0000 7fe91fe1c700 1 -- 172.21.15.156:0/3741400585 wait complete. 2023-11-28T11:10:41.859 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:10:41.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.984+0000 7f30e4272700 1 Processor -- start 2023-11-28T11:10:41.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.984+0000 7f30e4272700 1 -- start start 2023-11-28T11:10:41.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.985+0000 7f30e4272700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f30dc0cc260 0x7f30dc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:41.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.985+0000 7f30e4272700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30dc0cc7b0 con 0x7f30dc0d0470 2023-11-28T11:10:41.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.985+0000 7f30e4272700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30dc0cd610 con 0x7f30dc1596b0 2023-11-28T11:10:41.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.985+0000 7f30e4272700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30dc159a80 con 0x7f30dc0cc260 2023-11-28T11:10:41.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.985+0000 7f30e280f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f30dc0cc260 0x7f30dc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:41.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.985+0000 7f30e180d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f30dc0d0470 0x7f30dc06cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50218/0 (socket says 172.21.15.156:50218) 2023-11-28T11:10:41.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.985+0000 7f30e280f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f30dc0cc260 0x7f30dc071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35338/0 (socket says 172.21.15.156:35338) 2023-11-28T11:10:41.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.985+0000 7f30e180d700 1 -- 172.21.15.156:0/2170969154 learned_addr learned my addr 172.21.15.156:0/2170969154 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:41.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.986+0000 7f30e280f700 1 -- 172.21.15.156:0/2170969154 >> v1:172.21.15.186:6789/0 conn(0x7f30dc1596b0 legacy=0x7f30dc0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:41.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.986+0000 7f30e280f700 1 -- 172.21.15.156:0/2170969154 >> v1:172.21.15.156:6789/0 conn(0x7f30dc0d0470 legacy=0x7f30dc06cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:41.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.986+0000 7f30e280f700 1 -- 172.21.15.156:0/2170969154 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc0cca00 con 0x7f30dc0cc260 2023-11-28T11:10:41.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.986+0000 7f30e100c700 1 -- 172.21.15.156:0/2170969154 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348682642 0 0) 0x7f30dc0cc7b0 con 0x7f30dc0d0470 2023-11-28T11:10:41.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.986+0000 7f30e280f700 1 --2- 172.21.15.156:0/2170969154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30dc0cc260 0x7f30dc071410 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009d90 tx=0x7f30d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08dd565fd36584f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:41.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.986+0000 7f30e100c700 1 -- 172.21.15.156:0/2170969154 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d400e070 con 0x7f30dc0cc260 2023-11-28T11:10:41.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.986+0000 7f30e100c700 1 -- 172.21.15.156:0/2170969154 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d400bb90 con 0x7f30dc0cc260 2023-11-28T11:10:41.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.986+0000 7f30e100c700 1 -- 172.21.15.156:0/2170969154 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4003a20 con 0x7f30dc0cc260 2023-11-28T11:10:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.986+0000 7f30e4272700 1 -- 172.21.15.156:0/2170969154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30dc0cc260 msgr2=0x7f30dc071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.986+0000 7f30e4272700 1 --2- 172.21.15.156:0/2170969154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30dc0cc260 0x7f30dc071410 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009d90 tx=0x7f30d400b330 comp rx=0 tx=0).stop 2023-11-28T11:10:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.986+0000 7f30e4272700 1 -- 172.21.15.156:0/2170969154 shutdown_connections 2023-11-28T11:10:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.986+0000 7f30e4272700 1 --2- 172.21.15.156:0/2170969154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30dc0cc260 0x7f30dc071410 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.987+0000 7f30e4272700 1 -- 172.21.15.156:0/2170969154 >> 172.21.15.156:0/2170969154 conn(0x7f30dc06b380 msgr2=0x7f30dc06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.987+0000 7f30e4272700 1 -- 172.21.15.156:0/2170969154 shutdown_connections 2023-11-28T11:10:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.987+0000 7f30e4272700 1 -- 172.21.15.156:0/2170969154 wait complete. 2023-11-28T11:10:41.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.988+0000 7f30e4272700 1 Processor -- start 2023-11-28T11:10:41.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.988+0000 7f30e4272700 1 -- start start 2023-11-28T11:10:41.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.988+0000 7f30e4272700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30dc0cc260 0x7f30dc0c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:41.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.988+0000 7f30e4272700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30dc0d0470 0x7f30dc15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:41.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.988+0000 7f30e4272700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30dc1596b0 0x7f30dc0c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:41.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.988+0000 7f30e4272700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc1f22a0 con 0x7f30dc0cc260 2023-11-28T11:10:41.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.988+0000 7f30e4272700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc0c89a0 con 0x7f30dc0d0470 2023-11-28T11:10:41.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.988+0000 7f30e4272700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc0c8c10 con 0x7f30dc1596b0 2023-11-28T11:10:41.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.988+0000 7f30e200e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30dc0cc260 0x7f30dc0c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:41.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30e200e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30dc0cc260 0x7f30dc0c3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35342/0 (socket says 172.21.15.156:35342) 2023-11-28T11:10:41.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30e200e700 1 -- 172.21.15.156:0/3280486329 learned_addr learned my addr 172.21.15.156:0/3280486329 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:41.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30e200e700 1 -- 172.21.15.156:0/3280486329 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30dc1596b0 msgr2=0x7f30dc0c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:41.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30e200e700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30dc1596b0 0x7f30dc0c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:41.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30e180d700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30dc0d0470 0x7f30dc15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:41.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30e200e700 1 -- 172.21.15.156:0/3280486329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30dc0d0470 msgr2=0x7f30dc15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:41.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30e200e700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30dc0d0470 0x7f30dc15c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:41.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30e200e700 1 -- 172.21.15.156:0/3280486329 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d4009a70 con 0x7f30dc0cc260 2023-11-28T11:10:41.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30e200e700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30dc0cc260 0x7f30dc0c3900 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00b4a0 tx=0x7f30cc00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:41.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30d2ffd700 1 -- 172.21.15.156:0/3280486329 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc011830 con 0x7f30dc0cc260 2023-11-28T11:10:41.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30d2ffd700 1 -- 172.21.15.156:0/3280486329 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc011990 con 0x7f30dc0cc260 2023-11-28T11:10:41.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30e4272700 1 -- 172.21.15.156:0/3280486329 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30dc0ccfc0 con 0x7f30dc0cc260 2023-11-28T11:10:41.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30d2ffd700 1 -- 172.21.15.156:0/3280486329 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc011d40 con 0x7f30dc0cc260 2023-11-28T11:10:41.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.989+0000 7f30e4272700 1 -- 172.21.15.156:0/3280486329 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30dc0ccce0 con 0x7f30dc0cc260 2023-11-28T11:10:41.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.990+0000 7f30e4272700 1 -- 172.21.15.156:0/3280486329 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30dc0cd2a0 con 0x7f30dc0cc260 2023-11-28T11:10:41.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.990+0000 7f30d2ffd700 1 -- 172.21.15.156:0/3280486329 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f30cc00f050 con 0x7f30dc0cc260 2023-11-28T11:10:41.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.991+0000 7f30d2ffd700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f30c00682f0 0x7f30c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:41.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.991+0000 7f30d2ffd700 1 -- 172.21.15.156:0/3280486329 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f30cc085890 con 0x7f30dc0cc260 2023-11-28T11:10:41.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.991+0000 7f30e180d700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f30c00682f0 0x7f30c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:41.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.991+0000 7f30e180d700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f30c00682f0 0x7f30c006a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f30dc1f5890 tx=0x7f30d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:41.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:41.995+0000 7f30d2ffd700 1 -- 172.21.15.156:0/3280486329 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f30cc052060 con 0x7f30dc0cc260 2023-11-28T11:10:42.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.175+0000 7f30e4272700 1 -- 172.21.15.156:0/3280486329 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f30dc068690 con 0x7f30dc0cc260 2023-11-28T11:10:42.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.844+0000 7f30d2ffd700 1 -- 172.21.15.156:0/3280486329 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) v1 ==== 181+0+0 (secure 0 0 0) 0x7f30cc055680 con 0x7f30dc0cc260 2023-11-28T11:10:42.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.848+0000 7f30e4272700 1 -- 172.21.15.156:0/3280486329 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f30c00682f0 msgr2=0x7f30c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.848+0000 7f30e4272700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f30c00682f0 0x7f30c006a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f30dc1f5890 tx=0x7f30d8009040 comp rx=0 tx=0).stop 2023-11-28T11:10:42.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.848+0000 7f30e4272700 1 -- 172.21.15.156:0/3280486329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30dc0cc260 msgr2=0x7f30dc0c3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.848+0000 7f30e4272700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30dc0cc260 0x7f30dc0c3900 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00b4a0 tx=0x7f30cc00b860 comp rx=0 tx=0).stop 2023-11-28T11:10:42.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.848+0000 7f30e4272700 1 -- 172.21.15.156:0/3280486329 shutdown_connections 2023-11-28T11:10:42.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.848+0000 7f30e4272700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f30c00682f0 0x7f30c006a7a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.848+0000 7f30e4272700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30dc1596b0 0x7f30dc0c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.848+0000 7f30e4272700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30dc0cc260 0x7f30dc0c3900 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.848+0000 7f30e4272700 1 --2- 172.21.15.156:0/3280486329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30dc0d0470 0x7f30dc15c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.848+0000 7f30e4272700 1 -- 172.21.15.156:0/3280486329 >> 172.21.15.156:0/3280486329 conn(0x7f30dc06b380 msgr2=0x7f30dc0b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:42.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.848+0000 7f30e4272700 1 -- 172.21.15.156:0/3280486329 shutdown_connections 2023-11-28T11:10:42.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.848+0000 7f30e4272700 1 -- 172.21.15.156:0/3280486329 wait complete. 2023-11-28T11:10:42.850 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:10:42.864 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:10:42.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.993+0000 7f2d484e5700 1 Processor -- start 2023-11-28T11:10:42.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.993+0000 7f2d484e5700 1 -- start start 2023-11-28T11:10:42.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.994+0000 7f2d484e5700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2d400b6ae0 0x7f2d400cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.994+0000 7f2d484e5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d400cfa20 0x7f2d400d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.994+0000 7f2d484e5700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4015af70 con 0x7f2d4015aba0 2023-11-28T11:10:42.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.994+0000 7f2d484e5700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d400cfec0 con 0x7f2d400b6ae0 2023-11-28T11:10:42.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.994+0000 7f2d484e5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400ccaa0 con 0x7f2d400cfa20 2023-11-28T11:10:42.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.994+0000 7f2d45a80700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2d400b6ae0 0x7f2d400cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.994+0000 7f2d45a80700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2d400b6ae0 0x7f2d400cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35352/0 (socket says 172.21.15.156:35352) 2023-11-28T11:10:42.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.994+0000 7f2d45a80700 1 -- 172.21.15.156:0/1156831018 learned_addr learned my addr 172.21.15.156:0/1156831018 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:42.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.994+0000 7f2d46a82700 1 --2- 172.21.15.156:0/1156831018 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d400cfa20 0x7f2d400d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.995+0000 7f2d45a80700 1 -- 172.21.15.156:0/1156831018 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d400cfa20 msgr2=0x7f2d400d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:42.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.995+0000 7f2d45a80700 1 --2- 172.21.15.156:0/1156831018 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d400cfa20 0x7f2d400d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.995+0000 7f2d45a80700 1 -- 172.21.15.156:0/1156831018 >> v1:172.21.15.186:6789/0 conn(0x7f2d4015aba0 legacy=0x7f2d400b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.995+0000 7f2d45a80700 1 -- 172.21.15.156:0/1156831018 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d400cc870 con 0x7f2d400b6ae0 2023-11-28T11:10:42.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.995+0000 7f2d45a80700 1 --2- 172.21.15.156:0/1156831018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d400b6ae0 0x7f2d400cf4e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0098f0 tx=0x7f2d3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c875c634367b4524 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.995+0000 7f2d4527f700 1 -- 172.21.15.156:0/1156831018 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c011830 con 0x7f2d400b6ae0 2023-11-28T11:10:42.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.995+0000 7f2d4527f700 1 -- 172.21.15.156:0/1156831018 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c011990 con 0x7f2d400b6ae0 2023-11-28T11:10:42.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.995+0000 7f2d4527f700 1 -- 172.21.15.156:0/1156831018 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c011d20 con 0x7f2d400b6ae0 2023-11-28T11:10:42.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.995+0000 7f2d484e5700 1 -- 172.21.15.156:0/1156831018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d400b6ae0 msgr2=0x7f2d400cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.995+0000 7f2d484e5700 1 --2- 172.21.15.156:0/1156831018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d400b6ae0 0x7f2d400cf4e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0098f0 tx=0x7f2d3c009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:42.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.995+0000 7f2d484e5700 1 -- 172.21.15.156:0/1156831018 shutdown_connections 2023-11-28T11:10:42.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.995+0000 7f2d484e5700 1 --2- 172.21.15.156:0/1156831018 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d400cfa20 0x7f2d400d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.996+0000 7f2d484e5700 1 --2- 172.21.15.156:0/1156831018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d400b6ae0 0x7f2d400cf4e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.996+0000 7f2d484e5700 1 -- 172.21.15.156:0/1156831018 >> 172.21.15.156:0/1156831018 conn(0x7f2d4006df70 msgr2=0x7f2d4006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:42.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.996+0000 7f2d484e5700 1 -- 172.21.15.156:0/1156831018 shutdown_connections 2023-11-28T11:10:42.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.996+0000 7f2d484e5700 1 -- 172.21.15.156:0/1156831018 wait complete. 2023-11-28T11:10:43.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.997+0000 7f2d484e5700 1 Processor -- start 2023-11-28T11:10:43.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.997+0000 7f2d484e5700 1 -- start start 2023-11-28T11:10:43.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.997+0000 7f2d484e5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d400cfa20 0x7f2d401525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.997+0000 7f2d484e5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d4015aba0 0x7f2d400c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.997+0000 7f2d484e5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d400c8510 0x7f2d40154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.997+0000 7f2d484e5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4015b430 con 0x7f2d400cfa20 2023-11-28T11:10:43.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.997+0000 7f2d484e5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d401540e0 con 0x7f2d4015aba0 2023-11-28T11:10:43.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.997+0000 7f2d484e5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d401534d0 con 0x7f2d400c8510 2023-11-28T11:10:43.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d46281700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d400cfa20 0x7f2d401525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d46281700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d400cfa20 0x7f2d401525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35356/0 (socket says 172.21.15.156:35356) 2023-11-28T11:10:43.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d46281700 1 -- 172.21.15.156:0/1175675048 learned_addr learned my addr 172.21.15.156:0/1175675048 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:43.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d46a82700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d400c8510 0x7f2d40154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d45a80700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d4015aba0 0x7f2d400c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d46281700 1 -- 172.21.15.156:0/1175675048 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d400c8510 msgr2=0x7f2d40154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d46281700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d400c8510 0x7f2d40154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d46281700 1 -- 172.21.15.156:0/1175675048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d4015aba0 msgr2=0x7f2d400c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d46281700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d4015aba0 0x7f2d400c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d46281700 1 -- 172.21.15.156:0/1175675048 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c009550 con 0x7f2d400cfa20 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d45a80700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d4015aba0 0x7f2d400c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d46281700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d400cfa20 0x7f2d401525d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000c480 tx=0x7f2d3000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d36ffd700 1 -- 172.21.15.156:0/1175675048 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300052d0 con 0x7f2d400cfa20 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d36ffd700 1 -- 172.21.15.156:0/1175675048 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d30005430 con 0x7f2d400cfa20 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d36ffd700 1 -- 172.21.15.156:0/1175675048 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300038c0 con 0x7f2d400cfa20 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d484e5700 1 -- 172.21.15.156:0/1175675048 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d400cd060 con 0x7f2d400cfa20 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.998+0000 7f2d484e5700 1 -- 172.21.15.156:0/1175675048 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d400ccdc0 con 0x7f2d400cfa20 2023-11-28T11:10:43.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:42.999+0000 7f2d484e5700 1 -- 172.21.15.156:0/1175675048 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d400cd340 con 0x7f2d400cfa20 2023-11-28T11:10:43.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.000+0000 7f2d36ffd700 1 -- 172.21.15.156:0/1175675048 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2d30019030 con 0x7f2d400cfa20 2023-11-28T11:10:43.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.000+0000 7f2d36ffd700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2d240682f0 0x7f2d2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.000+0000 7f2d36ffd700 1 -- 172.21.15.156:0/1175675048 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f2d30084d80 con 0x7f2d400cfa20 2023-11-28T11:10:43.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.000+0000 7f2d45a80700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2d240682f0 0x7f2d2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.000+0000 7f2d45a80700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2d240682f0 0x7f2d2406a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009c40 tx=0x7f2d3c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:43.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.004+0000 7f2d36ffd700 1 -- 172.21.15.156:0/1175675048 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2d30051700 con 0x7f2d400cfa20 2023-11-28T11:10:43.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.187+0000 7f2d484e5700 1 -- 172.21.15.156:0/1175675048 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2d401543c0 con 0x7f2d400cfa20 2023-11-28T11:10:43.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.851+0000 7f2d36ffd700 1 -- 172.21.15.156:0/1175675048 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2d300170d0 con 0x7f2d400cfa20 2023-11-28T11:10:43.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.855+0000 7f2d484e5700 1 -- 172.21.15.156:0/1175675048 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2d240682f0 msgr2=0x7f2d2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.855+0000 7f2d484e5700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2d240682f0 0x7f2d2406a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009c40 tx=0x7f2d3c008030 comp rx=0 tx=0).stop 2023-11-28T11:10:43.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.856+0000 7f2d484e5700 1 -- 172.21.15.156:0/1175675048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d400cfa20 msgr2=0x7f2d401525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.856+0000 7f2d484e5700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d400cfa20 0x7f2d401525d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000c480 tx=0x7f2d3000c790 comp rx=0 tx=0).stop 2023-11-28T11:10:43.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.856+0000 7f2d484e5700 1 -- 172.21.15.156:0/1175675048 shutdown_connections 2023-11-28T11:10:43.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.856+0000 7f2d484e5700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2d240682f0 0x7f2d2406a7a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.856+0000 7f2d484e5700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d400c8510 0x7f2d40154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.856+0000 7f2d484e5700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d400cfa20 0x7f2d401525d0 secure :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000c480 tx=0x7f2d3000c790 comp rx=0 tx=0).stop 2023-11-28T11:10:43.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.856+0000 7f2d484e5700 1 --2- 172.21.15.156:0/1175675048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d4015aba0 0x7f2d400c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.856+0000 7f2d484e5700 1 -- 172.21.15.156:0/1175675048 >> 172.21.15.156:0/1175675048 conn(0x7f2d4006df70 msgr2=0x7f2d400ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.856+0000 7f2d484e5700 1 -- 172.21.15.156:0/1175675048 shutdown_connections 2023-11-28T11:10:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:43.856+0000 7f2d484e5700 1 -- 172.21.15.156:0/1175675048 wait complete. 2023-11-28T11:10:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' removed 2023-11-28T11:10:43.871 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:10:44.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.002+0000 7fc9d091f700 1 Processor -- start 2023-11-28T11:10:44.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.002+0000 7fc9d091f700 1 -- start start 2023-11-28T11:10:44.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.003+0000 7fc9d091f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc9cc0cd8c0 0x7fc9cc140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.003+0000 7fc9d091f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9cc06f850 0x7fc9cc06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.003+0000 7fc9d091f700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc13ff00 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.003+0000 7fc9d091f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9cc1404c0 con 0x7fc9cc0cd8c0 2023-11-28T11:10:44.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.003+0000 7fc9d091f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc15aa80 con 0x7fc9cc06f850 2023-11-28T11:10:44.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.003+0000 7fc9ca59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc9cc0cd8c0 0x7fc9cc140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.003+0000 7fc9ca59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc9cc0cd8c0 0x7fc9cc140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35364/0 (socket says 172.21.15.156:35364) 2023-11-28T11:10:44.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.003+0000 7fc9ca59c700 1 -- 172.21.15.156:0/208579472 learned_addr learned my addr 172.21.15.156:0/208579472 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:44.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.003+0000 7fc9cad9d700 1 --2- 172.21.15.156:0/208579472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9cc06f850 0x7fc9cc06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.004+0000 7fc9ca59c700 1 -- 172.21.15.156:0/208579472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9cc06f850 msgr2=0x7fc9cc06fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.004+0000 7fc9ca59c700 1 --2- 172.21.15.156:0/208579472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9cc06f850 0x7fc9cc06fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.004+0000 7fc9ca59c700 1 -- 172.21.15.156:0/208579472 >> v1:172.21.15.186:6789/0 conn(0x7fc9cc0cdc90 legacy=0x7fc9cc0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.004+0000 7fc9ca59c700 1 -- 172.21.15.156:0/208579472 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc0b3040 con 0x7fc9cc0cd8c0 2023-11-28T11:10:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.004+0000 7fc9ca59c700 1 --2- 172.21.15.156:0/208579472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9cc0cd8c0 0x7fc9cc140950 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0006fd0 tx=0x7fc9c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4b55144451de31 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.004+0000 7fc9c959a700 1 -- 172.21.15.156:0/208579472 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0005e30 con 0x7fc9cc0cd8c0 2023-11-28T11:10:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.004+0000 7fc9c959a700 1 -- 172.21.15.156:0/208579472 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c0004030 con 0x7fc9cc0cd8c0 2023-11-28T11:10:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.004+0000 7fc9c959a700 1 -- 172.21.15.156:0/208579472 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00043e0 con 0x7fc9cc0cd8c0 2023-11-28T11:10:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.004+0000 7fc9d091f700 1 -- 172.21.15.156:0/208579472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9cc0cd8c0 msgr2=0x7fc9cc140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.004+0000 7fc9d091f700 1 --2- 172.21.15.156:0/208579472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9cc0cd8c0 0x7fc9cc140950 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0006fd0 tx=0x7fc9c0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.005+0000 7fc9d091f700 1 -- 172.21.15.156:0/208579472 shutdown_connections 2023-11-28T11:10:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.005+0000 7fc9d091f700 1 --2- 172.21.15.156:0/208579472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9cc06f850 0x7fc9cc06fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.005+0000 7fc9d091f700 1 --2- 172.21.15.156:0/208579472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9cc0cd8c0 0x7fc9cc140950 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.005+0000 7fc9d091f700 1 -- 172.21.15.156:0/208579472 >> 172.21.15.156:0/208579472 conn(0x7fc9cc06aed0 msgr2=0x7fc9cc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:44.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.005+0000 7fc9d091f700 1 -- 172.21.15.156:0/208579472 shutdown_connections 2023-11-28T11:10:44.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.005+0000 7fc9d091f700 1 -- 172.21.15.156:0/208579472 wait complete. 2023-11-28T11:10:44.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.006+0000 7fc9d091f700 1 Processor -- start 2023-11-28T11:10:44.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.006+0000 7fc9d091f700 1 -- start start 2023-11-28T11:10:44.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.007+0000 7fc9d091f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9cc06f850 0x7fc9cc15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.007+0000 7fc9d091f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9cc0cd8c0 0x7fc9cc15b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.007+0000 7fc9d091f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9cc0cdc90 0x7fc9cc1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.007+0000 7fc9d091f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc15cdb0 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.007+0000 7fc9d091f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc15bb80 con 0x7fc9cc0cd8c0 2023-11-28T11:10:44.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.007+0000 7fc9d091f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc142860 con 0x7fc9cc06f850 2023-11-28T11:10:44.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.007+0000 7fc9cad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9cc0cdc90 0x7fc9cc1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.007+0000 7fc9cad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9cc0cdc90 0x7fc9cc1579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35374/0 (socket says 172.21.15.156:35374) 2023-11-28T11:10:44.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.007+0000 7fc9cad9d700 1 -- 172.21.15.156:0/3478037852 learned_addr learned my addr 172.21.15.156:0/3478037852 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:44.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9ca59c700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9cc06f850 0x7fc9cc15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9cad9d700 1 -- 172.21.15.156:0/3478037852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9cc06f850 msgr2=0x7fc9cc15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9cad9d700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9cc06f850 0x7fc9cc15d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9c9d9b700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9cc0cd8c0 0x7fc9cc15b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9cad9d700 1 -- 172.21.15.156:0/3478037852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9cc0cd8c0 msgr2=0x7fc9cc15b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9cad9d700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9cc0cd8c0 0x7fc9cc15b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9cad9d700 1 -- 172.21.15.156:0/3478037852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c0009500 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9cad9d700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9cc0cdc90 0x7fc9cc1579a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400efc0 tx=0x7fc9c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9baffd700 1 -- 172.21.15.156:0/3478037852 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c400cd40 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9baffd700 1 -- 172.21.15.156:0/3478037852 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c400f040 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9d091f700 1 -- 172.21.15.156:0/3478037852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc0b35f0 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9baffd700 1 -- 172.21.15.156:0/3478037852 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c40146c0 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.008+0000 7fc9d091f700 1 -- 172.21.15.156:0/3478037852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9cc0b3310 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.009+0000 7fc9d091f700 1 -- 172.21.15.156:0/3478037852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9cc140150 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.009+0000 7fc9baffd700 1 -- 172.21.15.156:0/3478037852 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc9c4014820 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.010+0000 7fc9baffd700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc9ac0682f0 0x7fc9ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.010+0000 7fc9baffd700 1 -- 172.21.15.156:0/3478037852 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc9c4086490 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.010+0000 7fc9ca59c700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc9ac0682f0 0x7fc9ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.010+0000 7fc9ca59c700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc9ac0682f0 0x7fc9ac06a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0006fd0 tx=0x7fc9c0008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.014+0000 7fc9baffd700 1 -- 172.21.15.156:0/3478037852 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc9c4052f20 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.199+0000 7fc9d091f700 1 -- 172.21.15.156:0/3478037852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc9cc068690 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.202+0000 7fc9baffd700 1 -- 172.21.15.156:0/3478037852 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc9c400cea0 con 0x7fc9cc0cdc90 2023-11-28T11:10:44.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.204+0000 7fc9d091f700 1 -- 172.21.15.156:0/3478037852 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc9ac0682f0 msgr2=0x7fc9ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.204+0000 7fc9d091f700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc9ac0682f0 0x7fc9ac06a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0006fd0 tx=0x7fc9c0008070 comp rx=0 tx=0).stop 2023-11-28T11:10:44.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.204+0000 7fc9d091f700 1 -- 172.21.15.156:0/3478037852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9cc0cdc90 msgr2=0x7fc9cc1579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.204+0000 7fc9d091f700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9cc0cdc90 0x7fc9cc1579a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400efc0 tx=0x7fc9c400c3a0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.204+0000 7fc9d091f700 1 -- 172.21.15.156:0/3478037852 shutdown_connections 2023-11-28T11:10:44.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.204+0000 7fc9d091f700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9cc0cdc90 0x7fc9cc1579a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.204+0000 7fc9d091f700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc9ac0682f0 0x7fc9ac06a7a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.204+0000 7fc9d091f700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9cc06f850 0x7fc9cc15d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.204+0000 7fc9d091f700 1 --2- 172.21.15.156:0/3478037852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9cc0cd8c0 0x7fc9cc15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.204+0000 7fc9d091f700 1 -- 172.21.15.156:0/3478037852 >> 172.21.15.156:0/3478037852 conn(0x7fc9cc06aed0 msgr2=0x7fc9cc0cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:44.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.205+0000 7fc9d091f700 1 -- 172.21.15.156:0/3478037852 shutdown_connections 2023-11-28T11:10:44.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.205+0000 7fc9d091f700 1 -- 172.21.15.156:0/3478037852 wait complete. 2023-11-28T11:10:44.218 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T11:10:44.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.337+0000 7f7a16099700 1 Processor -- start 2023-11-28T11:10:44.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.338+0000 7f7a16099700 1 -- start start 2023-11-28T11:10:44.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.338+0000 7f7a16099700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7a1006dee0 0x7f7a1006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.338+0000 7f7a16099700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7a10160540 0x7f7a10160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.338+0000 7f7a16099700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1015d060 con 0x7f7a10071040 2023-11-28T11:10:44.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.338+0000 7f7a16099700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a1015d340 con 0x7f7a10160540 2023-11-28T11:10:44.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.338+0000 7f7a16099700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a1015d620 con 0x7f7a1006dee0 2023-11-28T11:10:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.338+0000 7f7a0ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7a10160540 0x7f7a10160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.339+0000 7f7a0f7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7a10071040 0x7f7a10071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50256/0 (socket says 172.21.15.156:50256) 2023-11-28T11:10:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.339+0000 7f7a0f7fe700 1 -- 172.21.15.156:0/354745458 learned_addr learned my addr 172.21.15.156:0/354745458 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.339+0000 7f7a0effd700 1 --2- 172.21.15.156:0/354745458 >> v2:172.21.15.186:3300/0 conn(0x7f7a1006dee0 0x7f7a1006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.339+0000 7f7a0ffff700 1 -- 172.21.15.156:0/354745458 >> v2:172.21.15.186:3300/0 conn(0x7f7a1006dee0 msgr2=0x7f7a1006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.339+0000 7f7a0ffff700 1 --2- 172.21.15.156:0/354745458 >> v2:172.21.15.186:3300/0 conn(0x7f7a1006dee0 0x7f7a1006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.339+0000 7f7a0ffff700 1 -- 172.21.15.156:0/354745458 >> v1:172.21.15.156:6789/0 conn(0x7f7a10071040 legacy=0x7f7a10071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.339+0000 7f7a0ffff700 1 -- 172.21.15.156:0/354745458 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1015d900 con 0x7f7a10160540 2023-11-28T11:10:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.339+0000 7f7a0e7fc700 1 -- 172.21.15.156:0/354745458 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127687613 0 0) 0x7f7a1015d060 con 0x7f7a10071040 2023-11-28T11:10:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.339+0000 7f7a0ffff700 1 --2- 172.21.15.156:0/354745458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a10160540 0x7f7a10160980 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7a1006fb00 tx=0x7f7a0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1c689a7fc10f76 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.339+0000 7f7a0e7fc700 1 -- 172.21.15.156:0/354745458 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0800e070 con 0x7f7a10160540 2023-11-28T11:10:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.339+0000 7f7a0e7fc700 1 -- 172.21.15.156:0/354745458 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0800bbd0 con 0x7f7a10160540 2023-11-28T11:10:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.339+0000 7f7a0e7fc700 1 -- 172.21.15.156:0/354745458 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08012430 con 0x7f7a10160540 2023-11-28T11:10:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.340+0000 7f7a16099700 1 -- 172.21.15.156:0/354745458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a10160540 msgr2=0x7f7a10160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.340+0000 7f7a16099700 1 --2- 172.21.15.156:0/354745458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a10160540 0x7f7a10160980 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7a1006fb00 tx=0x7f7a0800b330 comp rx=0 tx=0).stop 2023-11-28T11:10:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.340+0000 7f7a16099700 1 -- 172.21.15.156:0/354745458 shutdown_connections 2023-11-28T11:10:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.340+0000 7f7a16099700 1 --2- 172.21.15.156:0/354745458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a10160540 0x7f7a10160980 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.340+0000 7f7a16099700 1 --2- 172.21.15.156:0/354745458 >> v2:172.21.15.186:3300/0 conn(0x7f7a1006dee0 0x7f7a1006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.340+0000 7f7a16099700 1 -- 172.21.15.156:0/354745458 >> 172.21.15.156:0/354745458 conn(0x7f7a100d0000 msgr2=0x7f7a100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.340+0000 7f7a16099700 1 -- 172.21.15.156:0/354745458 shutdown_connections 2023-11-28T11:10:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.340+0000 7f7a16099700 1 -- 172.21.15.156:0/354745458 wait complete. 2023-11-28T11:10:44.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.341+0000 7f7a16099700 1 Processor -- start 2023-11-28T11:10:44.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.341+0000 7f7a16099700 1 -- start start 2023-11-28T11:10:44.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a16099700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a1006dee0 0x7f7a100cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a16099700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a10071040 0x7f7a10149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a16099700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a10160540 0x7f7a1014a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a16099700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1014a920 con 0x7f7a10071040 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a16099700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a100c5ad0 con 0x7f7a10160540 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a16099700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a100c5db0 con 0x7f7a1006dee0 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a0effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a10071040 0x7f7a10149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a0effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a10071040 0x7f7a10149ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35386/0 (socket says 172.21.15.156:35386) 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a0effd700 1 -- 172.21.15.156:0/572187229 learned_addr learned my addr 172.21.15.156:0/572187229 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a0ffff700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a10160540 0x7f7a1014a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a0f7fe700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a1006dee0 0x7f7a100cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a0effd700 1 -- 172.21.15.156:0/572187229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a1006dee0 msgr2=0x7f7a100cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a0effd700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a1006dee0 0x7f7a100cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a0effd700 1 -- 172.21.15.156:0/572187229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a10160540 msgr2=0x7f7a1014a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a0effd700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a10160540 0x7f7a1014a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a0effd700 1 -- 172.21.15.156:0/572187229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a08009a70 con 0x7f7a10071040 2023-11-28T11:10:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a0f7fe700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a1006dee0 0x7f7a100cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:44.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.342+0000 7f7a0effd700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a10071040 0x7f7a10149ea0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000d800 tx=0x7f7a0000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.343+0000 7f79f7fff700 1 -- 172.21.15.156:0/572187229 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a000125f0 con 0x7f7a10071040 2023-11-28T11:10:44.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.343+0000 7f79f7fff700 1 -- 172.21.15.156:0/572187229 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a00012750 con 0x7f7a10071040 2023-11-28T11:10:44.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.343+0000 7f7a16099700 1 -- 172.21.15.156:0/572187229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1015ccd0 con 0x7f7a10071040 2023-11-28T11:10:44.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.343+0000 7f79f7fff700 1 -- 172.21.15.156:0/572187229 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00012b00 con 0x7f7a10071040 2023-11-28T11:10:44.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.343+0000 7f7a16099700 1 -- 172.21.15.156:0/572187229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1006e870 con 0x7f7a10071040 2023-11-28T11:10:44.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.344+0000 7f7a16099700 1 -- 172.21.15.156:0/572187229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1006a270 con 0x7f7a10071040 2023-11-28T11:10:44.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.344+0000 7f79f7fff700 1 -- 172.21.15.156:0/572187229 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a00012c60 con 0x7f7a10071040 2023-11-28T11:10:44.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.344+0000 7f79f7fff700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f79fc0682f0 0x7f79fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.345+0000 7f79f7fff700 1 -- 172.21.15.156:0/572187229 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7a00085e90 con 0x7f7a10071040 2023-11-28T11:10:44.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.345+0000 7f7a0f7fe700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f79fc0682f0 0x7f79fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.345+0000 7f7a0f7fe700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f79fc0682f0 0x7f79fc06a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7a04000f60 tx=0x7f7a04007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.349+0000 7f79f7fff700 1 -- 172.21.15.156:0/572187229 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7a00052960 con 0x7f7a10071040 2023-11-28T11:10:44.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.604+0000 7f7a16099700 1 -- 172.21.15.156:0/572187229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f7a100720a0 con 0x7f7a10071040 2023-11-28T11:10:44.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.852+0000 7f79f7fff700 1 -- 172.21.15.156:0/572187229 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v139) v1 ==== 149+0+0 (secure 0 0 0) 0x7f7a00055f80 con 0x7f7a10071040 2023-11-28T11:10:44.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.854+0000 7f7a16099700 1 -- 172.21.15.156:0/572187229 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f79fc0682f0 msgr2=0x7f79fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.854+0000 7f7a16099700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f79fc0682f0 0x7f79fc06a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7a04000f60 tx=0x7f7a04007d30 comp rx=0 tx=0).stop 2023-11-28T11:10:44.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.854+0000 7f7a16099700 1 -- 172.21.15.156:0/572187229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a10071040 msgr2=0x7f7a10149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.854+0000 7f7a16099700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a10071040 0x7f7a10149ea0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000d800 tx=0x7f7a0000db10 comp rx=0 tx=0).stop 2023-11-28T11:10:44.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.854+0000 7f7a16099700 1 -- 172.21.15.156:0/572187229 shutdown_connections 2023-11-28T11:10:44.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.854+0000 7f7a16099700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a10160540 0x7f7a1014a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.854+0000 7f7a16099700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f79fc0682f0 0x7f79fc06a7a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.854+0000 7f7a16099700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a1006dee0 0x7f7a100cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.854+0000 7f7a16099700 1 --2- 172.21.15.156:0/572187229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a10071040 0x7f7a10149ea0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.854+0000 7f7a16099700 1 -- 172.21.15.156:0/572187229 >> 172.21.15.156:0/572187229 conn(0x7f7a100d0000 msgr2=0x7f7a1006a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:44.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.854+0000 7f7a16099700 1 -- 172.21.15.156:0/572187229 shutdown_connections 2023-11-28T11:10:44.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.854+0000 7f7a16099700 1 -- 172.21.15.156:0/572187229 wait complete. 2023-11-28T11:10:44.870 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2023-11-28T11:10:44.870 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2023-11-28T11:10:44.870 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:10:44.870 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T11:10:44.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.990+0000 7f797e2da700 1 Processor -- start 2023-11-28T11:10:44.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.990+0000 7f797e2da700 1 -- start start 2023-11-28T11:10:44.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.990+0000 7f797e2da700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f797815cef0 0x7f79780af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.990+0000 7f797e2da700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79780cc5f0 con 0x7f79780cbdf0 2023-11-28T11:10:44.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.990+0000 7f797e2da700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797815d470 con 0x7f797806ba90 2023-11-28T11:10:44.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.990+0000 7f797e2da700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797815d750 con 0x7f797815cef0 2023-11-28T11:10:44.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f79777fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f79780cbdf0 0x7f797806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50274/0 (socket says 172.21.15.156:50274) 2023-11-28T11:10:44.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f79777fe700 1 -- 172.21.15.156:0/4081475253 learned_addr learned my addr 172.21.15.156:0/4081475253 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:44.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f7977fff700 1 --2- 172.21.15.156:0/4081475253 >> v2:172.21.15.186:3300/0 conn(0x7f797815cef0 0x7f79780af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f7976ffd700 1 -- 172.21.15.156:0/4081475253 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496061725 0 0) 0x7f79780cc5f0 con 0x7f79780cbdf0 2023-11-28T11:10:44.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f7976ffd700 1 -- 172.21.15.156:0/4081475253 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79600035b0 con 0x7f79780cbdf0 2023-11-28T11:10:44.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f7977fff700 1 -- 172.21.15.156:0/4081475253 >> v1:172.21.15.186:6789/0 conn(0x7f797806ba90 legacy=0x7f79780ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f7976ffd700 1 -- 172.21.15.156:0/4081475253 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1127974045 0 0) 0x7f79600035b0 con 0x7f79780cbdf0 2023-11-28T11:10:44.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f7977fff700 1 -- 172.21.15.156:0/4081475253 >> v1:172.21.15.156:6789/0 conn(0x7f79780cbdf0 legacy=0x7f797806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f7977fff700 1 -- 172.21.15.156:0/4081475253 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797815da30 con 0x7f797815cef0 2023-11-28T11:10:44.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f7977fff700 1 --2- 172.21.15.156:0/4081475253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f797815cef0 0x7f79780af400 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f796c009870 tx=0x7f796c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=297fe9e873ac4493 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f7976ffd700 1 -- 172.21.15.156:0/4081475253 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968001670 con 0x7f797815cef0 2023-11-28T11:10:44.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f7976ffd700 1 -- 172.21.15.156:0/4081475253 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c004030 con 0x7f797815cef0 2023-11-28T11:10:44.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.991+0000 7f7976ffd700 1 -- 172.21.15.156:0/4081475253 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c0043a0 con 0x7f797815cef0 2023-11-28T11:10:44.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.992+0000 7f797e2da700 1 -- 172.21.15.156:0/4081475253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f797815cef0 msgr2=0x7f79780af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.992+0000 7f797e2da700 1 --2- 172.21.15.156:0/4081475253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f797815cef0 0x7f79780af400 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f796c009870 tx=0x7f796c009b80 comp rx=0 tx=0).stop 2023-11-28T11:10:44.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.992+0000 7f797e2da700 1 -- 172.21.15.156:0/4081475253 shutdown_connections 2023-11-28T11:10:44.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.992+0000 7f797e2da700 1 --2- 172.21.15.156:0/4081475253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f797815cef0 0x7f79780af400 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.992+0000 7f797e2da700 1 -- 172.21.15.156:0/4081475253 >> 172.21.15.156:0/4081475253 conn(0x7f79780d0000 msgr2=0x7f79780d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:44.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.992+0000 7f797e2da700 1 -- 172.21.15.156:0/4081475253 shutdown_connections 2023-11-28T11:10:44.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.992+0000 7f797e2da700 1 -- 172.21.15.156:0/4081475253 wait complete. 2023-11-28T11:10:44.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.993+0000 7f797e2da700 1 Processor -- start 2023-11-28T11:10:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.993+0000 7f797e2da700 1 -- start start 2023-11-28T11:10:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f797e2da700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f797806ba90 0x7f79780c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f797e2da700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79780cbdf0 0x7f79780c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f797e2da700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f797815cef0 0x7f797814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f797e2da700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797814b3b0 con 0x7f797806ba90 2023-11-28T11:10:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f797e2da700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79780c8e70 con 0x7f79780cbdf0 2023-11-28T11:10:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f797e2da700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79780c9150 con 0x7f797815cef0 2023-11-28T11:10:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f7977fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f797806ba90 0x7f79780c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f7977fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f797806ba90 0x7f79780c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35398/0 (socket says 172.21.15.156:35398) 2023-11-28T11:10:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f7977fff700 1 -- 172.21.15.156:0/2095676649 learned_addr learned my addr 172.21.15.156:0/2095676649 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:44.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f797c877700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f797815cef0 0x7f797814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f79777fe700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79780cbdf0 0x7f79780c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f7977fff700 1 -- 172.21.15.156:0/2095676649 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f797815cef0 msgr2=0x7f797814ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f7977fff700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f797815cef0 0x7f797814ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f7977fff700 1 -- 172.21.15.156:0/2095676649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79780cbdf0 msgr2=0x7f79780c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f7977fff700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79780cbdf0 0x7f79780c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.994+0000 7f7977fff700 1 -- 172.21.15.156:0/2095676649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c0094a0 con 0x7f797806ba90 2023-11-28T11:10:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.995+0000 7f7977fff700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f797806ba90 0x7f79780c70d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f796c000c00 tx=0x7f796c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.995+0000 7f7974ff9700 1 -- 172.21.15.156:0/2095676649 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c011830 con 0x7f797806ba90 2023-11-28T11:10:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.995+0000 7f7974ff9700 1 -- 172.21.15.156:0/2095676649 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c0119b0 con 0x7f797806ba90 2023-11-28T11:10:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.995+0000 7f797e2da700 1 -- 172.21.15.156:0/2095676649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797815dff0 con 0x7f797806ba90 2023-11-28T11:10:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.995+0000 7f7974ff9700 1 -- 172.21.15.156:0/2095676649 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c011d60 con 0x7f797806ba90 2023-11-28T11:10:45.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.995+0000 7f797e2da700 1 -- 172.21.15.156:0/2095676649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797815dd10 con 0x7f797806ba90 2023-11-28T11:10:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.996+0000 7f797e2da700 1 -- 172.21.15.156:0/2095676649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79780ca170 con 0x7f797806ba90 2023-11-28T11:10:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.996+0000 7f7974ff9700 1 -- 172.21.15.156:0/2095676649 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f796c017020 con 0x7f797806ba90 2023-11-28T11:10:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.997+0000 7f7974ff9700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f79640682f0 0x7f796406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.997+0000 7f7974ff9700 1 -- 172.21.15.156:0/2095676649 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f796c086ff0 con 0x7f797806ba90 2023-11-28T11:10:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.997+0000 7f79777fe700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f79640682f0 0x7f796406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:44.997+0000 7f79777fe700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f79640682f0 0x7f796406a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f79680011e0 tx=0x7f7968009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:45.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.001+0000 7f7974ff9700 1 -- 172.21.15.156:0/2095676649 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f796c053b50 con 0x7f797806ba90 2023-11-28T11:10:45.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.259+0000 7f797e2da700 1 -- 172.21.15.156:0/2095676649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f79780cad40 con 0x7f797806ba90 2023-11-28T11:10:45.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.854+0000 7f7974ff9700 1 -- 172.21.15.156:0/2095676649 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v140) v1 ==== 172+0+0 (secure 0 0 0) 0x7f796c057170 con 0x7f797806ba90 2023-11-28T11:10:45.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.857+0000 7f797e2da700 1 -- 172.21.15.156:0/2095676649 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f79640682f0 msgr2=0x7f796406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.857+0000 7f797e2da700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f79640682f0 0x7f796406a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f79680011e0 tx=0x7f7968009040 comp rx=0 tx=0).stop 2023-11-28T11:10:45.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.857+0000 7f797e2da700 1 -- 172.21.15.156:0/2095676649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f797806ba90 msgr2=0x7f79780c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.857+0000 7f797e2da700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f797806ba90 0x7f79780c70d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f796c000c00 tx=0x7f796c0047c0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.857+0000 7f797e2da700 1 -- 172.21.15.156:0/2095676649 shutdown_connections 2023-11-28T11:10:45.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.857+0000 7f797e2da700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f79640682f0 0x7f796406a7a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.857+0000 7f797e2da700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f797815cef0 0x7f797814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.857+0000 7f797e2da700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f797806ba90 0x7f79780c70d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.857+0000 7f797e2da700 1 --2- 172.21.15.156:0/2095676649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79780cbdf0 0x7f79780c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.857+0000 7f797e2da700 1 -- 172.21.15.156:0/2095676649 >> 172.21.15.156:0/2095676649 conn(0x7f79780d0000 msgr2=0x7f79780afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:45.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.857+0000 7f797e2da700 1 -- 172.21.15.156:0/2095676649 shutdown_connections 2023-11-28T11:10:45.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:45.857+0000 7f797e2da700 1 -- 172.21.15.156:0/2095676649 wait complete. 2023-11-28T11:10:45.873 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:10:46.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.002+0000 7fa6d5b6c700 1 Processor -- start 2023-11-28T11:10:46.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.003+0000 7fa6d5b6c700 1 -- start start 2023-11-28T11:10:46.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.003+0000 7fa6d5b6c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa6d0154e70 0x7fa6d00cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.003+0000 7fa6d5b6c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6d00cf360 0x7fa6d0071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.003+0000 7fa6d5b6c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d00719e0 con 0x7fa6d0071580 2023-11-28T11:10:46.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.003+0000 7fa6d5b6c700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d0155240 con 0x7fa6d0154e70 2023-11-28T11:10:46.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.003+0000 7fa6d5b6c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d015b100 con 0x7fa6d00cf360 2023-11-28T11:10:46.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.003+0000 7fa6cffff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa6d0071580 0x7fa6d015e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50288/0 (socket says 172.21.15.156:50288) 2023-11-28T11:10:46.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.003+0000 7fa6cffff700 1 -- 172.21.15.156:0/1990110581 learned_addr learned my addr 172.21.15.156:0/1990110581 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:46.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6ceffd700 1 --2- 172.21.15.156:0/1990110581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6d00cf360 0x7fa6d0071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6cf7fe700 1 --2- 172.21.15.156:0/1990110581 >> v2:172.21.15.186:3300/0 conn(0x7fa6d0154e70 0x7fa6d00cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6ce7fc700 1 -- 172.21.15.156:0/1990110581 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982339374 0 0) 0x7fa6d00719e0 con 0x7fa6d0071580 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6ce7fc700 1 -- 172.21.15.156:0/1990110581 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6b80035b0 con 0x7fa6d0071580 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6ce7fc700 1 -- 172.21.15.156:0/1990110581 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1781136294 0 0) 0x7fa6b80035b0 con 0x7fa6d0071580 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6ceffd700 1 -- 172.21.15.156:0/1990110581 >> v2:172.21.15.186:3300/0 conn(0x7fa6d0154e70 msgr2=0x7fa6d00cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6ceffd700 1 --2- 172.21.15.156:0/1990110581 >> v2:172.21.15.186:3300/0 conn(0x7fa6d0154e70 0x7fa6d00cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6ceffd700 1 -- 172.21.15.156:0/1990110581 >> v1:172.21.15.156:6789/0 conn(0x7fa6d0071580 legacy=0x7fa6d015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6ceffd700 1 -- 172.21.15.156:0/1990110581 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d015b3e0 con 0x7fa6d00cf360 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6ceffd700 1 --2- 172.21.15.156:0/1990110581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6d00cf360 0x7fa6d0071040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009840 tx=0x7fa6c0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b62d26acc3da7d11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6ce7fc700 1 -- 172.21.15.156:0/1990110581 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c80019f0 con 0x7fa6d00cf360 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6ce7fc700 1 -- 172.21.15.156:0/1990110581 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c0004030 con 0x7fa6d00cf360 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.004+0000 7fa6ce7fc700 1 -- 172.21.15.156:0/1990110581 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c00043c0 con 0x7fa6d00cf360 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.005+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/1990110581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6d00cf360 msgr2=0x7fa6d0071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.005+0000 7fa6d5b6c700 1 --2- 172.21.15.156:0/1990110581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6d00cf360 0x7fa6d0071040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009840 tx=0x7fa6c0009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.005+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/1990110581 shutdown_connections 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.005+0000 7fa6d5b6c700 1 --2- 172.21.15.156:0/1990110581 >> v2:172.21.15.186:3300/0 conn(0x7fa6d0154e70 0x7fa6d00cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.005+0000 7fa6d5b6c700 1 --2- 172.21.15.156:0/1990110581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6d00cf360 0x7fa6d0071040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.005+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/1990110581 >> 172.21.15.156:0/1990110581 conn(0x7fa6d00d0000 msgr2=0x7fa6d00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.005+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/1990110581 shutdown_connections 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.005+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/1990110581 wait complete. 2023-11-28T11:10:46.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.006+0000 7fa6d5b6c700 1 Processor -- start 2023-11-28T11:10:46.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.006+0000 7fa6d5b6c700 1 -- start start 2023-11-28T11:10:46.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6d5b6c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6d0071580 0x7fa6d015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6d5b6c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6d00cf360 0x7fa6d00c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6d5b6c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6d0154e70 0x7fa6d01e3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6d5b6c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d0152dd0 con 0x7fa6d00cf360 2023-11-28T11:10:46.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6d5b6c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d015d010 con 0x7fa6d0071580 2023-11-28T11:10:46.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6d5b6c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d015d2f0 con 0x7fa6d0154e70 2023-11-28T11:10:46.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6ceffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6d00cf360 0x7fa6d00c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6ceffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6d00cf360 0x7fa6d00c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35414/0 (socket says 172.21.15.156:35414) 2023-11-28T11:10:46.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6ceffd700 1 -- 172.21.15.156:0/2157457531 learned_addr learned my addr 172.21.15.156:0/2157457531 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:46.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6cffff700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6d0154e70 0x7fa6d01e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6cf7fe700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6d0071580 0x7fa6d015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6ceffd700 1 -- 172.21.15.156:0/2157457531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6d0154e70 msgr2=0x7fa6d01e3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6ceffd700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6d0154e70 0x7fa6d01e3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6ceffd700 1 -- 172.21.15.156:0/2157457531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6d0071580 msgr2=0x7fa6d015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6ceffd700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6d0071580 0x7fa6d015c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.007+0000 7fa6ceffd700 1 -- 172.21.15.156:0/2157457531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c00094a0 con 0x7fa6d00cf360 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.008+0000 7fa6ceffd700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6d00cf360 0x7fa6d00c6110 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000a010 tx=0x7fa6c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.008+0000 7fa6b7fff700 1 -- 172.21.15.156:0/2157457531 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0003e10 con 0x7fa6d00cf360 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.008+0000 7fa6b7fff700 1 -- 172.21.15.156:0/2157457531 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c001f070 con 0x7fa6d00cf360 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.008+0000 7fa6b7fff700 1 -- 172.21.15.156:0/2157457531 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0011870 con 0x7fa6d00cf360 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.008+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/2157457531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d015b9a0 con 0x7fa6d00cf360 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.008+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/2157457531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d015b6c0 con 0x7fa6d00cf360 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.009+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/2157457531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d00af520 con 0x7fa6d00cf360 2023-11-28T11:10:46.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.009+0000 7fa6b7fff700 1 -- 172.21.15.156:0/2157457531 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa6c0017020 con 0x7fa6d00cf360 2023-11-28T11:10:46.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.009+0000 7fa6b7fff700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa6bc0682f0 0x7fa6bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.010+0000 7fa6b7fff700 1 -- 172.21.15.156:0/2157457531 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa6c0021070 con 0x7fa6d00cf360 2023-11-28T11:10:46.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.010+0000 7fa6cf7fe700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa6bc0682f0 0x7fa6bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.010+0000 7fa6cf7fe700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa6bc0682f0 0x7fa6bc06a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009a10 tx=0x7fa6c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.014+0000 7fa6b7fff700 1 -- 172.21.15.156:0/2157457531 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa6c0053e40 con 0x7fa6d00cf360 2023-11-28T11:10:46.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.197+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/2157457531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa6d0071b40 con 0x7fa6d00cf360 2023-11-28T11:10:46.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.200+0000 7fa6b7fff700 1 -- 172.21.15.156:0/2157457531 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa6c0057460 con 0x7fa6d00cf360 2023-11-28T11:10:46.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.202+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/2157457531 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa6bc0682f0 msgr2=0x7fa6bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.202+0000 7fa6d5b6c700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa6bc0682f0 0x7fa6bc06a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009a10 tx=0x7fa6c4008040 comp rx=0 tx=0).stop 2023-11-28T11:10:46.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.202+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/2157457531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6d00cf360 msgr2=0x7fa6d00c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.202+0000 7fa6d5b6c700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6d00cf360 0x7fa6d00c6110 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000a010 tx=0x7fa6c0004750 comp rx=0 tx=0).stop 2023-11-28T11:10:46.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.203+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/2157457531 shutdown_connections 2023-11-28T11:10:46.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.203+0000 7fa6d5b6c700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa6bc0682f0 0x7fa6bc06a7a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.203+0000 7fa6d5b6c700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6d0154e70 0x7fa6d01e3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.203+0000 7fa6d5b6c700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6d0071580 0x7fa6d015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.203+0000 7fa6d5b6c700 1 --2- 172.21.15.156:0/2157457531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6d00cf360 0x7fa6d00c6110 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.203+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/2157457531 >> 172.21.15.156:0/2157457531 conn(0x7fa6d00d0000 msgr2=0x7fa6d006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:46.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.203+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/2157457531 shutdown_connections 2023-11-28T11:10:46.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.203+0000 7fa6d5b6c700 1 -- 172.21.15.156:0/2157457531 wait complete. 2023-11-28T11:10:46.216 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:10:46.216 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:10:46.255 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:10:46.304 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:10:46.304 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:10:46.304 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:46.323 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2550 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:10:46.323 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:10:46.324 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:46.344 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:10:46.345 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:10:46.345 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:46.381 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2550 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:10:46.381 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:10:46.381 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:46.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:10:46.407 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:10:46.407 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:46.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.539+0000 7f6ae4503700 1 Processor -- start 2023-11-28T11:10:46.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.539+0000 7f6ae4503700 1 -- start start 2023-11-28T11:10:46.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.540+0000 7f6ae4503700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6adc06ba90 0x7f6adc0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.540+0000 7f6ae4503700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6adc0cbdf0 0x7f6adc0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.540+0000 7f6ae4503700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6adc0cc5f0 con 0x7f6adc15cef0 2023-11-28T11:10:46.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.540+0000 7f6ae4503700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6adc15d470 con 0x7f6adc0cbdf0 2023-11-28T11:10:46.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.540+0000 7f6ae4503700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc15d750 con 0x7f6adc06ba90 2023-11-28T11:10:46.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.540+0000 7f6ae2aa0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6adc0cbdf0 0x7f6adc0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.540+0000 7f6ae229f700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f6adc15cef0 0x7f6adc06b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50298/0 (socket says 172.21.15.156:50298) 2023-11-28T11:10:46.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.540+0000 7f6ae2aa0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6adc0cbdf0 0x7f6adc0ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35424/0 (socket says 172.21.15.156:35424) 2023-11-28T11:10:46.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.540+0000 7f6ae229f700 1 -- 172.21.15.156:0/112157134 learned_addr learned my addr 172.21.15.156:0/112157134 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:46.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.541+0000 7f6ae1a9e700 1 --2- 172.21.15.156:0/112157134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6adc06ba90 0x7f6adc0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.541+0000 7f6ae2aa0700 1 -- 172.21.15.156:0/112157134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6adc06ba90 msgr2=0x7f6adc0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:46.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.541+0000 7f6ae2aa0700 1 --2- 172.21.15.156:0/112157134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6adc06ba90 0x7f6adc0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.541+0000 7f6ae2aa0700 1 -- 172.21.15.156:0/112157134 >> v1:172.21.15.156:6789/0 conn(0x7f6adc15cef0 legacy=0x7f6adc06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.541+0000 7f6ae2aa0700 1 -- 172.21.15.156:0/112157134 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc15da30 con 0x7f6adc0cbdf0 2023-11-28T11:10:46.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.541+0000 7f6ae129d700 1 -- 172.21.15.156:0/112157134 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205925679 0 0) 0x7f6adc0cc5f0 con 0x7f6adc15cef0 2023-11-28T11:10:46.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.541+0000 7f6ae2aa0700 1 --2- 172.21.15.156:0/112157134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc0cbdf0 0x7f6adc0ca650 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009d90 tx=0x7f6ad400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882cf4d1de485048 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.541+0000 7f6ae129d700 1 -- 172.21.15.156:0/112157134 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad400e070 con 0x7f6adc0cbdf0 2023-11-28T11:10:46.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.541+0000 7f6ae129d700 1 -- 172.21.15.156:0/112157134 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad400bbd0 con 0x7f6adc0cbdf0 2023-11-28T11:10:46.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.541+0000 7f6ae129d700 1 -- 172.21.15.156:0/112157134 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4012430 con 0x7f6adc0cbdf0 2023-11-28T11:10:46.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.541+0000 7f6ae4503700 1 -- 172.21.15.156:0/112157134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc0cbdf0 msgr2=0x7f6adc0ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.541+0000 7f6ae4503700 1 --2- 172.21.15.156:0/112157134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc0cbdf0 0x7f6adc0ca650 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009d90 tx=0x7f6ad400b330 comp rx=0 tx=0).stop 2023-11-28T11:10:46.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.542+0000 7f6ae4503700 1 -- 172.21.15.156:0/112157134 shutdown_connections 2023-11-28T11:10:46.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.542+0000 7f6ae4503700 1 --2- 172.21.15.156:0/112157134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc0cbdf0 0x7f6adc0ca650 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.542+0000 7f6ae4503700 1 --2- 172.21.15.156:0/112157134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6adc06ba90 0x7f6adc0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.542+0000 7f6ae4503700 1 -- 172.21.15.156:0/112157134 >> 172.21.15.156:0/112157134 conn(0x7f6adc0d0000 msgr2=0x7f6adc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:46.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.542+0000 7f6ae4503700 1 -- 172.21.15.156:0/112157134 shutdown_connections 2023-11-28T11:10:46.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.542+0000 7f6ae4503700 1 -- 172.21.15.156:0/112157134 wait complete. 2023-11-28T11:10:46.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.543+0000 7f6ae4503700 1 Processor -- start 2023-11-28T11:10:46.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.543+0000 7f6ae4503700 1 -- start start 2023-11-28T11:10:46.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.543+0000 7f6ae4503700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc06ba90 0x7f6adc151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.543+0000 7f6ae4503700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6adc0cbdf0 0x7f6adc0b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.543+0000 7f6ae4503700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6adc15cef0 0x7f6adc0caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae4503700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc09bb10 con 0x7f6adc06ba90 2023-11-28T11:10:46.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae4503700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0b4920 con 0x7f6adc15cef0 2023-11-28T11:10:46.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae4503700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0b4c00 con 0x7f6adc0cbdf0 2023-11-28T11:10:46.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae229f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc06ba90 0x7f6adc151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae229f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc06ba90 0x7f6adc151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35426/0 (socket says 172.21.15.156:35426) 2023-11-28T11:10:46.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae229f700 1 -- 172.21.15.156:0/1180412339 learned_addr learned my addr 172.21.15.156:0/1180412339 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:46.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae2aa0700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6adc15cef0 0x7f6adc0caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae1a9e700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6adc0cbdf0 0x7f6adc0b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae229f700 1 -- 172.21.15.156:0/1180412339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6adc0cbdf0 msgr2=0x7f6adc0b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae229f700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6adc0cbdf0 0x7f6adc0b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae229f700 1 -- 172.21.15.156:0/1180412339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6adc15cef0 msgr2=0x7f6adc0caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae229f700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6adc15cef0 0x7f6adc0caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae229f700 1 -- 172.21.15.156:0/1180412339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4009a70 con 0x7f6adc06ba90 2023-11-28T11:10:46.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae1a9e700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6adc0cbdf0 0x7f6adc0b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:46.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae229f700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc06ba90 0x7f6adc151420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00c550 tx=0x7f6acc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ad2ffd700 1 -- 172.21.15.156:0/1180412339 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc004c80 con 0x7f6adc06ba90 2023-11-28T11:10:46.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ad2ffd700 1 -- 172.21.15.156:0/1180412339 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc004e00 con 0x7f6adc06ba90 2023-11-28T11:10:46.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ad2ffd700 1 -- 172.21.15.156:0/1180412339 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc007310 con 0x7f6adc06ba90 2023-11-28T11:10:46.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae4503700 1 -- 172.21.15.156:0/1180412339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc15dff0 con 0x7f6adc06ba90 2023-11-28T11:10:46.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.544+0000 7f6ae4503700 1 -- 172.21.15.156:0/1180412339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc15dd10 con 0x7f6adc06ba90 2023-11-28T11:10:46.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.545+0000 7f6ae4503700 1 -- 172.21.15.156:0/1180412339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6adc0cab90 con 0x7f6adc06ba90 2023-11-28T11:10:46.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.545+0000 7f6ad2ffd700 1 -- 172.21.15.156:0/1180412339 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6acc01a020 con 0x7f6adc06ba90 2023-11-28T11:10:46.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.546+0000 7f6ad2ffd700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6ac00682f0 0x7f6ac006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.546+0000 7f6ad2ffd700 1 -- 172.21.15.156:0/1180412339 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6acc0854a0 con 0x7f6adc06ba90 2023-11-28T11:10:46.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.546+0000 7f6ae1a9e700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6ac00682f0 0x7f6ac006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.546+0000 7f6ae1a9e700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6ac00682f0 0x7f6ac006a7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0b57f0 tx=0x7f6ad8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.550+0000 7f6ad2ffd700 1 -- 172.21.15.156:0/1180412339 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6acc052000 con 0x7f6adc06ba90 2023-11-28T11:10:46.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.776+0000 7f6ae4503700 1 -- 172.21.15.156:0/1180412339 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6adc0cde40 con 0x7f6adc06ba90 2023-11-28T11:10:46.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.776+0000 7f6ad2ffd700 1 -- 172.21.15.156:0/1180412339 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f6acc055620 con 0x7f6adc06ba90 2023-11-28T11:10:46.778 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:46.778 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/365552946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":365552946},{"type":"v1","addr":"172.21.15.186:6834","nonce":365552946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1918325985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":1918325985},{"type":"v1","addr":"172.21.15.156:6837","nonce":1918325985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/2889641252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2889641252},{"type":"v1","addr":"172.21.15.156:6836","nonce":2889641252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1704773839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":1704773839},{"type":"v1","addr":"172.21.15.186:6835","nonce":1704773839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[]} 2023-11-28T11:10:46.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.778+0000 7f6ae4503700 1 -- 172.21.15.156:0/1180412339 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6ac00682f0 msgr2=0x7f6ac006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.778+0000 7f6ae4503700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6ac00682f0 0x7f6ac006a7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0b57f0 tx=0x7f6ad8008040 comp rx=0 tx=0).stop 2023-11-28T11:10:46.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.778+0000 7f6ae4503700 1 -- 172.21.15.156:0/1180412339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc06ba90 msgr2=0x7f6adc151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.778+0000 7f6ae4503700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc06ba90 0x7f6adc151420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00c550 tx=0x7f6acc00c860 comp rx=0 tx=0).stop 2023-11-28T11:10:46.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.778+0000 7f6ae4503700 1 -- 172.21.15.156:0/1180412339 shutdown_connections 2023-11-28T11:10:46.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.778+0000 7f6ae4503700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6adc15cef0 0x7f6adc0caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.778+0000 7f6ae4503700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc06ba90 0x7f6adc151420 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.778+0000 7f6ae4503700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f6ac00682f0 0x7f6ac006a7a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.778+0000 7f6ae4503700 1 --2- 172.21.15.156:0/1180412339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6adc0cbdf0 0x7f6adc0b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.778+0000 7f6ae4503700 1 -- 172.21.15.156:0/1180412339 >> 172.21.15.156:0/1180412339 conn(0x7f6adc0d0000 msgr2=0x7f6adc15c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:46.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.778+0000 7f6ae4503700 1 -- 172.21.15.156:0/1180412339 shutdown_connections 2023-11-28T11:10:46.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.778+0000 7f6ae4503700 1 -- 172.21.15.156:0/1180412339 wait complete. 2023-11-28T11:10:46.781 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 25 2023-11-28T11:10:46.792 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:10:46.792 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:10:46.792 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:10:46.792 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:46.792 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:10:46.792 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:10:46.793 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:10:46.793 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:46.793 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:10:46.793 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:10:46.793 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:10:46.793 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:46.793 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:10:46.794 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:10:46.794 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:10:46.794 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:46.801 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:10:46.801 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:10:46.802 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:10:46.802 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:10:46.805 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:10:46.806 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:10:46.806 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:10:46.807 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:10:46.844 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:10:46.845 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:46.845 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:10:46.845 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:46.849 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:10:46.849 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:46.849 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:10:46.850 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:46.851 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:10:46.925 INFO:tasks.ceph.mds.c.smithi156.stdout:starting mds.c at 2023-11-28T11:10:46.925 INFO:tasks.ceph.mds.a.smithi156.stdout:starting mds.a at 2023-11-28T11:10:46.939 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2023-11-28T11:10:46.939 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-11-28T11:10:46.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.981+0000 7fd767e15700 1 Processor -- start 2023-11-28T11:10:46.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.981+0000 7fd767e15700 1 -- start start 2023-11-28T11:10:46.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd767e15700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd76015cd30 0x7fd760161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd767e15700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7600cfda0 0x7fd7600d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd767e15700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd76006a270 0x7fd76006a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd767e15700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76006ac30 con 0x7fd76015cd30 2023-11-28T11:10:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd767e15700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7600d0840 con 0x7fd76006a270 2023-11-28T11:10:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd767e15700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7600d0b20 con 0x7fd7600cfda0 2023-11-28T11:10:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd765bb1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd76015cd30 0x7fd760161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd765bb1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd76015cd30 0x7fd760161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35458/0 (socket says 172.21.15.156:35458) 2023-11-28T11:10:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd765bb1700 1 -- 172.21.15.156:0/677635287 learned_addr learned my addr 172.21.15.156:0/677635287 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd7663b2700 1 --2- 172.21.15.156:0/677635287 >> v2:172.21.15.186:3300/0 conn(0x7fd76006a270 0x7fd76006a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd7653b0700 1 --2- 172.21.15.156:0/677635287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7600cfda0 0x7fd7600d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd765bb1700 1 -- 172.21.15.156:0/677635287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7600cfda0 msgr2=0x7fd7600d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd765bb1700 1 --2- 172.21.15.156:0/677635287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7600cfda0 0x7fd7600d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd765bb1700 1 -- 172.21.15.156:0/677635287 >> v2:172.21.15.186:3300/0 conn(0x7fd76006a270 msgr2=0x7fd76006a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:46.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd765bb1700 1 --2- 172.21.15.156:0/677635287 >> v2:172.21.15.186:3300/0 conn(0x7fd76006a270 0x7fd76006a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd765bb1700 1 -- 172.21.15.156:0/677635287 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd760160670 con 0x7fd76015cd30 2023-11-28T11:10:46.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd7663b2700 1 --2- 172.21.15.156:0/677635287 >> v2:172.21.15.186:3300/0 conn(0x7fd76006a270 0x7fd76006a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:46.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd7653b0700 1 --2- 172.21.15.156:0/677635287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7600cfda0 0x7fd7600d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:46.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd765bb1700 1 --2- 172.21.15.156:0/677635287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76015cd30 0x7fd760161150 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd7600cd8b0 tx=0x7fd750009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4cd5f5b5b62c8c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd764baf700 1 -- 172.21.15.156:0/677635287 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75000b870 con 0x7fd76015cd30 2023-11-28T11:10:46.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd764baf700 1 -- 172.21.15.156:0/677635287 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75000b9d0 con 0x7fd76015cd30 2023-11-28T11:10:46.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.982+0000 7fd764baf700 1 -- 172.21.15.156:0/677635287 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75000bd60 con 0x7fd76015cd30 2023-11-28T11:10:46.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.983+0000 7fd767e15700 1 -- 172.21.15.156:0/677635287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76015cd30 msgr2=0x7fd760161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.983+0000 7fd767e15700 1 --2- 172.21.15.156:0/677635287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76015cd30 0x7fd760161150 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd7600cd8b0 tx=0x7fd750009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.983+0000 7fd767e15700 1 -- 172.21.15.156:0/677635287 shutdown_connections 2023-11-28T11:10:46.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.983+0000 7fd767e15700 1 --2- 172.21.15.156:0/677635287 >> v2:172.21.15.186:3300/0 conn(0x7fd76006a270 0x7fd76006a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.983+0000 7fd767e15700 1 --2- 172.21.15.156:0/677635287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76015cd30 0x7fd760161150 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.983+0000 7fd767e15700 1 --2- 172.21.15.156:0/677635287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7600cfda0 0x7fd7600d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.983+0000 7fd767e15700 1 -- 172.21.15.156:0/677635287 >> 172.21.15.156:0/677635287 conn(0x7fd76006b380 msgr2=0x7fd76006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:46.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.983+0000 7fd767e15700 1 -- 172.21.15.156:0/677635287 shutdown_connections 2023-11-28T11:10:46.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.983+0000 7fd767e15700 1 -- 172.21.15.156:0/677635287 wait complete. 2023-11-28T11:10:46.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.984+0000 7fd767e15700 1 Processor -- start 2023-11-28T11:10:46.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.984+0000 7fd767e15700 1 -- start start 2023-11-28T11:10:46.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd767e15700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd76006a270 0x7fd760147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd767e15700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7600cfda0 0x7fd7600b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd767e15700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76015cd30 0x7fd7600c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd767e15700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7600ca280 con 0x7fd76015cd30 2023-11-28T11:10:46.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd767e15700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd760148120 con 0x7fd76006a270 2023-11-28T11:10:46.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd767e15700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7600b4070 con 0x7fd7600cfda0 2023-11-28T11:10:46.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd7663b2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76015cd30 0x7fd7600c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd7663b2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76015cd30 0x7fd7600c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35468/0 (socket says 172.21.15.156:35468) 2023-11-28T11:10:46.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd7663b2700 1 -- 172.21.15.156:0/2212507271 learned_addr learned my addr 172.21.15.156:0/2212507271 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd7653b0700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7600cfda0 0x7fd7600b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd765bb1700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd76006a270 0x7fd760147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd7663b2700 1 -- 172.21.15.156:0/2212507271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7600cfda0 msgr2=0x7fd7600b5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd7663b2700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7600cfda0 0x7fd7600b5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd7663b2700 1 -- 172.21.15.156:0/2212507271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd76006a270 msgr2=0x7fd760147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd7663b2700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd76006a270 0x7fd760147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd7663b2700 1 -- 172.21.15.156:0/2212507271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd750009550 con 0x7fd76015cd30 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.985+0000 7fd7663b2700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76015cd30 0x7fd7600c6470 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd758007e40 tx=0x7fd75800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.986+0000 7fd7567fc700 1 -- 172.21.15.156:0/2212507271 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75800dd20 con 0x7fd76015cd30 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.986+0000 7fd7567fc700 1 -- 172.21.15.156:0/2212507271 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75800f040 con 0x7fd76015cd30 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.986+0000 7fd767e15700 1 -- 172.21.15.156:0/2212507271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd760160c30 con 0x7fd76015cd30 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.986+0000 7fd7567fc700 1 -- 172.21.15.156:0/2212507271 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd758003c60 con 0x7fd76015cd30 2023-11-28T11:10:46.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.986+0000 7fd767e15700 1 -- 172.21.15.156:0/2212507271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd760160950 con 0x7fd76015cd30 2023-11-28T11:10:46.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.986+0000 7fd767e15700 1 -- 172.21.15.156:0/2212507271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7600cc280 con 0x7fd76015cd30 2023-11-28T11:10:46.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.987+0000 7fd7567fc700 1 -- 172.21.15.156:0/2212507271 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd75801c030 con 0x7fd76015cd30 2023-11-28T11:10:46.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.987+0000 7fd7567fc700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7440682f0 0x7fd74406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.987+0000 7fd7567fc700 1 -- 172.21.15.156:0/2212507271 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd758085820 con 0x7fd76015cd30 2023-11-28T11:10:46.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.987+0000 7fd765bb1700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7440682f0 0x7fd74406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.988+0000 7fd765bb1700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7440682f0 0x7fd74406a7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd750014040 tx=0x7fd750008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:46.992+0000 7fd7567fc700 1 -- 172.21.15.156:0/2212507271 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd758052380 con 0x7fd76015cd30 2023-11-28T11:10:47.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.191+0000 7fd767e15700 1 -- 172.21.15.156:0/2212507271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd76006b130 con 0x7fd76015cd30 2023-11-28T11:10:47.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.962+0000 7fd7567fc700 1 -- 172.21.15.156:0/2212507271 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd75801a080 con 0x7fd76015cd30 2023-11-28T11:10:47.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.965+0000 7fd767e15700 1 -- 172.21.15.156:0/2212507271 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7440682f0 msgr2=0x7fd74406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:47.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.965+0000 7fd767e15700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7440682f0 0x7fd74406a7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd750014040 tx=0x7fd750008030 comp rx=0 tx=0).stop 2023-11-28T11:10:47.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.965+0000 7fd767e15700 1 -- 172.21.15.156:0/2212507271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76015cd30 msgr2=0x7fd7600c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:47.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.965+0000 7fd767e15700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76015cd30 0x7fd7600c6470 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd758007e40 tx=0x7fd75800d3b0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.965+0000 7fd767e15700 1 -- 172.21.15.156:0/2212507271 shutdown_connections 2023-11-28T11:10:47.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.965+0000 7fd767e15700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76015cd30 0x7fd7600c6470 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.965+0000 7fd767e15700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd76006a270 0x7fd760147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.965+0000 7fd767e15700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd7440682f0 0x7fd74406a7a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.965+0000 7fd767e15700 1 --2- 172.21.15.156:0/2212507271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7600cfda0 0x7fd7600b5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.965+0000 7fd767e15700 1 -- 172.21.15.156:0/2212507271 >> 172.21.15.156:0/2212507271 conn(0x7fd76006b380 msgr2=0x7fd760158310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:47.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.965+0000 7fd767e15700 1 -- 172.21.15.156:0/2212507271 shutdown_connections 2023-11-28T11:10:47.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:47.965+0000 7fd767e15700 1 -- 172.21.15.156:0/2212507271 wait complete. 2023-11-28T11:10:47.968 INFO:teuthology.orchestra.run.smithi156.stderr: removed all blocklist entries 2023-11-28T11:10:47.979 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:10:48.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.103+0000 7fbf8aaf3700 1 Processor -- start 2023-11-28T11:10:48.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.103+0000 7fbf8aaf3700 1 -- start start 2023-11-28T11:10:48.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf8aaf3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbf8406ba90 0x7fbf840af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf8aaf3700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbf840cbdf0 0x7fbf840ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf8aaf3700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf840cc5f0 con 0x7fbf8415cef0 2023-11-28T11:10:48.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf8aaf3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf8415d470 con 0x7fbf8406ba90 2023-11-28T11:10:48.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf8aaf3700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf8415d750 con 0x7fbf840cbdf0 2023-11-28T11:10:48.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf83fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbf8406ba90 0x7fbf840af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf83fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbf8406ba90 0x7fbf840af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35478/0 (socket says 172.21.15.156:35478) 2023-11-28T11:10:48.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf83fff700 1 -- 172.21.15.156:0/1087065276 learned_addr learned my addr 172.21.15.156:0/1087065276 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:48.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf89090700 1 --2- 172.21.15.156:0/1087065276 >> v2:172.21.15.186:3300/0 conn(0x7fbf840cbdf0 0x7fbf840ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf83fff700 1 -- 172.21.15.156:0/1087065276 >> v2:172.21.15.186:3300/0 conn(0x7fbf840cbdf0 msgr2=0x7fbf840ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:48.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf83fff700 1 --2- 172.21.15.156:0/1087065276 >> v2:172.21.15.186:3300/0 conn(0x7fbf840cbdf0 0x7fbf840ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf83fff700 1 -- 172.21.15.156:0/1087065276 >> v1:172.21.15.186:6789/0 conn(0x7fbf8415cef0 legacy=0x7fbf8406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf83fff700 1 -- 172.21.15.156:0/1087065276 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8415da30 con 0x7fbf8406ba90 2023-11-28T11:10:48.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf83fff700 1 --2- 172.21.15.156:0/1087065276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8406ba90 0x7fbf840af400 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbf84073b10 tx=0x7fbf70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5a2f6af85833e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:48.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.104+0000 7fbf837fe700 1 -- 172.21.15.156:0/1087065276 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000b870 con 0x7fbf8406ba90 2023-11-28T11:10:48.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.105+0000 7fbf837fe700 1 -- 172.21.15.156:0/1087065276 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7000b9d0 con 0x7fbf8406ba90 2023-11-28T11:10:48.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.105+0000 7fbf837fe700 1 -- 172.21.15.156:0/1087065276 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000bd60 con 0x7fbf8406ba90 2023-11-28T11:10:48.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.105+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/1087065276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8406ba90 msgr2=0x7fbf840af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.105+0000 7fbf8aaf3700 1 --2- 172.21.15.156:0/1087065276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8406ba90 0x7fbf840af400 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbf84073b10 tx=0x7fbf70009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.105+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/1087065276 shutdown_connections 2023-11-28T11:10:48.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.105+0000 7fbf8aaf3700 1 --2- 172.21.15.156:0/1087065276 >> v2:172.21.15.186:3300/0 conn(0x7fbf840cbdf0 0x7fbf840ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.105+0000 7fbf8aaf3700 1 --2- 172.21.15.156:0/1087065276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8406ba90 0x7fbf840af400 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.105+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/1087065276 >> 172.21.15.156:0/1087065276 conn(0x7fbf840d0000 msgr2=0x7fbf840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:48.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.105+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/1087065276 shutdown_connections 2023-11-28T11:10:48.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.105+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/1087065276 wait complete. 2023-11-28T11:10:48.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.106+0000 7fbf8aaf3700 1 Processor -- start 2023-11-28T11:10:48.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.106+0000 7fbf8aaf3700 1 -- start start 2023-11-28T11:10:48.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.107+0000 7fbf8aaf3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf8406ba90 0x7fbf840c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.107+0000 7fbf8aaf3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf840cbdf0 0x7fbf840b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.107+0000 7fbf8aaf3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8415cef0 0x7fbf840b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.107+0000 7fbf8aaf3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf840c5e30 con 0x7fbf8415cef0 2023-11-28T11:10:48.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.107+0000 7fbf8aaf3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf84153640 con 0x7fbf8406ba90 2023-11-28T11:10:48.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.107+0000 7fbf8aaf3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf840c51b0 con 0x7fbf840cbdf0 2023-11-28T11:10:48.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.107+0000 7fbf89090700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8415cef0 0x7fbf840b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.107+0000 7fbf89090700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8415cef0 0x7fbf840b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35486/0 (socket says 172.21.15.156:35486) 2023-11-28T11:10:48.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.107+0000 7fbf89090700 1 -- 172.21.15.156:0/3356429969 learned_addr learned my addr 172.21.15.156:0/3356429969 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:48.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.107+0000 7fbf8888f700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf8406ba90 0x7fbf840c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.107+0000 7fbf83fff700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf840cbdf0 0x7fbf840b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.108+0000 7fbf89090700 1 -- 172.21.15.156:0/3356429969 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf840cbdf0 msgr2=0x7fbf840b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.108+0000 7fbf89090700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf840cbdf0 0x7fbf840b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.108+0000 7fbf89090700 1 -- 172.21.15.156:0/3356429969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf8406ba90 msgr2=0x7fbf840c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.108+0000 7fbf89090700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf8406ba90 0x7fbf840c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.108+0000 7fbf89090700 1 -- 172.21.15.156:0/3356429969 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf70009550 con 0x7fbf8415cef0 2023-11-28T11:10:48.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.108+0000 7fbf89090700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8415cef0 0x7fbf840b3370 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c007e40 tx=0x7fbf7c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:48.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.108+0000 7fbf817fa700 1 -- 172.21.15.156:0/3356429969 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c00ecd0 con 0x7fbf8415cef0 2023-11-28T11:10:48.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.108+0000 7fbf817fa700 1 -- 172.21.15.156:0/3356429969 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c00ee50 con 0x7fbf8415cef0 2023-11-28T11:10:48.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.108+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/3356429969 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8415dff0 con 0x7fbf8415cef0 2023-11-28T11:10:48.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.108+0000 7fbf817fa700 1 -- 172.21.15.156:0/3356429969 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c014660 con 0x7fbf8415cef0 2023-11-28T11:10:48.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.108+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/3356429969 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8415dd10 con 0x7fbf8415cef0 2023-11-28T11:10:48.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.109+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/3356429969 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf840cab90 con 0x7fbf8415cef0 2023-11-28T11:10:48.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.109+0000 7fbf817fa700 1 -- 172.21.15.156:0/3356429969 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbf7c012050 con 0x7fbf8415cef0 2023-11-28T11:10:48.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.110+0000 7fbf817fa700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbf680682f0 0x7fbf6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.110+0000 7fbf817fa700 1 -- 172.21.15.156:0/3356429969 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbf7c085ec0 con 0x7fbf8415cef0 2023-11-28T11:10:48.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.110+0000 7fbf8888f700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbf680682f0 0x7fbf6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.110+0000 7fbf8888f700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbf680682f0 0x7fbf6806a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbf84152d90 tx=0x7fbf78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:48.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.114+0000 7fbf817fa700 1 -- 172.21.15.156:0/3356429969 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbf7c052a70 con 0x7fbf8415cef0 2023-11-28T11:10:48.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.369+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/3356429969 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbf840cda20 con 0x7fbf8415cef0 2023-11-28T11:10:48.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.370+0000 7fbf817fa700 1 -- 172.21.15.156:0/3356429969 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fbf7c056090 con 0x7fbf8415cef0 2023-11-28T11:10:48.372 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:48.372 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:10:48.372 INFO:teuthology.orchestra.run.smithi156.stdout: "auth_dump": [ 2023-11-28T11:10:48.372 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.372 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.a", 2023-11-28T11:10:48.372 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlKLGIAhAAFJHYsC+LCoq5Qq1eh+MsgA==", 2023-11-28T11:10:48.372 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.372 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:10:48.372 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:10:48.373 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:10:48.373 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.373 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.373 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.373 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.373 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.b", 2023-11-28T11:10:48.373 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlr6ZnDxAAnJd4CtD5MUHauaV9GVqqVA==", 2023-11-28T11:10:48.373 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.373 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:10:48.373 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:10:48.373 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:10:48.373 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.c", 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlygniCRAAI6YYq5P02LiCGC4/tdGwqg==", 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.374 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.d", 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlc4gEFxAA6J82vSOm+mN6hvRs7Ec8SA==", 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.0", 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDsyWVlhbi+ORAAG5EKPXefX8NjPR7TNi6Wgg==", 2023-11-28T11:10:48.375 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.1", 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDvyWVlu+nEOBAAmvcXFjC+MNeWkDsZsI30PA==", 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.376 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.2", 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDyyWVlYMdIOBAABzlpv33IMXmgXqwROZ1n7A==", 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.3", 2023-11-28T11:10:48.377 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD1yWVltreINxAAcPoQjsD28WEyEYN7kZZEUg==", 2023-11-28T11:10:48.378 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.378 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:48.378 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:10:48.378 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.378 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.378 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.378 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.379 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.4", 2023-11-28T11:10:48.379 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD8yWVlliukARAA3YVAwd8OePh6WT7H0Jykog==", 2023-11-28T11:10:48.379 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.379 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:48.379 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:10:48.379 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.379 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.379 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.379 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.379 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.5", 2023-11-28T11:10:48.379 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD/yWVl6nilARAAfFTZi6p2kdxqOgtWonRRKA==", 2023-11-28T11:10:48.379 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.6", 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQACymVleb4IARAAXz8dDUDjem6eqAczZ5omDg==", 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.380 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.381 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.381 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.7", 2023-11-28T11:10:48.381 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAFymVlloyRABAAMa2gdE3weqqPWgiCI28jbQ==", 2023-11-28T11:10:48.381 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.381 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:48.381 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T11:10:48.381 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.381 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.381 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.381 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.381 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.0", 2023-11-28T11:10:48.381 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVl5LhsGxAA7WC9xFoSeBWdUPsmO5QaSg==", 2023-11-28T11:10:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:10:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow r", 2023-11-28T11:10:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T11:10:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.1", 2023-11-28T11:10:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlXV40IhAAhx1t1uwqXPA2g7GmoSXVmA==", 2023-11-28T11:10:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:10:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.2", 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlnJDEHhAAEZX8g8t6Gbm6InUWOLqetw==", 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T11:10:48.383 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.3", 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDryWVlPtmDJRAAoEvqSTpIIIWm42myVE3jVw==", 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.admin", 2023-11-28T11:10:48.384 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDqyWVl8mP6HRAAudHt/W3jSGkgKdYHCgM2uQ==", 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mgr.x", 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDqyWVly3m5OhAAWirKz8b4mHzNgLftULkWQg==", 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T11:10:48.385 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile mgr", 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mgr.y", 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQDqyWVlI+zRMxAANSXxnBj90PtANlg25RwW1A==", 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile mgr", 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:48.386 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2023-11-28T11:10:48.387 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:10:48.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.372+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/3356429969 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbf680682f0 msgr2=0x7fbf6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.372+0000 7fbf8aaf3700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbf680682f0 0x7fbf6806a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbf84152d90 tx=0x7fbf78009040 comp rx=0 tx=0).stop 2023-11-28T11:10:48.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.372+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/3356429969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8415cef0 msgr2=0x7fbf840b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.372+0000 7fbf8aaf3700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8415cef0 0x7fbf840b3370 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c007e40 tx=0x7fbf7c00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.372+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/3356429969 shutdown_connections 2023-11-28T11:10:48.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.372+0000 7fbf8aaf3700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8415cef0 0x7fbf840b3370 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.372+0000 7fbf8aaf3700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf8406ba90 0x7fbf840c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.372+0000 7fbf8aaf3700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fbf680682f0 0x7fbf6806a7a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.372+0000 7fbf8aaf3700 1 --2- 172.21.15.156:0/3356429969 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf840cbdf0 0x7fbf840b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.372+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/3356429969 >> 172.21.15.156:0/3356429969 conn(0x7fbf840d0000 msgr2=0x7fbf840cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:48.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.372+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/3356429969 shutdown_connections 2023-11-28T11:10:48.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.372+0000 7fbf8aaf3700 1 -- 172.21.15.156:0/3356429969 wait complete. 2023-11-28T11:10:48.389 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:48.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.508+0000 7f67c383a700 1 Processor -- start 2023-11-28T11:10:48.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.508+0000 7f67c383a700 1 -- start start 2023-11-28T11:10:48.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.509+0000 7f67c383a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67bc071040 0x7f67bc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.509+0000 7f67c383a700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f67bc06dee0 0x7f67bc06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.509+0000 7f67c383a700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc071950 con 0x7f67bc06cdc0 2023-11-28T11:10:48.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.509+0000 7f67c383a700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67bc161550 con 0x7f67bc06dee0 2023-11-28T11:10:48.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.509+0000 7f67c383a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc15d060 con 0x7f67bc071040 2023-11-28T11:10:48.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.510+0000 7f67c15d6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67bc071040 0x7f67bc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.510+0000 7f67c0dd5700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f67bc06dee0 0x7f67bc06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.510+0000 7f67c0dd5700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f67bc06dee0 0x7f67bc06e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:46504/0 (socket says 172.21.15.156:46504) 2023-11-28T11:10:48.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.510+0000 7f67c0dd5700 1 -- 172.21.15.156:0/2015686853 learned_addr learned my addr 172.21.15.156:0/2015686853 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:48.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.510+0000 7f67c1dd7700 1 --1- 172.21.15.156:0/2015686853 >> v1:172.21.15.186:6789/0 conn(0x7f67bc06cdc0 0x7f67bc06d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:60732/0 (socket says 172.21.15.156:60732) 2023-11-28T11:10:48.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.510+0000 7f67b3fff700 1 -- 172.21.15.156:0/2015686853 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018889037 0 0) 0x7f67bc071950 con 0x7f67bc06cdc0 2023-11-28T11:10:48.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.510+0000 7f67b3fff700 1 -- 172.21.15.156:0/2015686853 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67a80035b0 con 0x7f67bc06cdc0 2023-11-28T11:10:48.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.510+0000 7f67c0dd5700 1 -- 172.21.15.156:0/2015686853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67bc071040 msgr2=0x7f67bc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:48.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.510+0000 7f67c0dd5700 1 --2- 172.21.15.156:0/2015686853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67bc071040 0x7f67bc071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.510+0000 7f67c0dd5700 1 -- 172.21.15.156:0/2015686853 >> v1:172.21.15.186:6789/0 conn(0x7f67bc06cdc0 legacy=0x7f67bc06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.510+0000 7f67c0dd5700 1 -- 172.21.15.156:0/2015686853 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc15d340 con 0x7f67bc06dee0 2023-11-28T11:10:48.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.510+0000 7f67c0dd5700 1 --2- 172.21.15.156:0/2015686853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67bc06dee0 0x7f67bc06e350 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f67b80098f0 tx=0x7f67b8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=650a50e97c669d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:48.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.511+0000 7f67b3fff700 1 -- 172.21.15.156:0/2015686853 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8011830 con 0x7f67bc06dee0 2023-11-28T11:10:48.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.511+0000 7f67b3fff700 1 -- 172.21.15.156:0/2015686853 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b8011990 con 0x7f67bc06dee0 2023-11-28T11:10:48.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.511+0000 7f67b3fff700 1 -- 172.21.15.156:0/2015686853 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8011d20 con 0x7f67bc06dee0 2023-11-28T11:10:48.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.511+0000 7f67c383a700 1 -- 172.21.15.156:0/2015686853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67bc06dee0 msgr2=0x7f67bc06e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.511+0000 7f67c383a700 1 --2- 172.21.15.156:0/2015686853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67bc06dee0 0x7f67bc06e350 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f67b80098f0 tx=0x7f67b8009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:48.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.511+0000 7f67c383a700 1 -- 172.21.15.156:0/2015686853 shutdown_connections 2023-11-28T11:10:48.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.511+0000 7f67c383a700 1 --2- 172.21.15.156:0/2015686853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67bc071040 0x7f67bc071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.511+0000 7f67c383a700 1 --2- 172.21.15.156:0/2015686853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67bc06dee0 0x7f67bc06e350 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.511+0000 7f67c383a700 1 -- 172.21.15.156:0/2015686853 >> 172.21.15.156:0/2015686853 conn(0x7f67bc0d0000 msgr2=0x7f67bc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:48.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.511+0000 7f67c383a700 1 -- 172.21.15.156:0/2015686853 shutdown_connections 2023-11-28T11:10:48.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.511+0000 7f67c383a700 1 -- 172.21.15.156:0/2015686853 wait complete. 2023-11-28T11:10:48.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.512+0000 7f67c383a700 1 Processor -- start 2023-11-28T11:10:48.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.513+0000 7f67c383a700 1 -- start start 2023-11-28T11:10:48.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.513+0000 7f67c383a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67bc06cdc0 0x7f67bc0b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.513+0000 7f67c383a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67bc06dee0 0x7f67bc0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.513+0000 7f67c383a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67bc071040 0x7f67bc155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.513+0000 7f67c383a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc155ff0 con 0x7f67bc071040 2023-11-28T11:10:48.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.513+0000 7f67c383a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0c4b70 con 0x7f67bc06cdc0 2023-11-28T11:10:48.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.513+0000 7f67c383a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0c6b20 con 0x7f67bc06dee0 2023-11-28T11:10:48.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.513+0000 7f67c1dd7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67bc071040 0x7f67bc155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.513+0000 7f67c1dd7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67bc071040 0x7f67bc155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35500/0 (socket says 172.21.15.156:35500) 2023-11-28T11:10:48.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.513+0000 7f67c1dd7700 1 -- 172.21.15.156:0/1586006039 learned_addr learned my addr 172.21.15.156:0/1586006039 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:48.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67c15d6700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67bc06cdc0 0x7f67bc0b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67c0dd5700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67bc06dee0 0x7f67bc0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67c1dd7700 1 -- 172.21.15.156:0/1586006039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67bc06dee0 msgr2=0x7f67bc0b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67c1dd7700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67bc06dee0 0x7f67bc0b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67c1dd7700 1 -- 172.21.15.156:0/1586006039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67bc06cdc0 msgr2=0x7f67bc0b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67c1dd7700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67bc06cdc0 0x7f67bc0b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67c1dd7700 1 -- 172.21.15.156:0/1586006039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b8009550 con 0x7f67bc071040 2023-11-28T11:10:48.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67c0dd5700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67bc06dee0 0x7f67bc0b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:48.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67c1dd7700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67bc071040 0x7f67bc155660 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f67b400cbf0 tx=0x7f67b400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:48.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67b1ffb700 1 -- 172.21.15.156:0/1586006039 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b400ab80 con 0x7f67bc071040 2023-11-28T11:10:48.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67b1ffb700 1 -- 172.21.15.156:0/1586006039 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b400ace0 con 0x7f67bc071040 2023-11-28T11:10:48.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67c383a700 1 -- 172.21.15.156:0/1586006039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67bc15d900 con 0x7f67bc071040 2023-11-28T11:10:48.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67b1ffb700 1 -- 172.21.15.156:0/1586006039 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4007290 con 0x7f67bc071040 2023-11-28T11:10:48.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.514+0000 7f67c383a700 1 -- 172.21.15.156:0/1586006039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc15d620 con 0x7f67bc071040 2023-11-28T11:10:48.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.515+0000 7f67c383a700 1 -- 172.21.15.156:0/1586006039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67bc06e890 con 0x7f67bc071040 2023-11-28T11:10:48.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.516+0000 7f67b1ffb700 1 -- 172.21.15.156:0/1586006039 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f67b4019030 con 0x7f67bc071040 2023-11-28T11:10:48.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.516+0000 7f67b1ffb700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67a00682f0 0x7f67a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.516+0000 7f67b1ffb700 1 -- 172.21.15.156:0/1586006039 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f67b40856c0 con 0x7f67bc071040 2023-11-28T11:10:48.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.516+0000 7f67c15d6700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67a00682f0 0x7f67a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.516+0000 7f67c15d6700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67a00682f0 0x7f67a006a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009ad0 tx=0x7f67ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:48.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.520+0000 7f67b1ffb700 1 -- 172.21.15.156:0/1586006039 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f67b4052270 con 0x7f67bc071040 2023-11-28T11:10:48.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.703+0000 7f67c383a700 1 -- 172.21.15.156:0/1586006039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67bc06b040 con 0x7f67bc071040 2023-11-28T11:10:48.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.704+0000 7f67b1ffb700 1 -- 172.21.15.156:0/1586006039 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+14315 (secure 0 0 0) 0x7f67b4017020 con 0x7f67bc071040 2023-11-28T11:10:48.707 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:48.707 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":59,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:10:47.959926+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:48.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.706+0000 7f67c383a700 1 -- 172.21.15.156:0/1586006039 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67a00682f0 msgr2=0x7f67a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.706+0000 7f67c383a700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67a00682f0 0x7f67a006a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009ad0 tx=0x7f67ac008040 comp rx=0 tx=0).stop 2023-11-28T11:10:48.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.706+0000 7f67c383a700 1 -- 172.21.15.156:0/1586006039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67bc071040 msgr2=0x7f67bc155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.706+0000 7f67c383a700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67bc071040 0x7f67bc155660 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f67b400cbf0 tx=0x7f67b400a060 comp rx=0 tx=0).stop 2023-11-28T11:10:48.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.706+0000 7f67c383a700 1 -- 172.21.15.156:0/1586006039 shutdown_connections 2023-11-28T11:10:48.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.706+0000 7f67c383a700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67bc071040 0x7f67bc155660 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.706+0000 7f67c383a700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67bc06cdc0 0x7f67bc0b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.706+0000 7f67c383a700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f67a00682f0 0x7f67a006a7a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.706+0000 7f67c383a700 1 --2- 172.21.15.156:0/1586006039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67bc06dee0 0x7f67bc0b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.707+0000 7f67c383a700 1 -- 172.21.15.156:0/1586006039 >> 172.21.15.156:0/1586006039 conn(0x7f67bc0d0000 msgr2=0x7f67bc0cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:48.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.707+0000 7f67c383a700 1 -- 172.21.15.156:0/1586006039 shutdown_connections 2023-11-28T11:10:48.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.707+0000 7f67c383a700 1 -- 172.21.15.156:0/1586006039 wait complete. 2023-11-28T11:10:48.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:10:48.721 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:10:48.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.843+0000 7f9f24853700 1 Processor -- start 2023-11-28T11:10:48.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.843+0000 7f9f24853700 1 -- start start 2023-11-28T11:10:48.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.844+0000 7f9f24853700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9f200ce040 0x7f9f2015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.844+0000 7f9f24853700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2006b420 con 0x7f9f200d0a00 2023-11-28T11:10:48.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.844+0000 7f9f24853700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2006b700 con 0x7f9f200b2930 2023-11-28T11:10:48.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.844+0000 7f9f24853700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2006ff60 con 0x7f9f200ce040 2023-11-28T11:10:48.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.844+0000 7f9f1e59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9f200ce040 0x7f9f2015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.844+0000 7f9f1dd9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f9f200d0a00 0x7f9f2006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50382/0 (socket says 172.21.15.156:50382) 2023-11-28T11:10:48.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.844+0000 7f9f1e59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9f200ce040 0x7f9f2015df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35504/0 (socket says 172.21.15.156:35504) 2023-11-28T11:10:48.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.844+0000 7f9f1dd9b700 1 -- 172.21.15.156:0/3477213193 learned_addr learned my addr 172.21.15.156:0/3477213193 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:48.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.845+0000 7f9f1e59c700 1 -- 172.21.15.156:0/3477213193 >> v1:172.21.15.186:6789/0 conn(0x7f9f200b2930 legacy=0x7f9f2006c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.845+0000 7f9f1e59c700 1 -- 172.21.15.156:0/3477213193 >> v1:172.21.15.156:6789/0 conn(0x7f9f200d0a00 legacy=0x7f9f2006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.845+0000 7f9f1e59c700 1 -- 172.21.15.156:0/3477213193 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2006b9e0 con 0x7f9f200ce040 2023-11-28T11:10:48.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.845+0000 7f9f1d59a700 1 -- 172.21.15.156:0/3477213193 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911180819 0 0) 0x7f9f2006b420 con 0x7f9f200d0a00 2023-11-28T11:10:48.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.845+0000 7f9f1e59c700 1 --2- 172.21.15.156:0/3477213193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f200ce040 0x7f9f2015df10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009820 tx=0x7f9f14009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e50633d2606b08 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:48.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.845+0000 7f9f1d59a700 1 -- 172.21.15.156:0/3477213193 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14005e30 con 0x7f9f200ce040 2023-11-28T11:10:48.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.845+0000 7f9f1d59a700 1 -- 172.21.15.156:0/3477213193 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f14006030 con 0x7f9f200ce040 2023-11-28T11:10:48.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.845+0000 7f9f1d59a700 1 -- 172.21.15.156:0/3477213193 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f140063e0 con 0x7f9f200ce040 2023-11-28T11:10:48.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.846+0000 7f9f24853700 1 -- 172.21.15.156:0/3477213193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f200ce040 msgr2=0x7f9f2015df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.846+0000 7f9f24853700 1 --2- 172.21.15.156:0/3477213193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f200ce040 0x7f9f2015df10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009820 tx=0x7f9f14009b30 comp rx=0 tx=0).stop 2023-11-28T11:10:48.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.846+0000 7f9f24853700 1 -- 172.21.15.156:0/3477213193 shutdown_connections 2023-11-28T11:10:48.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.846+0000 7f9f24853700 1 --2- 172.21.15.156:0/3477213193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f200ce040 0x7f9f2015df10 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.846+0000 7f9f24853700 1 -- 172.21.15.156:0/3477213193 >> 172.21.15.156:0/3477213193 conn(0x7f9f2006aed0 msgr2=0x7f9f2006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:48.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.846+0000 7f9f24853700 1 -- 172.21.15.156:0/3477213193 shutdown_connections 2023-11-28T11:10:48.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.846+0000 7f9f24853700 1 -- 172.21.15.156:0/3477213193 wait complete. 2023-11-28T11:10:48.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.847+0000 7f9f24853700 1 Processor -- start 2023-11-28T11:10:48.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.847+0000 7f9f24853700 1 -- start start 2023-11-28T11:10:48.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f24853700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f200b2930 0x7f9f200b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f24853700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f200ce040 0x7f9f200c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f24853700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f200d0a00 0x7f9f20151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f24853700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200c8470 con 0x7f9f200ce040 2023-11-28T11:10:48.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f24853700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20152ca0 con 0x7f9f200d0a00 2023-11-28T11:10:48.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f24853700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200b6770 con 0x7f9f200b2930 2023-11-28T11:10:48.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f1dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f200ce040 0x7f9f200c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f1dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f200ce040 0x7f9f200c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35512/0 (socket says 172.21.15.156:35512) 2023-11-28T11:10:48.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f1dd9b700 1 -- 172.21.15.156:0/3229715179 learned_addr learned my addr 172.21.15.156:0/3229715179 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:48.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f1e59c700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f200b2930 0x7f9f200b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f1ed9d700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f200d0a00 0x7f9f20151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f1dd9b700 1 -- 172.21.15.156:0/3229715179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f200b2930 msgr2=0x7f9f200b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f1dd9b700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f200b2930 0x7f9f200b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f1dd9b700 1 -- 172.21.15.156:0/3229715179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f200d0a00 msgr2=0x7f9f20151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f1dd9b700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f200d0a00 0x7f9f20151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f1dd9b700 1 -- 172.21.15.156:0/3229715179 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f14009430 con 0x7f9f200ce040 2023-11-28T11:10:48.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f1e59c700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f200b2930 0x7f9f200b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:48.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.848+0000 7f9f1dd9b700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f200ce040 0x7f9f200c7080 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000c550 tx=0x7f9f1000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:48.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.849+0000 7f9f0effd700 1 -- 172.21.15.156:0/3229715179 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10004c80 con 0x7f9f200ce040 2023-11-28T11:10:48.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.849+0000 7f9f0effd700 1 -- 172.21.15.156:0/3229715179 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f10004de0 con 0x7f9f200ce040 2023-11-28T11:10:48.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.849+0000 7f9f24853700 1 -- 172.21.15.156:0/3229715179 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2006bfa0 con 0x7f9f200ce040 2023-11-28T11:10:48.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.849+0000 7f9f0effd700 1 -- 172.21.15.156:0/3229715179 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10007310 con 0x7f9f200ce040 2023-11-28T11:10:48.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.849+0000 7f9f24853700 1 -- 172.21.15.156:0/3229715179 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2006bcc0 con 0x7f9f200ce040 2023-11-28T11:10:48.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.850+0000 7f9f24853700 1 -- 172.21.15.156:0/3229715179 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f200b2db0 con 0x7f9f200ce040 2023-11-28T11:10:48.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.850+0000 7f9f0effd700 1 -- 172.21.15.156:0/3229715179 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9f1001a020 con 0x7f9f200ce040 2023-11-28T11:10:48.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.850+0000 7f9f0effd700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9f00068320 0x7f9f0006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.851+0000 7f9f0effd700 1 -- 172.21.15.156:0/3229715179 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9f10085480 con 0x7f9f200ce040 2023-11-28T11:10:48.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.851+0000 7f9f1e59c700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9f00068320 0x7f9f0006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.851+0000 7f9f1e59c700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9f00068320 0x7f9f0006a7d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f14013040 tx=0x7f9f14008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:48.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:48.855+0000 7f9f0effd700 1 -- 172.21.15.156:0/3229715179 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9f10052030 con 0x7f9f200ce040 2023-11-28T11:10:49.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.035+0000 7f9f24853700 1 -- 172.21.15.156:0/3229715179 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f20068690 con 0x7f9f200ce040 2023-11-28T11:10:49.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.036+0000 7f9f0effd700 1 -- 172.21.15.156:0/3229715179 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f10055650 con 0x7f9f200ce040 2023-11-28T11:10:49.037 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:10:49.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.038+0000 7f9f24853700 1 -- 172.21.15.156:0/3229715179 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9f00068320 msgr2=0x7f9f0006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.038+0000 7f9f24853700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9f00068320 0x7f9f0006a7d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f14013040 tx=0x7f9f14008070 comp rx=0 tx=0).stop 2023-11-28T11:10:49.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.038+0000 7f9f24853700 1 -- 172.21.15.156:0/3229715179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f200ce040 msgr2=0x7f9f200c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.038+0000 7f9f24853700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f200ce040 0x7f9f200c7080 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000c550 tx=0x7f9f1000c860 comp rx=0 tx=0).stop 2023-11-28T11:10:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.038+0000 7f9f24853700 1 -- 172.21.15.156:0/3229715179 shutdown_connections 2023-11-28T11:10:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.038+0000 7f9f24853700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f200d0a00 0x7f9f20151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.038+0000 7f9f24853700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9f00068320 0x7f9f0006a7d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.038+0000 7f9f24853700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f200b2930 0x7f9f200b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.038+0000 7f9f24853700 1 --2- 172.21.15.156:0/3229715179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f200ce040 0x7f9f200c7080 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.038+0000 7f9f24853700 1 -- 172.21.15.156:0/3229715179 >> 172.21.15.156:0/3229715179 conn(0x7f9f2006aed0 msgr2=0x7f9f2015d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.038+0000 7f9f24853700 1 -- 172.21.15.156:0/3229715179 shutdown_connections 2023-11-28T11:10:49.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.038+0000 7f9f24853700 1 -- 172.21.15.156:0/3229715179 wait complete. 2023-11-28T11:10:49.051 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:10:49.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.173+0000 7f2fefd3f700 1 Processor -- start 2023-11-28T11:10:49.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.174+0000 7f2fefd3f700 1 -- start start 2023-11-28T11:10:49.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.174+0000 7f2fefd3f700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2fe806ba90 0x7f2fe80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.174+0000 7f2fefd3f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2fe80cbdf0 0x7f2fe80ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.174+0000 7f2fefd3f700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe80cc5f0 con 0x7f2fe815cef0 2023-11-28T11:10:49.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.174+0000 7f2fefd3f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe815d470 con 0x7f2fe80cbdf0 2023-11-28T11:10:49.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.174+0000 7f2fefd3f700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe815d750 con 0x7f2fe806ba90 2023-11-28T11:10:49.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.175+0000 7f2fee2dc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2fe80cbdf0 0x7f2fe80ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.175+0000 7f2fee2dc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2fe80cbdf0 0x7f2fe80ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35520/0 (socket says 172.21.15.156:35520) 2023-11-28T11:10:49.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.175+0000 7f2fee2dc700 1 -- 172.21.15.156:0/125832927 learned_addr learned my addr 172.21.15.156:0/125832927 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:49.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.175+0000 7f2fed2da700 1 --2- 172.21.15.156:0/125832927 >> v2:172.21.15.186:3300/0 conn(0x7f2fe806ba90 0x7f2fe80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.175+0000 7f2fee2dc700 1 -- 172.21.15.156:0/125832927 >> v2:172.21.15.186:3300/0 conn(0x7f2fe806ba90 msgr2=0x7f2fe80af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.175+0000 7f2fee2dc700 1 --2- 172.21.15.156:0/125832927 >> v2:172.21.15.186:3300/0 conn(0x7f2fe806ba90 0x7f2fe80af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.175+0000 7f2fee2dc700 1 -- 172.21.15.156:0/125832927 >> v1:172.21.15.186:6789/0 conn(0x7f2fe815cef0 legacy=0x7f2fe806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.175+0000 7f2fee2dc700 1 -- 172.21.15.156:0/125832927 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe815da30 con 0x7f2fe80cbdf0 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.175+0000 7f2fee2dc700 1 --2- 172.21.15.156:0/125832927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe80cbdf0 0x7f2fe80ca650 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2fe806fbb0 tx=0x7f2fe000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ebac225b35bf8f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.175+0000 7f2fecad9700 1 -- 172.21.15.156:0/125832927 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe000e070 con 0x7f2fe80cbdf0 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.175+0000 7f2fecad9700 1 -- 172.21.15.156:0/125832927 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe0003a20 con 0x7f2fe80cbdf0 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.175+0000 7f2fecad9700 1 -- 172.21.15.156:0/125832927 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0003dd0 con 0x7f2fe80cbdf0 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.176+0000 7f2fefd3f700 1 -- 172.21.15.156:0/125832927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe80cbdf0 msgr2=0x7f2fe80ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.176+0000 7f2fefd3f700 1 --2- 172.21.15.156:0/125832927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe80cbdf0 0x7f2fe80ca650 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2fe806fbb0 tx=0x7f2fe000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.176+0000 7f2fefd3f700 1 -- 172.21.15.156:0/125832927 shutdown_connections 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.176+0000 7f2fefd3f700 1 --2- 172.21.15.156:0/125832927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe80cbdf0 0x7f2fe80ca650 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.176+0000 7f2fefd3f700 1 --2- 172.21.15.156:0/125832927 >> v2:172.21.15.186:3300/0 conn(0x7f2fe806ba90 0x7f2fe80af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.176+0000 7f2fefd3f700 1 -- 172.21.15.156:0/125832927 >> 172.21.15.156:0/125832927 conn(0x7f2fe80d0000 msgr2=0x7f2fe80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:49.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.176+0000 7f2fefd3f700 1 -- 172.21.15.156:0/125832927 shutdown_connections 2023-11-28T11:10:49.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.176+0000 7f2fefd3f700 1 -- 172.21.15.156:0/125832927 wait complete. 2023-11-28T11:10:49.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.177+0000 7f2fefd3f700 1 Processor -- start 2023-11-28T11:10:49.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.177+0000 7f2fefd3f700 1 -- start start 2023-11-28T11:10:49.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fefd3f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fe806ba90 0x7f2fe814cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fefd3f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe80cbdf0 0x7f2fe80b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fefd3f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe815cef0 0x7f2fe80b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fefd3f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80c7d30 con 0x7f2fe815cef0 2023-11-28T11:10:49.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fefd3f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80c70d0 con 0x7f2fe806ba90 2023-11-28T11:10:49.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fefd3f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80c7390 con 0x7f2fe80cbdf0 2023-11-28T11:10:49.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fee2dc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe815cef0 0x7f2fe80b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fee2dc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe815cef0 0x7f2fe80b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35526/0 (socket says 172.21.15.156:35526) 2023-11-28T11:10:49.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fee2dc700 1 -- 172.21.15.156:0/3897569355 learned_addr learned my addr 172.21.15.156:0/3897569355 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:49.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fedadb700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fe806ba90 0x7f2fe814cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fee2dc700 1 -- 172.21.15.156:0/3897569355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe80cbdf0 msgr2=0x7f2fe80b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fee2dc700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe80cbdf0 0x7f2fe80b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fee2dc700 1 -- 172.21.15.156:0/3897569355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fe806ba90 msgr2=0x7f2fe814cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fee2dc700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fe806ba90 0x7f2fe814cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fee2dc700 1 -- 172.21.15.156:0/3897569355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe0009b30 con 0x7f2fe815cef0 2023-11-28T11:10:49.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fee2dc700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe815cef0 0x7f2fe80b1000 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0015040 tx=0x7f2fe0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:49.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fde7fc700 1 -- 172.21.15.156:0/3897569355 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0004440 con 0x7f2fe815cef0 2023-11-28T11:10:49.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fde7fc700 1 -- 172.21.15.156:0/3897569355 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe00045a0 con 0x7f2fe815cef0 2023-11-28T11:10:49.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fefd3f700 1 -- 172.21.15.156:0/3897569355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe815dff0 con 0x7f2fe815cef0 2023-11-28T11:10:49.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fde7fc700 1 -- 172.21.15.156:0/3897569355 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0004950 con 0x7f2fe815cef0 2023-11-28T11:10:49.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.178+0000 7f2fefd3f700 1 -- 172.21.15.156:0/3897569355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe815dd10 con 0x7f2fe815cef0 2023-11-28T11:10:49.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.179+0000 7f2fefd3f700 1 -- 172.21.15.156:0/3897569355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe80cab90 con 0x7f2fe815cef0 2023-11-28T11:10:49.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.180+0000 7f2fde7fc700 1 -- 172.21.15.156:0/3897569355 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2fe0022030 con 0x7f2fe815cef0 2023-11-28T11:10:49.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.180+0000 7f2fde7fc700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2fcc0682f0 0x7f2fcc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.180+0000 7f2fde7fc700 1 -- 172.21.15.156:0/3897569355 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2fe0086d80 con 0x7f2fe815cef0 2023-11-28T11:10:49.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.180+0000 7f2fedadb700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2fcc0682f0 0x7f2fcc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.180+0000 7f2fedadb700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2fcc0682f0 0x7f2fcc06a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800a7b0 tx=0x7f2fd8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:49.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.184+0000 7f2fde7fc700 1 -- 172.21.15.156:0/3897569355 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2fe00538b0 con 0x7f2fe815cef0 2023-11-28T11:10:49.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.366+0000 7f2fefd3f700 1 -- 172.21.15.156:0/3897569355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2fe80c7e70 con 0x7f2fe815cef0 2023-11-28T11:10:49.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.367+0000 7f2fde7fc700 1 -- 172.21.15.156:0/3897569355 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2fe0017020 con 0x7f2fe815cef0 2023-11-28T11:10:49.368 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:10:49.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.369+0000 7f2fefd3f700 1 -- 172.21.15.156:0/3897569355 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2fcc0682f0 msgr2=0x7f2fcc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.369+0000 7f2fefd3f700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2fcc0682f0 0x7f2fcc06a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800a7b0 tx=0x7f2fd8008040 comp rx=0 tx=0).stop 2023-11-28T11:10:49.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.369+0000 7f2fefd3f700 1 -- 172.21.15.156:0/3897569355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe815cef0 msgr2=0x7f2fe80b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.369+0000 7f2fefd3f700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe815cef0 0x7f2fe80b1000 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0015040 tx=0x7f2fe0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.369+0000 7f2fefd3f700 1 -- 172.21.15.156:0/3897569355 shutdown_connections 2023-11-28T11:10:49.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.369+0000 7f2fefd3f700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe815cef0 0x7f2fe80b1000 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.369+0000 7f2fefd3f700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fe806ba90 0x7f2fe814cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.369+0000 7f2fefd3f700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2fcc0682f0 0x7f2fcc06a7a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.369+0000 7f2fefd3f700 1 --2- 172.21.15.156:0/3897569355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe80cbdf0 0x7f2fe80b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.369+0000 7f2fefd3f700 1 -- 172.21.15.156:0/3897569355 >> 172.21.15.156:0/3897569355 conn(0x7f2fe80d0000 msgr2=0x7f2fe815c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:49.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.369+0000 7f2fefd3f700 1 -- 172.21.15.156:0/3897569355 shutdown_connections 2023-11-28T11:10:49.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.369+0000 7f2fefd3f700 1 -- 172.21.15.156:0/3897569355 wait complete. 2023-11-28T11:10:49.382 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:10:49.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.502+0000 7fb66e096700 1 Processor -- start 2023-11-28T11:10:49.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.502+0000 7fb66e096700 1 -- start start 2023-11-28T11:10:49.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.502+0000 7fb66e096700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb66815cef0 0x7fb6680af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.502+0000 7fb66e096700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6680cbdf0 0x7fb66806b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.502+0000 7fb66e096700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb66806b8c0 0x7fb668071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.502+0000 7fb66e096700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66806bd40 con 0x7fb66815cef0 2023-11-28T11:10:49.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.502+0000 7fb66e096700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6680cc5f0 con 0x7fb66806b8c0 2023-11-28T11:10:49.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.502+0000 7fb66e096700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66815d470 con 0x7fb6680cbdf0 2023-11-28T11:10:49.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb6677fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb66815cef0 0x7fb6680af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb6677fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb66815cef0 0x7fb6680af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35530/0 (socket says 172.21.15.156:35530) 2023-11-28T11:10:49.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb6677fe700 1 -- 172.21.15.156:0/1723215482 learned_addr learned my addr 172.21.15.156:0/1723215482 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:49.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb666ffd700 1 --2- 172.21.15.156:0/1723215482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6680cbdf0 0x7fb66806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb667fff700 1 --2- 172.21.15.156:0/1723215482 >> v2:172.21.15.186:3300/0 conn(0x7fb66806b8c0 0x7fb668071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb6677fe700 1 -- 172.21.15.156:0/1723215482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6680cbdf0 msgr2=0x7fb66806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:49.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb6677fe700 1 --2- 172.21.15.156:0/1723215482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6680cbdf0 0x7fb66806b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb6677fe700 1 -- 172.21.15.156:0/1723215482 >> v2:172.21.15.186:3300/0 conn(0x7fb66806b8c0 msgr2=0x7fb668071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:49.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb6677fe700 1 --2- 172.21.15.156:0/1723215482 >> v2:172.21.15.186:3300/0 conn(0x7fb66806b8c0 0x7fb668071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb6677fe700 1 -- 172.21.15.156:0/1723215482 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66815d750 con 0x7fb66815cef0 2023-11-28T11:10:49.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb6677fe700 1 --2- 172.21.15.156:0/1723215482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb66815cef0 0x7fb6680af400 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb668073a70 tx=0x7fb65800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d74855783f5668 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:49.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb6667fc700 1 -- 172.21.15.156:0/1723215482 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65800dd30 con 0x7fb66815cef0 2023-11-28T11:10:49.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb6667fc700 1 -- 172.21.15.156:0/1723215482 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65800de90 con 0x7fb66815cef0 2023-11-28T11:10:49.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.503+0000 7fb6667fc700 1 -- 172.21.15.156:0/1723215482 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658010690 con 0x7fb66815cef0 2023-11-28T11:10:49.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.504+0000 7fb66e096700 1 -- 172.21.15.156:0/1723215482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb66815cef0 msgr2=0x7fb6680af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.504+0000 7fb66e096700 1 --2- 172.21.15.156:0/1723215482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb66815cef0 0x7fb6680af400 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb668073a70 tx=0x7fb65800be70 comp rx=0 tx=0).stop 2023-11-28T11:10:49.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.504+0000 7fb66e096700 1 -- 172.21.15.156:0/1723215482 shutdown_connections 2023-11-28T11:10:49.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.504+0000 7fb66e096700 1 --2- 172.21.15.156:0/1723215482 >> v2:172.21.15.186:3300/0 conn(0x7fb66806b8c0 0x7fb668071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.504+0000 7fb66e096700 1 --2- 172.21.15.156:0/1723215482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb66815cef0 0x7fb6680af400 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.504+0000 7fb66e096700 1 --2- 172.21.15.156:0/1723215482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6680cbdf0 0x7fb66806b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.504+0000 7fb66e096700 1 -- 172.21.15.156:0/1723215482 >> 172.21.15.156:0/1723215482 conn(0x7fb6680d0000 msgr2=0x7fb6680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:49.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.504+0000 7fb66e096700 1 -- 172.21.15.156:0/1723215482 shutdown_connections 2023-11-28T11:10:49.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.504+0000 7fb66e096700 1 -- 172.21.15.156:0/1723215482 wait complete. 2023-11-28T11:10:49.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.505+0000 7fb66e096700 1 Processor -- start 2023-11-28T11:10:49.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.506+0000 7fb66e096700 1 -- start start 2023-11-28T11:10:49.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.506+0000 7fb66e096700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb66806b8c0 0x7fb668153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.506+0000 7fb66e096700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6680cbdf0 0x7fb668157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.506+0000 7fb66e096700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb66815cef0 0x7fb668157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.506+0000 7fb66e096700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6680b2e30 con 0x7fb66815cef0 2023-11-28T11:10:49.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.506+0000 7fb66e096700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6680b3080 con 0x7fb6680cbdf0 2023-11-28T11:10:49.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.506+0000 7fb66e096700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6680b3360 con 0x7fb66806b8c0 2023-11-28T11:10:49.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb667fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb66815cef0 0x7fb668157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb667fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb66815cef0 0x7fb668157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35538/0 (socket says 172.21.15.156:35538) 2023-11-28T11:10:49.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb667fff700 1 -- 172.21.15.156:0/2049660502 learned_addr learned my addr 172.21.15.156:0/2049660502 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:49.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb6677fe700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb66806b8c0 0x7fb668153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb666ffd700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6680cbdf0 0x7fb668157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb667fff700 1 -- 172.21.15.156:0/2049660502 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb66806b8c0 msgr2=0x7fb668153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb667fff700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb66806b8c0 0x7fb668153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb667fff700 1 -- 172.21.15.156:0/2049660502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6680cbdf0 msgr2=0x7fb668157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb667fff700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6680cbdf0 0x7fb668157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb667fff700 1 -- 172.21.15.156:0/2049660502 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65800b7f0 con 0x7fb66815cef0 2023-11-28T11:10:49.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb667fff700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb66815cef0 0x7fb668157570 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb66000ccf0 tx=0x7fb66000a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:49.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb64ffff700 1 -- 172.21.15.156:0/2049660502 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66000ad60 con 0x7fb66815cef0 2023-11-28T11:10:49.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb64ffff700 1 -- 172.21.15.156:0/2049660502 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb660003a20 con 0x7fb66815cef0 2023-11-28T11:10:49.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb66e096700 1 -- 172.21.15.156:0/2049660502 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66815dd10 con 0x7fb66815cef0 2023-11-28T11:10:49.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb64ffff700 1 -- 172.21.15.156:0/2049660502 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660003db0 con 0x7fb66815cef0 2023-11-28T11:10:49.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.507+0000 7fb66e096700 1 -- 172.21.15.156:0/2049660502 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66815da30 con 0x7fb66815cef0 2023-11-28T11:10:49.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.508+0000 7fb66e096700 1 -- 172.21.15.156:0/2049660502 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb66815dff0 con 0x7fb66815cef0 2023-11-28T11:10:49.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.509+0000 7fb64ffff700 1 -- 172.21.15.156:0/2049660502 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb66001c020 con 0x7fb66815cef0 2023-11-28T11:10:49.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.509+0000 7fb64ffff700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb6540682f0 0x7fb65406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.509+0000 7fb64ffff700 1 -- 172.21.15.156:0/2049660502 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb660012070 con 0x7fb66815cef0 2023-11-28T11:10:49.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.509+0000 7fb6677fe700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb6540682f0 0x7fb65406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.510+0000 7fb6677fe700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb6540682f0 0x7fb65406a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb65800beb0 tx=0x7fb658012000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:49.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.513+0000 7fb64ffff700 1 -- 172.21.15.156:0/2049660502 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb660052b30 con 0x7fb66815cef0 2023-11-28T11:10:49.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.699+0000 7fb66e096700 1 -- 172.21.15.156:0/2049660502 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb6680d0e60 con 0x7fb66815cef0 2023-11-28T11:10:49.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.699+0000 7fb64ffff700 1 -- 172.21.15.156:0/2049660502 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) v1 ==== 84+0+126 (secure 0 0 0) 0x7fb660056150 con 0x7fb66815cef0 2023-11-28T11:10:49.701 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:49.701 INFO:teuthology.orchestra.run.smithi156.stdout:[ 2023-11-28T11:10:49.701 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:49.701 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 1, 2023-11-28T11:10:49.701 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": ".mgr" 2023-11-28T11:10:49.701 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:10:49.701 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:10:49.701 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 2, 2023-11-28T11:10:49.701 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": "rbd" 2023-11-28T11:10:49.701 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:10:49.702 INFO:teuthology.orchestra.run.smithi156.stdout:] 2023-11-28T11:10:49.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.701+0000 7fb66e096700 1 -- 172.21.15.156:0/2049660502 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb6540682f0 msgr2=0x7fb65406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.701+0000 7fb66e096700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb6540682f0 0x7fb65406a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb65800beb0 tx=0x7fb658012000 comp rx=0 tx=0).stop 2023-11-28T11:10:49.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.701+0000 7fb66e096700 1 -- 172.21.15.156:0/2049660502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb66815cef0 msgr2=0x7fb668157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.701+0000 7fb66e096700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb66815cef0 0x7fb668157570 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb66000ccf0 tx=0x7fb66000a290 comp rx=0 tx=0).stop 2023-11-28T11:10:49.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.701+0000 7fb66e096700 1 -- 172.21.15.156:0/2049660502 shutdown_connections 2023-11-28T11:10:49.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.701+0000 7fb66e096700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb66815cef0 0x7fb668157570 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.701+0000 7fb66e096700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb6540682f0 0x7fb65406a7a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.701+0000 7fb66e096700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb66806b8c0 0x7fb668153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.701+0000 7fb66e096700 1 --2- 172.21.15.156:0/2049660502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6680cbdf0 0x7fb668157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.701+0000 7fb66e096700 1 -- 172.21.15.156:0/2049660502 >> 172.21.15.156:0/2049660502 conn(0x7fb6680d0000 msgr2=0x7fb668159d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:49.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.701+0000 7fb66e096700 1 -- 172.21.15.156:0/2049660502 shutdown_connections 2023-11-28T11:10:49.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.702+0000 7fb66e096700 1 -- 172.21.15.156:0/2049660502 wait complete. 2023-11-28T11:10:49.715 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:10:49.715 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:10:49.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.840+0000 7f6310e30700 1 Processor -- start 2023-11-28T11:10:49.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.840+0000 7f6310e30700 1 -- start start 2023-11-28T11:10:49.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f6310e30700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0af060 0x7f630c070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f6310e30700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f630c0af430 0x7f630c06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f6310e30700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630c0713c0 con 0x7f630c06a270 2023-11-28T11:10:49.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f6310e30700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630c06dee0 con 0x7f630c0af430 2023-11-28T11:10:49.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f6310e30700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c0716a0 con 0x7f630c0af060 2023-11-28T11:10:49.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f6309d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f630c0af430 0x7f630c06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f630ad9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f630c06a270 0x7f630c160590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50422/0 (socket says 172.21.15.156:50422) 2023-11-28T11:10:49.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f6309d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f630c0af430 0x7f630c06d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35544/0 (socket says 172.21.15.156:35544) 2023-11-28T11:10:49.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f630ad9d700 1 -- 172.21.15.156:0/2383296654 learned_addr learned my addr 172.21.15.156:0/2383296654 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:49.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f630a59c700 1 --2- 172.21.15.156:0/2383296654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0af060 0x7f630c070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f6309d9b700 1 -- 172.21.15.156:0/2383296654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0af060 msgr2=0x7f630c070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:49.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f6309d9b700 1 --2- 172.21.15.156:0/2383296654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0af060 0x7f630c070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.841+0000 7f6309d9b700 1 -- 172.21.15.156:0/2383296654 >> v1:172.21.15.156:6789/0 conn(0x7f630c06a270 legacy=0x7f630c160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f630959a700 1 -- 172.21.15.156:0/2383296654 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766856978 0 0) 0x7f630c0713c0 con 0x7f630c06a270 2023-11-28T11:10:49.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f6309d9b700 1 -- 172.21.15.156:0/2383296654 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c071980 con 0x7f630c0af430 2023-11-28T11:10:49.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f6309d9b700 1 --2- 172.21.15.156:0/2383296654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f630c0af430 0x7f630c06d9a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009840 tx=0x7f62fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cc0128a991bbc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:49.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f630959a700 1 -- 172.21.15.156:0/2383296654 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc011830 con 0x7f630c0af430 2023-11-28T11:10:49.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f630959a700 1 -- 172.21.15.156:0/2383296654 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc011990 con 0x7f630c0af430 2023-11-28T11:10:49.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f630959a700 1 -- 172.21.15.156:0/2383296654 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc011d20 con 0x7f630c0af430 2023-11-28T11:10:49.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f6310e30700 1 -- 172.21.15.156:0/2383296654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f630c0af430 msgr2=0x7f630c06d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f6310e30700 1 --2- 172.21.15.156:0/2383296654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f630c0af430 0x7f630c06d9a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009840 tx=0x7f62fc009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:49.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f6310e30700 1 -- 172.21.15.156:0/2383296654 shutdown_connections 2023-11-28T11:10:49.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f6310e30700 1 --2- 172.21.15.156:0/2383296654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0af060 0x7f630c070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f6310e30700 1 --2- 172.21.15.156:0/2383296654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f630c0af430 0x7f630c06d9a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f6310e30700 1 -- 172.21.15.156:0/2383296654 >> 172.21.15.156:0/2383296654 conn(0x7f630c06b380 msgr2=0x7f630c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:49.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f6310e30700 1 -- 172.21.15.156:0/2383296654 shutdown_connections 2023-11-28T11:10:49.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.842+0000 7f6310e30700 1 -- 172.21.15.156:0/2383296654 wait complete. 2023-11-28T11:10:49.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.844+0000 7f6310e30700 1 Processor -- start 2023-11-28T11:10:49.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.844+0000 7f6310e30700 1 -- start start 2023-11-28T11:10:49.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.844+0000 7f6310e30700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c06a270 0x7f630c15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f6310e30700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0af060 0x7f630c156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f6310e30700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f630c0af430 0x7f630c1f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f6310e30700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c15ffc0 con 0x7f630c0af430 2023-11-28T11:10:49.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f6310e30700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c1569a0 con 0x7f630c06a270 2023-11-28T11:10:49.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f6310e30700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c1f33b0 con 0x7f630c0af060 2023-11-28T11:10:49.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f630ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f630c0af430 0x7f630c1f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f630ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f630c0af430 0x7f630c1f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35550/0 (socket says 172.21.15.156:35550) 2023-11-28T11:10:49.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f630ad9d700 1 -- 172.21.15.156:0/1809850632 learned_addr learned my addr 172.21.15.156:0/1809850632 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:49.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f630a59c700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c06a270 0x7f630c15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f6309d9b700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0af060 0x7f630c156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f630ad9d700 1 -- 172.21.15.156:0/1809850632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0af060 msgr2=0x7f630c156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f630ad9d700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0af060 0x7f630c156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f630ad9d700 1 -- 172.21.15.156:0/1809850632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c06a270 msgr2=0x7f630c15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f630ad9d700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c06a270 0x7f630c15fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f630ad9d700 1 -- 172.21.15.156:0/1809850632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc0094a0 con 0x7f630c0af430 2023-11-28T11:10:49.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f630ad9d700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f630c0af430 0x7f630c1f2370 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f630400cba0 tx=0x7f630400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:49.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f62faffd700 1 -- 172.21.15.156:0/1809850632 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630400ab80 con 0x7f630c0af430 2023-11-28T11:10:49.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f62faffd700 1 -- 172.21.15.156:0/1809850632 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630400ad00 con 0x7f630c0af430 2023-11-28T11:10:49.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f6310e30700 1 -- 172.21.15.156:0/1809850632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c071f40 con 0x7f630c0af430 2023-11-28T11:10:49.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f62faffd700 1 -- 172.21.15.156:0/1809850632 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6304007310 con 0x7f630c0af430 2023-11-28T11:10:49.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.845+0000 7f6310e30700 1 -- 172.21.15.156:0/1809850632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630c071c60 con 0x7f630c0af430 2023-11-28T11:10:49.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.846+0000 7f6310e30700 1 -- 172.21.15.156:0/1809850632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630c0cb670 con 0x7f630c0af430 2023-11-28T11:10:49.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.847+0000 7f62faffd700 1 -- 172.21.15.156:0/1809850632 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f630401a030 con 0x7f630c0af430 2023-11-28T11:10:49.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.847+0000 7f62faffd700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f62ec0682f0 0x7f62ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.847+0000 7f62faffd700 1 -- 172.21.15.156:0/1809850632 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6304085670 con 0x7f630c0af430 2023-11-28T11:10:49.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.847+0000 7f630a59c700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f62ec0682f0 0x7f62ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.848+0000 7f630a59c700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f62ec0682f0 0x7f62ec06a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6300009950 tx=0x7f6300008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:49.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:49.851+0000 7f62faffd700 1 -- 172.21.15.156:0/1809850632 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6304052220 con 0x7f630c0af430 2023-11-28T11:10:50.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.036+0000 7f6310e30700 1 -- 172.21.15.156:0/1809850632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f630c1f28b0 con 0x7f630c0af430 2023-11-28T11:10:50.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.972+0000 7f62faffd700 1 -- 172.21.15.156:0/1809850632 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) v1 ==== 138+0+0 (secure 0 0 0) 0x7f63040180b0 con 0x7f630c0af430 2023-11-28T11:10:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.976+0000 7f6310e30700 1 -- 172.21.15.156:0/1809850632 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f62ec0682f0 msgr2=0x7f62ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.976+0000 7f6310e30700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f62ec0682f0 0x7f62ec06a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6300009950 tx=0x7f6300008040 comp rx=0 tx=0).stop 2023-11-28T11:10:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.976+0000 7f6310e30700 1 -- 172.21.15.156:0/1809850632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f630c0af430 msgr2=0x7f630c1f2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.976+0000 7f6310e30700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f630c0af430 0x7f630c1f2370 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f630400cba0 tx=0x7f630400cf60 comp rx=0 tx=0).stop 2023-11-28T11:10:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.976+0000 7f6310e30700 1 -- 172.21.15.156:0/1809850632 shutdown_connections 2023-11-28T11:10:50.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.976+0000 7f6310e30700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f630c0af430 0x7f630c1f2370 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.976+0000 7f6310e30700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c06a270 0x7f630c15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.976+0000 7f6310e30700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f62ec0682f0 0x7f62ec06a7a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.976+0000 7f6310e30700 1 --2- 172.21.15.156:0/1809850632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0af060 0x7f630c156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.976+0000 7f6310e30700 1 -- 172.21.15.156:0/1809850632 >> 172.21.15.156:0/1809850632 conn(0x7f630c06b380 msgr2=0x7f630c06cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:50.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.977+0000 7f6310e30700 1 -- 172.21.15.156:0/1809850632 shutdown_connections 2023-11-28T11:10:50.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:50.977+0000 7f6310e30700 1 -- 172.21.15.156:0/1809850632 wait complete. 2023-11-28T11:10:50.983 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' created 2023-11-28T11:10:50.997 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:10:51.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.130+0000 7fd6790b7700 1 Processor -- start 2023-11-28T11:10:51.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.130+0000 7fd6790b7700 1 -- start start 2023-11-28T11:10:51.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd6790b7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6740cd8c0 0x7fd674140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd6790b7700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd6740cdc90 0x7fd6740b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd6790b7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd6740b2e70 0x7fd67406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd6790b7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67413ff00 con 0x7fd6740b2e70 2023-11-28T11:10:51.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd6790b7700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6741404c0 con 0x7fd6740cdc90 2023-11-28T11:10:51.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd6790b7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67415aa80 con 0x7fd6740cd8c0 2023-11-28T11:10:51.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd67359e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd6740b2e70 0x7fd67406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd67359e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd6740b2e70 0x7fd67406f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35556/0 (socket says 172.21.15.156:35556) 2023-11-28T11:10:51.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd67359e700 1 -- 172.21.15.156:0/278654299 learned_addr learned my addr 172.21.15.156:0/278654299 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:51.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd672d9d700 1 --2- 172.21.15.156:0/278654299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6740cd8c0 0x7fd674140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd67259c700 1 --2- 172.21.15.156:0/278654299 >> v2:172.21.15.186:3300/0 conn(0x7fd6740cdc90 0x7fd6740b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd67359e700 1 -- 172.21.15.156:0/278654299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6740cd8c0 msgr2=0x7fd674140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:51.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd67359e700 1 --2- 172.21.15.156:0/278654299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6740cd8c0 0x7fd674140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd67359e700 1 -- 172.21.15.156:0/278654299 >> v2:172.21.15.186:3300/0 conn(0x7fd6740cdc90 msgr2=0x7fd6740b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:51.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd67359e700 1 --2- 172.21.15.156:0/278654299 >> v2:172.21.15.186:3300/0 conn(0x7fd6740cdc90 0x7fd6740b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd67359e700 1 -- 172.21.15.156:0/278654299 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6740b3310 con 0x7fd6740b2e70 2023-11-28T11:10:51.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.131+0000 7fd672d9d700 1 --2- 172.21.15.156:0/278654299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6740cd8c0 0x7fd674140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:51.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd67359e700 1 --2- 172.21.15.156:0/278654299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6740b2e70 0x7fd67406f850 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd674067c60 tx=0x7fd66c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fad34f3ea58c5d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:51.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd671d9b700 1 -- 172.21.15.156:0/278654299 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c00e070 con 0x7fd6740b2e70 2023-11-28T11:10:51.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd671d9b700 1 -- 172.21.15.156:0/278654299 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c00b9c0 con 0x7fd6740b2e70 2023-11-28T11:10:51.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd671d9b700 1 -- 172.21.15.156:0/278654299 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c00bd70 con 0x7fd6740b2e70 2023-11-28T11:10:51.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd6790b7700 1 -- 172.21.15.156:0/278654299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6740b2e70 msgr2=0x7fd67406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd6790b7700 1 --2- 172.21.15.156:0/278654299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6740b2e70 0x7fd67406f850 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd674067c60 tx=0x7fd66c00b330 comp rx=0 tx=0).stop 2023-11-28T11:10:51.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd6790b7700 1 -- 172.21.15.156:0/278654299 shutdown_connections 2023-11-28T11:10:51.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd6790b7700 1 --2- 172.21.15.156:0/278654299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6740b2e70 0x7fd67406f850 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd6790b7700 1 --2- 172.21.15.156:0/278654299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6740cd8c0 0x7fd674140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd6790b7700 1 --2- 172.21.15.156:0/278654299 >> v2:172.21.15.186:3300/0 conn(0x7fd6740cdc90 0x7fd6740b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd6790b7700 1 -- 172.21.15.156:0/278654299 >> 172.21.15.156:0/278654299 conn(0x7fd67406aed0 msgr2=0x7fd67406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:51.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd6790b7700 1 -- 172.21.15.156:0/278654299 shutdown_connections 2023-11-28T11:10:51.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.132+0000 7fd6790b7700 1 -- 172.21.15.156:0/278654299 wait complete. 2023-11-28T11:10:51.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.134+0000 7fd6790b7700 1 Processor -- start 2023-11-28T11:10:51.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.134+0000 7fd6790b7700 1 -- start start 2023-11-28T11:10:51.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd6790b7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6740b2e70 0x7fd6740b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd6790b7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6740cd8c0 0x7fd67415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd6790b7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6740cdc90 0x7fd67415cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd6790b7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740c4380 con 0x7fd6740cdc90 2023-11-28T11:10:51.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd6790b7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6741f05b0 con 0x7fd6740cd8c0 2023-11-28T11:10:51.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd6790b7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740c4a80 con 0x7fd6740b2e70 2023-11-28T11:10:51.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd67359e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6740cdc90 0x7fd67415cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd67359e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6740cdc90 0x7fd67415cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35564/0 (socket says 172.21.15.156:35564) 2023-11-28T11:10:51.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd67359e700 1 -- 172.21.15.156:0/1810960489 learned_addr learned my addr 172.21.15.156:0/1810960489 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:51.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd67259c700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6740cd8c0 0x7fd67415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd672d9d700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6740b2e70 0x7fd6740b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd67359e700 1 -- 172.21.15.156:0/1810960489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6740b2e70 msgr2=0x7fd6740b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd67359e700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6740b2e70 0x7fd6740b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd67359e700 1 -- 172.21.15.156:0/1810960489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6740cd8c0 msgr2=0x7fd67415bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd67359e700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6740cd8c0 0x7fd67415bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd67359e700 1 -- 172.21.15.156:0/1810960489 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66c009b30 con 0x7fd6740cdc90 2023-11-28T11:10:51.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd672d9d700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6740b2e70 0x7fd6740b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:51.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd67359e700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6740cdc90 0x7fd67415cf20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00b370 tx=0x7fd66c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:51.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd65b7fe700 1 -- 172.21.15.156:0/1810960489 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c004310 con 0x7fd6740cdc90 2023-11-28T11:10:51.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd65b7fe700 1 -- 172.21.15.156:0/1810960489 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c004470 con 0x7fd6740cdc90 2023-11-28T11:10:51.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd6790b7700 1 -- 172.21.15.156:0/1810960489 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd674140150 con 0x7fd6740cdc90 2023-11-28T11:10:51.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd65b7fe700 1 -- 172.21.15.156:0/1810960489 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c004820 con 0x7fd6740cdc90 2023-11-28T11:10:51.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.135+0000 7fd6790b7700 1 -- 172.21.15.156:0/1810960489 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6740b35f0 con 0x7fd6740cdc90 2023-11-28T11:10:51.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.136+0000 7fd6790b7700 1 -- 172.21.15.156:0/1810960489 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6740ce100 con 0x7fd6740cdc90 2023-11-28T11:10:51.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.137+0000 7fd65b7fe700 1 -- 172.21.15.156:0/1810960489 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd66c01b430 con 0x7fd6740cdc90 2023-11-28T11:10:51.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.137+0000 7fd65b7fe700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd660068320 0x7fd66006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.137+0000 7fd65b7fe700 1 -- 172.21.15.156:0/1810960489 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fd66c086e90 con 0x7fd6740cdc90 2023-11-28T11:10:51.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.137+0000 7fd672d9d700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd660068320 0x7fd66006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.138+0000 7fd672d9d700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd660068320 0x7fd66006a7d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd668006fd0 tx=0x7fd668008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:51.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.142+0000 7fd65b7fe700 1 -- 172.21.15.156:0/1810960489 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd66c053870 con 0x7fd6740cdc90 2023-11-28T11:10:51.218 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:10:51.220 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:10:51.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.331+0000 7fd6790b7700 1 -- 172.21.15.156:0/1810960489 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd6740c50a0 con 0x7fd6740cdc90 2023-11-28T11:10:51.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.978+0000 7fd65b7fe700 1 -- 172.21.15.156:0/1810960489 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd66c01f0d0 con 0x7fd6740cdc90 2023-11-28T11:10:51.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.986+0000 7fd6790b7700 1 -- 172.21.15.156:0/1810960489 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd660068320 msgr2=0x7fd66006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.986+0000 7fd6790b7700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd660068320 0x7fd66006a7d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd668006fd0 tx=0x7fd668008040 comp rx=0 tx=0).stop 2023-11-28T11:10:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.986+0000 7fd6790b7700 1 -- 172.21.15.156:0/1810960489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6740cdc90 msgr2=0x7fd67415cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.986+0000 7fd6790b7700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6740cdc90 0x7fd67415cf20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00b370 tx=0x7fd66c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.986+0000 7fd6790b7700 1 -- 172.21.15.156:0/1810960489 shutdown_connections 2023-11-28T11:10:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.986+0000 7fd6790b7700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6740cdc90 0x7fd67415cf20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.986+0000 7fd6790b7700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd660068320 0x7fd66006a7d0 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd668006fd0 tx=0x7fd668008040 comp rx=0 tx=0).stop 2023-11-28T11:10:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.986+0000 7fd6790b7700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6740b2e70 0x7fd6740b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.986+0000 7fd6790b7700 1 --2- 172.21.15.156:0/1810960489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6740cd8c0 0x7fd67415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.986+0000 7fd6790b7700 1 -- 172.21.15.156:0/1810960489 >> 172.21.15.156:0/1810960489 conn(0x7fd67406aed0 msgr2=0x7fd6740d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.986+0000 7fd6790b7700 1 -- 172.21.15.156:0/1810960489 shutdown_connections 2023-11-28T11:10:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:51.987+0000 7fd6790b7700 1 -- 172.21.15.156:0/1810960489 wait complete. 2023-11-28T11:10:51.992 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' created 2023-11-28T11:10:52.006 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:10:52.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.130+0000 7f154d29b700 1 Processor -- start 2023-11-28T11:10:52.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.130+0000 7f154d29b700 1 -- start start 2023-11-28T11:10:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f154d29b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f154815c9b0 0x7f15480b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f154d29b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1548161340 0x7f15480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f154d29b700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15480d1950 con 0x7f15480d2320 2023-11-28T11:10:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f154d29b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15480b5800 con 0x7f1548161340 2023-11-28T11:10:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f154d29b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15480b5520 con 0x7f154815c9b0 2023-11-28T11:10:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f15467fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1548161340 0x7f15480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f15467fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1548161340 0x7f15480d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35570/0 (socket says 172.21.15.156:35570) 2023-11-28T11:10:52.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f15467fc700 1 -- 172.21.15.156:0/1858345185 learned_addr learned my addr 172.21.15.156:0/1858345185 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:52.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f1546ffd700 1 --2- 172.21.15.156:0/1858345185 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f154815c9b0 0x7f15480b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f15467fc700 1 -- 172.21.15.156:0/1858345185 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f154815c9b0 msgr2=0x7f15480b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:52.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f15467fc700 1 --2- 172.21.15.156:0/1858345185 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f154815c9b0 0x7f15480b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f15467fc700 1 -- 172.21.15.156:0/1858345185 >> v1:172.21.15.186:6789/0 conn(0x7f15480d2320 legacy=0x7f15480cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f15467fc700 1 -- 172.21.15.156:0/1858345185 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15480d1300 con 0x7f1548161340 2023-11-28T11:10:52.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.131+0000 7f1546ffd700 1 --2- 172.21.15.156:0/1858345185 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f154815c9b0 0x7f15480b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:52.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.132+0000 7f15467fc700 1 --2- 172.21.15.156:0/1858345185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1548161340 0x7f15480d1de0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f15380098f0 tx=0x7f1538009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa5552c806128f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:52.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.132+0000 7f1545ffb700 1 -- 172.21.15.156:0/1858345185 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153800b8a0 con 0x7f1548161340 2023-11-28T11:10:52.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.132+0000 7f1545ffb700 1 -- 172.21.15.156:0/1858345185 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153800ba00 con 0x7f1548161340 2023-11-28T11:10:52.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.132+0000 7f1545ffb700 1 -- 172.21.15.156:0/1858345185 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153800bd90 con 0x7f1548161340 2023-11-28T11:10:52.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.132+0000 7f154d29b700 1 -- 172.21.15.156:0/1858345185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1548161340 msgr2=0x7f15480d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.132+0000 7f154d29b700 1 --2- 172.21.15.156:0/1858345185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1548161340 0x7f15480d1de0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f15380098f0 tx=0x7f1538009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:52.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.132+0000 7f154d29b700 1 -- 172.21.15.156:0/1858345185 shutdown_connections 2023-11-28T11:10:52.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.132+0000 7f154d29b700 1 --2- 172.21.15.156:0/1858345185 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f154815c9b0 0x7f15480b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.132+0000 7f154d29b700 1 --2- 172.21.15.156:0/1858345185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1548161340 0x7f15480d1de0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.132+0000 7f154d29b700 1 -- 172.21.15.156:0/1858345185 >> 172.21.15.156:0/1858345185 conn(0x7f1548072600 msgr2=0x7f15480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:52.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.132+0000 7f154d29b700 1 -- 172.21.15.156:0/1858345185 shutdown_connections 2023-11-28T11:10:52.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.132+0000 7f154d29b700 1 -- 172.21.15.156:0/1858345185 wait complete. 2023-11-28T11:10:52.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.134+0000 7f154d29b700 1 Processor -- start 2023-11-28T11:10:52.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.134+0000 7f154d29b700 1 -- start start 2023-11-28T11:10:52.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.134+0000 7f154d29b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15480d2320 0x7f1548151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.134+0000 7f154d29b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f154815c9b0 0x7f154814fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.134+0000 7f154d29b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1548161340 0x7f1548159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.134+0000 7f154d29b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15481527a0 con 0x7f15480d2320 2023-11-28T11:10:52.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.134+0000 7f154d29b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1548151e90 con 0x7f154815c9b0 2023-11-28T11:10:52.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.134+0000 7f154d29b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15480c8680 con 0x7f1548161340 2023-11-28T11:10:52.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f1546ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15480d2320 0x7f1548151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f1546ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15480d2320 0x7f1548151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35574/0 (socket says 172.21.15.156:35574) 2023-11-28T11:10:52.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f1546ffd700 1 -- 172.21.15.156:0/4139132573 learned_addr learned my addr 172.21.15.156:0/4139132573 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:52.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f15467fc700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f154815c9b0 0x7f154814fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f1546ffd700 1 -- 172.21.15.156:0/4139132573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1548161340 msgr2=0x7f1548159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f1546ffd700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1548161340 0x7f1548159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f1546ffd700 1 -- 172.21.15.156:0/4139132573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f154815c9b0 msgr2=0x7f154814fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f1546ffd700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f154815c9b0 0x7f154814fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f1546ffd700 1 -- 172.21.15.156:0/4139132573 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1538009550 con 0x7f15480d2320 2023-11-28T11:10:52.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f1546ffd700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15480d2320 0x7f1548151500 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f153c00c750 tx=0x7f153c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:52.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f152f7fe700 1 -- 172.21.15.156:0/4139132573 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c011800 con 0x7f15480d2320 2023-11-28T11:10:52.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f152f7fe700 1 -- 172.21.15.156:0/4139132573 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c011960 con 0x7f15480d2320 2023-11-28T11:10:52.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f152f7fe700 1 -- 172.21.15.156:0/4139132573 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c011cd0 con 0x7f15480d2320 2023-11-28T11:10:52.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f154d29b700 1 -- 172.21.15.156:0/4139132573 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15480b5130 con 0x7f15480d2320 2023-11-28T11:10:52.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.135+0000 7f154d29b700 1 -- 172.21.15.156:0/4139132573 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15480d15e0 con 0x7f15480d2320 2023-11-28T11:10:52.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.136+0000 7f154d29b700 1 -- 172.21.15.156:0/4139132573 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15480b52c0 con 0x7f15480d2320 2023-11-28T11:10:52.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.137+0000 7f152f7fe700 1 -- 172.21.15.156:0/4139132573 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f153c020070 con 0x7f15480d2320 2023-11-28T11:10:52.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.137+0000 7f152f7fe700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f15340682f0 0x7f153406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.137+0000 7f152f7fe700 1 -- 172.21.15.156:0/4139132573 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f153c085ab0 con 0x7f15480d2320 2023-11-28T11:10:52.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.137+0000 7f15467fc700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f15340682f0 0x7f153406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.138+0000 7f15467fc700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f15340682f0 0x7f153406a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1538009c40 tx=0x7f1538008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:52.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.141+0000 7f152f7fe700 1 -- 172.21.15.156:0/4139132573 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f153c052370 con 0x7f15480d2320 2023-11-28T11:10:52.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.365+0000 7f154d29b700 1 -- 172.21.15.156:0/4139132573 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f15481528e0 con 0x7f15480d2320 2023-11-28T11:10:52.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.371+0000 7f152f7fe700 1 -- 172.21.15.156:0/4139132573 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v27) v1 ==== 173+0+0 (secure 0 0 0) 0x7f153c055990 con 0x7f15480d2320 2023-11-28T11:10:52.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.375+0000 7f154d29b700 1 -- 172.21.15.156:0/4139132573 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f15340682f0 msgr2=0x7f153406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.375+0000 7f154d29b700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f15340682f0 0x7f153406a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1538009c40 tx=0x7f1538008030 comp rx=0 tx=0).stop 2023-11-28T11:10:52.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.375+0000 7f154d29b700 1 -- 172.21.15.156:0/4139132573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15480d2320 msgr2=0x7f1548151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.375+0000 7f154d29b700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15480d2320 0x7f1548151500 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f153c00c750 tx=0x7f153c00cb10 comp rx=0 tx=0).stop 2023-11-28T11:10:52.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.375+0000 7f154d29b700 1 -- 172.21.15.156:0/4139132573 shutdown_connections 2023-11-28T11:10:52.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.375+0000 7f154d29b700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f15340682f0 0x7f153406a7a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.375+0000 7f154d29b700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1548161340 0x7f1548159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.375+0000 7f154d29b700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15480d2320 0x7f1548151500 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.375+0000 7f154d29b700 1 --2- 172.21.15.156:0/4139132573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f154815c9b0 0x7f154814fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.375+0000 7f154d29b700 1 -- 172.21.15.156:0/4139132573 >> 172.21.15.156:0/4139132573 conn(0x7f1548072600 msgr2=0x7f15480b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:52.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.377+0000 7f154d29b700 1 -- 172.21.15.156:0/4139132573 shutdown_connections 2023-11-28T11:10:52.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.377+0000 7f154d29b700 1 -- 172.21.15.156:0/4139132573 wait complete. 2023-11-28T11:10:52.386 INFO:teuthology.orchestra.run.smithi156.stderr:new fs with metadata pool 9 and data pool 10 2023-11-28T11:10:52.400 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:52.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.520+0000 7fa86da08700 1 Processor -- start 2023-11-28T11:10:52.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.520+0000 7fa86da08700 1 -- start start 2023-11-28T11:10:52.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.521+0000 7fa86da08700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa8681658c0 0x7fa868167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.521+0000 7fa86da08700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86815d060 con 0x7fa868071040 2023-11-28T11:10:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.521+0000 7fa86da08700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86815d340 con 0x7fa86806dee0 2023-11-28T11:10:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.521+0000 7fa86da08700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa868161550 con 0x7fa8681658c0 2023-11-28T11:10:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.521+0000 7fa866ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa868071040 0x7fa868071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50460/0 (socket says 172.21.15.156:50460) 2023-11-28T11:10:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.521+0000 7fa866ffd700 1 -- 172.21.15.156:0/138676301 learned_addr learned my addr 172.21.15.156:0/138676301 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.521+0000 7fa8677fe700 1 --2- 172.21.15.156:0/138676301 >> v2:172.21.15.186:3300/0 conn(0x7fa8681658c0 0x7fa868167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.521+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421385011 0 0) 0x7fa86815d060 con 0x7fa868071040 2023-11-28T11:10:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.521+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8500035b0 con 0x7fa868071040 2023-11-28T11:10:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375627288 0 0) 0x7fa86815d340 con 0x7fa86806dee0 2023-11-28T11:10:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa86815d060 con 0x7fa86806dee0 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4143667287 0 0) 0x7fa8500035b0 con 0x7fa868071040 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 >> v2:172.21.15.186:3300/0 conn(0x7fa8681658c0 msgr2=0x7fa868167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 --2- 172.21.15.156:0/138676301 >> v2:172.21.15.186:3300/0 conn(0x7fa8681658c0 0x7fa868167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 >> v1:172.21.15.186:6789/0 conn(0x7fa86806dee0 legacy=0x7fa86806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86815d620 con 0x7fa868071040 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fa85c001670 con 0x7fa868071040 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 >> v1:172.21.15.156:6789/0 conn(0x7fa868071040 legacy=0x7fa868071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 --2- 172.21.15.156:0/138676301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa850003b90 0x7fa850003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 --2- 172.21.15.156:0/138676301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa850004850 0x7fa850004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 --2- 172.21.15.156:0/138676301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa850005270 0x7fa85000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa868161550 con 0x7fa850004850 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85000c1b0 con 0x7fa850003b90 2023-11-28T11:10:52.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85000c370 con 0x7fa850005270 2023-11-28T11:10:52.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa8667fc700 1 --2- 172.21.15.156:0/138676301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa850004850 0x7fa850004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa866ffd700 1 --2- 172.21.15.156:0/138676301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa850003b90 0x7fa850003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa8677fe700 1 --2- 172.21.15.156:0/138676301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa850005270 0x7fa85000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa8667fc700 1 -- 172.21.15.156:0/138676301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa850005270 msgr2=0x7fa85000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:52.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa8667fc700 1 --2- 172.21.15.156:0/138676301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa850005270 0x7fa85000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa8667fc700 1 -- 172.21.15.156:0/138676301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa850003b90 msgr2=0x7fa850003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:52.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa8667fc700 1 --2- 172.21.15.156:0/138676301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa850003b90 0x7fa850003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa8667fc700 1 -- 172.21.15.156:0/138676301 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8500035b0 con 0x7fa850004850 2023-11-28T11:10:52.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa8677fe700 1 -- 172.21.15.156:0/138676301 reap_dead start 2023-11-28T11:10:52.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa8667fc700 1 --2- 172.21.15.156:0/138676301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa850004850 0x7fa850004cc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa858001980 tx=0x7fa850005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0df045d1fcad4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:52.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858001470 con 0x7fa850004850 2023-11-28T11:10:52.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8580052f0 con 0x7fa850004850 2023-11-28T11:10:52.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.522+0000 7fa865ffb700 1 -- 172.21.15.156:0/138676301 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858003680 con 0x7fa850004850 2023-11-28T11:10:52.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.523+0000 7fa86da08700 1 -- 172.21.15.156:0/138676301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa850004850 msgr2=0x7fa850004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.523+0000 7fa86da08700 1 --2- 172.21.15.156:0/138676301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa850004850 0x7fa850004cc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa858001980 tx=0x7fa850005880 comp rx=0 tx=0).stop 2023-11-28T11:10:52.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.523+0000 7fa86da08700 1 -- 172.21.15.156:0/138676301 shutdown_connections 2023-11-28T11:10:52.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.523+0000 7fa86da08700 1 --2- 172.21.15.156:0/138676301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa850004850 0x7fa850004cc0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.523+0000 7fa86da08700 1 -- 172.21.15.156:0/138676301 >> 172.21.15.156:0/138676301 conn(0x7fa8680d0000 msgr2=0x7fa8680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:52.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.523+0000 7fa86da08700 1 -- 172.21.15.156:0/138676301 shutdown_connections 2023-11-28T11:10:52.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.523+0000 7fa86da08700 1 -- 172.21.15.156:0/138676301 wait complete. 2023-11-28T11:10:52.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.524+0000 7fa86da08700 1 Processor -- start 2023-11-28T11:10:52.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.524+0000 7fa86da08700 1 -- start start 2023-11-28T11:10:52.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa86da08700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa850004850 0x7fa8680ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa86da08700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8680c5820 0x7fa8680ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa86da08700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680c4250 0x7fa8680b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa86da08700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8680b3c20 con 0x7fa8680c4250 2023-11-28T11:10:52.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa86da08700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8680ada00 con 0x7fa8680c5820 2023-11-28T11:10:52.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa86da08700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8680acdf0 con 0x7fa850004850 2023-11-28T11:10:52.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa8677fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680c4250 0x7fa8680b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa8677fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680c4250 0x7fa8680b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35598/0 (socket says 172.21.15.156:35598) 2023-11-28T11:10:52.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa8677fe700 1 -- 172.21.15.156:0/1163650932 learned_addr learned my addr 172.21.15.156:0/1163650932 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:52.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa866ffd700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa850004850 0x7fa8680ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa8667fc700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8680c5820 0x7fa8680ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa8677fe700 1 -- 172.21.15.156:0/1163650932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa850004850 msgr2=0x7fa8680ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa8677fe700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa850004850 0x7fa8680ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa8677fe700 1 -- 172.21.15.156:0/1163650932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8680c5820 msgr2=0x7fa8680ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa8677fe700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8680c5820 0x7fa8680ae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.525+0000 7fa8677fe700 1 -- 172.21.15.156:0/1163650932 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86815ccd0 con 0x7fa8680c4250 2023-11-28T11:10:52.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.526+0000 7fa8677fe700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680c4250 0x7fa8680b4fe0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa86809e110 tx=0x7fa860011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:52.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.526+0000 7fa84ffff700 1 -- 172.21.15.156:0/1163650932 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000e040 con 0x7fa8680c4250 2023-11-28T11:10:52.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.526+0000 7fa84ffff700 1 -- 172.21.15.156:0/1163650932 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86000f040 con 0x7fa8680c4250 2023-11-28T11:10:52.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.526+0000 7fa86da08700 1 -- 172.21.15.156:0/1163650932 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86815d900 con 0x7fa8680c4250 2023-11-28T11:10:52.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.526+0000 7fa84ffff700 1 -- 172.21.15.156:0/1163650932 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000f890 con 0x7fa8680c4250 2023-11-28T11:10:52.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.526+0000 7fa86da08700 1 -- 172.21.15.156:0/1163650932 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86806a270 con 0x7fa8680c4250 2023-11-28T11:10:52.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.527+0000 7fa86da08700 1 -- 172.21.15.156:0/1163650932 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8680adb40 con 0x7fa8680c4250 2023-11-28T11:10:52.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.527+0000 7fa84ffff700 1 -- 172.21.15.156:0/1163650932 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa86000f9f0 con 0x7fa8680c4250 2023-11-28T11:10:52.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.528+0000 7fa84ffff700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa8540682f0 0x7fa85406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.528+0000 7fa866ffd700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa8540682f0 0x7fa85406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.528+0000 7fa84ffff700 1 -- 172.21.15.156:0/1163650932 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa860086390 con 0x7fa8680c4250 2023-11-28T11:10:52.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.528+0000 7fa866ffd700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa8540682f0 0x7fa85406a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa85c001eb0 tx=0x7fa85c001e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:52.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.532+0000 7fa84ffff700 1 -- 172.21.15.156:0/1163650932 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa860052b90 con 0x7fa8680c4250 2023-11-28T11:10:52.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.714+0000 7fa86da08700 1 -- 172.21.15.156:0/1163650932 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa86806dc10 con 0x7fa8680c4250 2023-11-28T11:10:52.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.716+0000 7fa84ffff700 1 -- 172.21.15.156:0/1163650932 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17982 (secure 0 0 0) 0x7fa8600561b0 con 0x7fa8680c4250 2023-11-28T11:10:52.718 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:52.718 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":62,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:10:52.367234+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:50.055098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:51.350508+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:52.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.718+0000 7fa86da08700 1 -- 172.21.15.156:0/1163650932 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa8540682f0 msgr2=0x7fa85406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.718+0000 7fa86da08700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa8540682f0 0x7fa85406a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa85c001eb0 tx=0x7fa85c001e00 comp rx=0 tx=0).stop 2023-11-28T11:10:52.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.718+0000 7fa86da08700 1 -- 172.21.15.156:0/1163650932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680c4250 msgr2=0x7fa8680b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.718+0000 7fa86da08700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680c4250 0x7fa8680b4fe0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa86809e110 tx=0x7fa860011ed0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.718+0000 7fa86da08700 1 -- 172.21.15.156:0/1163650932 shutdown_connections 2023-11-28T11:10:52.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.718+0000 7fa86da08700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680c4250 0x7fa8680b4fe0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.718+0000 7fa86da08700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa8540682f0 0x7fa85406a7a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.718+0000 7fa86da08700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa850004850 0x7fa8680ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.718+0000 7fa86da08700 1 --2- 172.21.15.156:0/1163650932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8680c5820 0x7fa8680ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.718+0000 7fa86da08700 1 -- 172.21.15.156:0/1163650932 >> 172.21.15.156:0/1163650932 conn(0x7fa8680d0000 msgr2=0x7fa86815f760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:52.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.718+0000 7fa86da08700 1 -- 172.21.15.156:0/1163650932 shutdown_connections 2023-11-28T11:10:52.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.718+0000 7fa86da08700 1 -- 172.21.15.156:0/1163650932 wait complete. 2023-11-28T11:10:52.733 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:52.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.851+0000 7f582442e700 1 Processor -- start 2023-11-28T11:10:52.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.851+0000 7f582442e700 1 -- start start 2023-11-28T11:10:52.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.852+0000 7f582442e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f581c15cef0 0x7f581c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.852+0000 7f582442e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f581c06ba90 0x7f581c0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.852+0000 7f582442e700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c0cc5f0 con 0x7f581c0cbdf0 2023-11-28T11:10:52.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.852+0000 7f582442e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c15d470 con 0x7f581c15cef0 2023-11-28T11:10:52.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.852+0000 7f582442e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c15d750 con 0x7f581c06ba90 2023-11-28T11:10:52.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.852+0000 7f58221ca700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f581c15cef0 0x7f581c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.853+0000 7f58221ca700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f581c15cef0 0x7f581c0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35602/0 (socket says 172.21.15.156:35602) 2023-11-28T11:10:52.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.853+0000 7f58221ca700 1 -- 172.21.15.156:0/1981003005 learned_addr learned my addr 172.21.15.156:0/1981003005 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:52.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.853+0000 7f58229cb700 1 --2- 172.21.15.156:0/1981003005 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f581c06ba90 0x7f581c0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.853+0000 7f58221ca700 1 -- 172.21.15.156:0/1981003005 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f581c06ba90 msgr2=0x7f581c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:52.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.853+0000 7f58221ca700 1 --2- 172.21.15.156:0/1981003005 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f581c06ba90 0x7f581c0ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.853+0000 7f58221ca700 1 -- 172.21.15.156:0/1981003005 >> v1:172.21.15.186:6789/0 conn(0x7f581c0cbdf0 legacy=0x7f581c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:52.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.853+0000 7f58221ca700 1 -- 172.21.15.156:0/1981003005 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c15da30 con 0x7f581c15cef0 2023-11-28T11:10:52.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.853+0000 7f58229cb700 1 --2- 172.21.15.156:0/1981003005 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f581c06ba90 0x7f581c0ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:52.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.853+0000 7f58221ca700 1 --2- 172.21.15.156:0/1981003005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f581c15cef0 0x7f581c0af400 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f580c009e80 tx=0x7f580c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2dd87c3b0ac5a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:52.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.853+0000 7f58211c8700 1 -- 172.21.15.156:0/1981003005 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00e070 con 0x7f581c15cef0 2023-11-28T11:10:52.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.853+0000 7f58211c8700 1 -- 172.21.15.156:0/1981003005 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c00b980 con 0x7f581c15cef0 2023-11-28T11:10:52.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.853+0000 7f58211c8700 1 -- 172.21.15.156:0/1981003005 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00bd30 con 0x7f581c15cef0 2023-11-28T11:10:52.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.854+0000 7f582442e700 1 -- 172.21.15.156:0/1981003005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f581c15cef0 msgr2=0x7f581c0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.854+0000 7f582442e700 1 --2- 172.21.15.156:0/1981003005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f581c15cef0 0x7f581c0af400 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f580c009e80 tx=0x7f580c00b330 comp rx=0 tx=0).stop 2023-11-28T11:10:52.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.854+0000 7f582442e700 1 -- 172.21.15.156:0/1981003005 shutdown_connections 2023-11-28T11:10:52.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.854+0000 7f582442e700 1 --2- 172.21.15.156:0/1981003005 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f581c06ba90 0x7f581c0ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.854+0000 7f582442e700 1 --2- 172.21.15.156:0/1981003005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f581c15cef0 0x7f581c0af400 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.854+0000 7f582442e700 1 -- 172.21.15.156:0/1981003005 >> 172.21.15.156:0/1981003005 conn(0x7f581c0d0000 msgr2=0x7f581c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:52.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.854+0000 7f582442e700 1 -- 172.21.15.156:0/1981003005 shutdown_connections 2023-11-28T11:10:52.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.854+0000 7f582442e700 1 -- 172.21.15.156:0/1981003005 wait complete. 2023-11-28T11:10:52.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.855+0000 7f582442e700 1 Processor -- start 2023-11-28T11:10:52.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.855+0000 7f582442e700 1 -- start start 2023-11-28T11:10:52.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f582442e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f581c06ba90 0x7f581c15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f582442e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f581c0cbdf0 0x7f581c09b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f582442e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f581c15cef0 0x7f581c0b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f582442e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c15aaf0 con 0x7f581c0cbdf0 2023-11-28T11:10:52.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f582442e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0c7440 con 0x7f581c06ba90 2023-11-28T11:10:52.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f582442e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0c7720 con 0x7f581c15cef0 2023-11-28T11:10:52.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f58219c9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f581c0cbdf0 0x7f581c09b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f58221ca700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f581c06ba90 0x7f581c15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f58219c9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f581c0cbdf0 0x7f581c09b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35608/0 (socket says 172.21.15.156:35608) 2023-11-28T11:10:52.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f58219c9700 1 -- 172.21.15.156:0/1278908239 learned_addr learned my addr 172.21.15.156:0/1278908239 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:52.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f58229cb700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f581c15cef0 0x7f581c0b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f58219c9700 1 -- 172.21.15.156:0/1278908239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f581c15cef0 msgr2=0x7f581c0b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f58219c9700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f581c15cef0 0x7f581c0b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f58219c9700 1 -- 172.21.15.156:0/1278908239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f581c06ba90 msgr2=0x7f581c15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f58219c9700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f581c06ba90 0x7f581c15a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f58219c9700 1 -- 172.21.15.156:0/1278908239 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c009b30 con 0x7f581c0cbdf0 2023-11-28T11:10:52.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f58229cb700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f581c15cef0 0x7f581c0b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:52.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f58219c9700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f581c0cbdf0 0x7f581c09b310 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f581000e940 tx=0x7f581000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:52.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f581affd700 1 -- 172.21.15.156:0/1278908239 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58100052d0 con 0x7f581c0cbdf0 2023-11-28T11:10:52.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f581affd700 1 -- 172.21.15.156:0/1278908239 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5810005430 con 0x7f581c0cbdf0 2023-11-28T11:10:52.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f582442e700 1 -- 172.21.15.156:0/1278908239 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c15dff0 con 0x7f581c0cbdf0 2023-11-28T11:10:52.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f581affd700 1 -- 172.21.15.156:0/1278908239 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581000ca20 con 0x7f581c0cbdf0 2023-11-28T11:10:52.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.856+0000 7f582442e700 1 -- 172.21.15.156:0/1278908239 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c15dd10 con 0x7f581c0cbdf0 2023-11-28T11:10:52.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.857+0000 7f582442e700 1 -- 172.21.15.156:0/1278908239 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c0cab90 con 0x7f581c0cbdf0 2023-11-28T11:10:52.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.858+0000 7f581affd700 1 -- 172.21.15.156:0/1278908239 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5810021070 con 0x7f581c0cbdf0 2023-11-28T11:10:52.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.858+0000 7f581affd700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f58080682f0 0x7f580806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.858+0000 7f581affd700 1 -- 172.21.15.156:0/1278908239 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5810087c30 con 0x7f581c0cbdf0 2023-11-28T11:10:52.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.858+0000 7f58221ca700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f58080682f0 0x7f580806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.859+0000 7f58221ca700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f58080682f0 0x7f580806a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b370 tx=0x7f580c009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:52.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:52.862+0000 7f581affd700 1 -- 172.21.15.156:0/1278908239 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5810054430 con 0x7f581c0cbdf0 2023-11-28T11:10:53.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.046+0000 7f582442e700 1 -- 172.21.15.156:0/1278908239 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f581c0d0e60 con 0x7f581c0cbdf0 2023-11-28T11:10:53.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.047+0000 7f581affd700 1 -- 172.21.15.156:0/1278908239 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17982 (secure 0 0 0) 0x7f581001f020 con 0x7f581c0cbdf0 2023-11-28T11:10:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:53.049 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":62,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:10:52.367234+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:50.055098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:51.350508+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:53.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.049+0000 7f582442e700 1 -- 172.21.15.156:0/1278908239 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f58080682f0 msgr2=0x7f580806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.049+0000 7f582442e700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f58080682f0 0x7f580806a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b370 tx=0x7f580c009000 comp rx=0 tx=0).stop 2023-11-28T11:10:53.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.049+0000 7f582442e700 1 -- 172.21.15.156:0/1278908239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f581c0cbdf0 msgr2=0x7f581c09b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.049+0000 7f582442e700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f581c0cbdf0 0x7f581c09b310 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f581000e940 tx=0x7f581000ec50 comp rx=0 tx=0).stop 2023-11-28T11:10:53.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.049+0000 7f582442e700 1 -- 172.21.15.156:0/1278908239 shutdown_connections 2023-11-28T11:10:53.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.049+0000 7f582442e700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f58080682f0 0x7f580806a7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.049+0000 7f582442e700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f581c15cef0 0x7f581c0b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.049+0000 7f582442e700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f581c06ba90 0x7f581c15a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.049+0000 7f582442e700 1 --2- 172.21.15.156:0/1278908239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f581c0cbdf0 0x7f581c09b310 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.049+0000 7f582442e700 1 -- 172.21.15.156:0/1278908239 >> 172.21.15.156:0/1278908239 conn(0x7f581c0d0000 msgr2=0x7f581c06d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:53.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.049+0000 7f582442e700 1 -- 172.21.15.156:0/1278908239 shutdown_connections 2023-11-28T11:10:53.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.049+0000 7f582442e700 1 -- 172.21.15.156:0/1278908239 wait complete. 2023-11-28T11:10:53.064 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:10:53.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.185+0000 7f76beba0700 1 Processor -- start 2023-11-28T11:10:53.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.185+0000 7f76beba0700 1 -- start start 2023-11-28T11:10:53.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.186+0000 7f76beba0700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f76b80cd8c0 0x7f76b8140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:53.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.186+0000 7f76beba0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76b806f850 0x7f76b806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:53.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.186+0000 7f76beba0700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b813ff00 con 0x7f76b80cdc90 2023-11-28T11:10:53.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.186+0000 7f76beba0700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b81404c0 con 0x7f76b80cd8c0 2023-11-28T11:10:53.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.186+0000 7f76beba0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b815aa80 con 0x7f76b806f850 2023-11-28T11:10:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.186+0000 7f76bd13d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76b806f850 0x7f76b806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.186+0000 7f76bc93c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f76b80cd8c0 0x7f76b8140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.186+0000 7f76bd13d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76b806f850 0x7f76b806fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:39482/0 (socket says 172.21.15.156:39482) 2023-11-28T11:10:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.186+0000 7f76bd13d700 1 -- 172.21.15.156:0/1481324010 learned_addr learned my addr 172.21.15.156:0/1481324010 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.186+0000 7f76b7fff700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f76b80cdc90 0x7f76b80b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:60852/0 (socket says 172.21.15.156:60852) 2023-11-28T11:10:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.186+0000 7f76bc93c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f76b80cd8c0 0x7f76b8140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:46630/0 (socket says 172.21.15.156:46630) 2023-11-28T11:10:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.187+0000 7f76bc93c700 1 -- 172.21.15.156:0/1481324010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76b806f850 msgr2=0x7f76b806fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.187+0000 7f76bc93c700 1 --2- 172.21.15.156:0/1481324010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76b806f850 0x7f76b806fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.187+0000 7f76bc93c700 1 -- 172.21.15.156:0/1481324010 >> v1:172.21.15.186:6789/0 conn(0x7f76b80cdc90 legacy=0x7f76b80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.187+0000 7f76bc93c700 1 -- 172.21.15.156:0/1481324010 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b80b3040 con 0x7f76b80cd8c0 2023-11-28T11:10:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.187+0000 7f76b77fe700 1 -- 172.21.15.156:0/1481324010 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355596704 0 0) 0x7f76b813ff00 con 0x7f76b80cdc90 2023-11-28T11:10:53.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.187+0000 7f76bd13d700 1 --2- 172.21.15.156:0/1481324010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76b806f850 0x7f76b806fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:53.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.187+0000 7f76bc93c700 1 --2- 172.21.15.156:0/1481324010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76b80cd8c0 0x7f76b8140950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009820 tx=0x7f76ac009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76b828060db865c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:53.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.187+0000 7f76b77fe700 1 -- 172.21.15.156:0/1481324010 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac005e30 con 0x7f76b80cd8c0 2023-11-28T11:10:53.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.187+0000 7f76b77fe700 1 -- 172.21.15.156:0/1481324010 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac006030 con 0x7f76b80cd8c0 2023-11-28T11:10:53.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.187+0000 7f76b77fe700 1 -- 172.21.15.156:0/1481324010 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac0063c0 con 0x7f76b80cd8c0 2023-11-28T11:10:53.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.188+0000 7f76beba0700 1 -- 172.21.15.156:0/1481324010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76b80cd8c0 msgr2=0x7f76b8140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.188+0000 7f76beba0700 1 --2- 172.21.15.156:0/1481324010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76b80cd8c0 0x7f76b8140950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009820 tx=0x7f76ac009b30 comp rx=0 tx=0).stop 2023-11-28T11:10:53.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.188+0000 7f76beba0700 1 -- 172.21.15.156:0/1481324010 shutdown_connections 2023-11-28T11:10:53.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.188+0000 7f76beba0700 1 --2- 172.21.15.156:0/1481324010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76b806f850 0x7f76b806fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.188+0000 7f76beba0700 1 --2- 172.21.15.156:0/1481324010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76b80cd8c0 0x7f76b8140950 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.188+0000 7f76beba0700 1 -- 172.21.15.156:0/1481324010 >> 172.21.15.156:0/1481324010 conn(0x7f76b806aed0 msgr2=0x7f76b806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:53.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.188+0000 7f76beba0700 1 -- 172.21.15.156:0/1481324010 shutdown_connections 2023-11-28T11:10:53.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.188+0000 7f76beba0700 1 -- 172.21.15.156:0/1481324010 wait complete. 2023-11-28T11:10:53.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.189+0000 7f76beba0700 1 Processor -- start 2023-11-28T11:10:53.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.189+0000 7f76beba0700 1 -- start start 2023-11-28T11:10:53.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76beba0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76b806f850 0x7f76b80ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:53.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76beba0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76b80cd8c0 0x7f76b813f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:53.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76beba0700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76b80cdc90 0x7f76b8156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:53.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76beba0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80c9860 con 0x7f76b806f850 2023-11-28T11:10:53.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76beba0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80c9b40 con 0x7f76b80cd8c0 2023-11-28T11:10:53.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76beba0700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80c9e20 con 0x7f76b80cdc90 2023-11-28T11:10:53.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76bc93c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76b806f850 0x7f76b80ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:53.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76bc93c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76b806f850 0x7f76b80ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35620/0 (socket says 172.21.15.156:35620) 2023-11-28T11:10:53.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76bc93c700 1 -- 172.21.15.156:0/634013196 learned_addr learned my addr 172.21.15.156:0/634013196 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:53.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76bd13d700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76b80cdc90 0x7f76b8156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:53.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76b7fff700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76b80cd8c0 0x7f76b813f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:53.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76bc93c700 1 -- 172.21.15.156:0/634013196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76b80cdc90 msgr2=0x7f76b8156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76bc93c700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76b80cdc90 0x7f76b8156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76bc93c700 1 -- 172.21.15.156:0/634013196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76b80cd8c0 msgr2=0x7f76b813f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76bc93c700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76b80cd8c0 0x7f76b813f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.190+0000 7f76bc93c700 1 -- 172.21.15.156:0/634013196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0009b30 con 0x7f76b806f850 2023-11-28T11:10:53.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.191+0000 7f76bc93c700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76b806f850 0x7f76b80ca900 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f76ac000c00 tx=0x7f76ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:53.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.191+0000 7f76b57fa700 1 -- 172.21.15.156:0/634013196 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac0061a0 con 0x7f76b806f850 2023-11-28T11:10:53.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.191+0000 7f76b57fa700 1 -- 172.21.15.156:0/634013196 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac020070 con 0x7f76b806f850 2023-11-28T11:10:53.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.191+0000 7f76b57fa700 1 -- 172.21.15.156:0/634013196 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac006540 con 0x7f76b806f850 2023-11-28T11:10:53.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.191+0000 7f76beba0700 1 -- 172.21.15.156:0/634013196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ac009430 con 0x7f76b806f850 2023-11-28T11:10:53.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.191+0000 7f76beba0700 1 -- 172.21.15.156:0/634013196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b80b35f0 con 0x7f76b806f850 2023-11-28T11:10:53.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.192+0000 7f76beba0700 1 -- 172.21.15.156:0/634013196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b80b3310 con 0x7f76b806f850 2023-11-28T11:10:53.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.192+0000 7f76b57fa700 1 -- 172.21.15.156:0/634013196 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f76ac02b020 con 0x7f76b806f850 2023-11-28T11:10:53.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.192+0000 7f76b57fa700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f769c0682f0 0x7f769c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:53.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.192+0000 7f76b57fa700 1 -- 172.21.15.156:0/634013196 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76ac087330 con 0x7f76b806f850 2023-11-28T11:10:53.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.192+0000 7f76b7fff700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f769c0682f0 0x7f769c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:53.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.193+0000 7f76b7fff700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f769c0682f0 0x7f769c06a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f76b80d0d20 tx=0x7f76a4007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:53.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.198+0000 7f76b57fa700 1 -- 172.21.15.156:0/634013196 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f76ac053b10 con 0x7f76b806f850 2023-11-28T11:10:53.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:53.426+0000 7f76beba0700 1 -- 172.21.15.156:0/634013196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f76b8140150 con 0x7f76b806f850 2023-11-28T11:10:54.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.380+0000 7f76b57fa700 1 -- 172.21.15.156:0/634013196 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f76ac057130 con 0x7f76b806f850 2023-11-28T11:10:54.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.382+0000 7f76beba0700 1 -- 172.21.15.156:0/634013196 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f769c0682f0 msgr2=0x7f769c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.382+0000 7f76beba0700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f769c0682f0 0x7f769c06a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f76b80d0d20 tx=0x7f76a4007e70 comp rx=0 tx=0).stop 2023-11-28T11:10:54.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.382+0000 7f76beba0700 1 -- 172.21.15.156:0/634013196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76b806f850 msgr2=0x7f76b80ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.382+0000 7f76beba0700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76b806f850 0x7f76b80ca900 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f76ac000c00 tx=0x7f76ac004750 comp rx=0 tx=0).stop 2023-11-28T11:10:54.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.383+0000 7f76beba0700 1 -- 172.21.15.156:0/634013196 shutdown_connections 2023-11-28T11:10:54.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.383+0000 7f76beba0700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f769c0682f0 0x7f769c06a7a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.383+0000 7f76beba0700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76b80cdc90 0x7f76b8156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.383+0000 7f76beba0700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76b806f850 0x7f76b80ca900 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.383+0000 7f76beba0700 1 --2- 172.21.15.156:0/634013196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76b80cd8c0 0x7f76b813f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.383+0000 7f76beba0700 1 -- 172.21.15.156:0/634013196 >> 172.21.15.156:0/634013196 conn(0x7f76b806aed0 msgr2=0x7f76b80cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:54.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.383+0000 7f76beba0700 1 -- 172.21.15.156:0/634013196 shutdown_connections 2023-11-28T11:10:54.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.383+0000 7f76beba0700 1 -- 172.21.15.156:0/634013196 wait complete. 2023-11-28T11:10:54.400 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:10:54.400 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:10:54.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.528+0000 7f4725c25700 1 Processor -- start 2023-11-28T11:10:54.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.528+0000 7f4725c25700 1 -- start start 2023-11-28T11:10:54.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.529+0000 7f4725c25700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f472006dee0 0x7f472006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.529+0000 7f4725c25700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472015d060 con 0x7f4720071040 2023-11-28T11:10:54.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.529+0000 7f4725c25700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472015d340 con 0x7f4720160540 2023-11-28T11:10:54.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.529+0000 7f4725c25700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472015d620 con 0x7f472006dee0 2023-11-28T11:10:54.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.529+0000 7f471effd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f472006dee0 0x7f472006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.529+0000 7f471f7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4720071040 0x7f4720071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50506/0 (socket says 172.21.15.156:50506) 2023-11-28T11:10:54.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.529+0000 7f471f7fe700 1 -- 172.21.15.156:0/417532537 learned_addr learned my addr 172.21.15.156:0/417532537 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:54.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.529+0000 7f471effd700 1 -- 172.21.15.156:0/417532537 >> v1:172.21.15.186:6789/0 conn(0x7f4720160540 legacy=0x7f4720160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:54.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.529+0000 7f471effd700 1 -- 172.21.15.156:0/417532537 >> v1:172.21.15.156:6789/0 conn(0x7f4720071040 legacy=0x7f4720071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.529+0000 7f471effd700 1 -- 172.21.15.156:0/417532537 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472015d900 con 0x7f472006dee0 2023-11-28T11:10:54.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.529+0000 7f471e7fc700 1 -- 172.21.15.156:0/417532537 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030299428 0 0) 0x7f472015d060 con 0x7f4720071040 2023-11-28T11:10:54.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.529+0000 7f471effd700 1 --2- 172.21.15.156:0/417532537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f472006dee0 0x7f472006e330 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4710009840 tx=0x7f4710009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1795ab6e67fd7349 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:54.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.530+0000 7f471e7fc700 1 -- 172.21.15.156:0/417532537 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710004230 con 0x7f472006dee0 2023-11-28T11:10:54.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.530+0000 7f471e7fc700 1 -- 172.21.15.156:0/417532537 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4710004390 con 0x7f472006dee0 2023-11-28T11:10:54.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.530+0000 7f471e7fc700 1 -- 172.21.15.156:0/417532537 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100036a0 con 0x7f472006dee0 2023-11-28T11:10:54.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.530+0000 7f4725c25700 1 -- 172.21.15.156:0/417532537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f472006dee0 msgr2=0x7f472006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.530+0000 7f4725c25700 1 --2- 172.21.15.156:0/417532537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f472006dee0 0x7f472006e330 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4710009840 tx=0x7f4710009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:54.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.530+0000 7f4725c25700 1 -- 172.21.15.156:0/417532537 shutdown_connections 2023-11-28T11:10:54.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.530+0000 7f4725c25700 1 --2- 172.21.15.156:0/417532537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f472006dee0 0x7f472006e330 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.530+0000 7f4725c25700 1 -- 172.21.15.156:0/417532537 >> 172.21.15.156:0/417532537 conn(0x7f47200d0000 msgr2=0x7f47200d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:54.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.530+0000 7f4725c25700 1 -- 172.21.15.156:0/417532537 shutdown_connections 2023-11-28T11:10:54.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.530+0000 7f4725c25700 1 -- 172.21.15.156:0/417532537 wait complete. 2023-11-28T11:10:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.531+0000 7f4725c25700 1 Processor -- start 2023-11-28T11:10:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.532+0000 7f4725c25700 1 -- start start 2023-11-28T11:10:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.532+0000 7f4725c25700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f472006dee0 0x7f47200b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.532+0000 7f4725c25700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4720071040 0x7f47200c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.532+0000 7f4725c25700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4720160540 0x7f47200aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.532+0000 7f4725c25700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47201410b0 con 0x7f472006dee0 2023-11-28T11:10:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.532+0000 7f4725c25700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4720141390 con 0x7f4720160540 2023-11-28T11:10:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.532+0000 7f4725c25700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4720141670 con 0x7f4720071040 2023-11-28T11:10:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.532+0000 7f471f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f472006dee0 0x7f47200b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.532+0000 7f471f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f472006dee0 0x7f47200b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35634/0 (socket says 172.21.15.156:35634) 2023-11-28T11:10:54.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.532+0000 7f471f7fe700 1 -- 172.21.15.156:0/4037223456 learned_addr learned my addr 172.21.15.156:0/4037223456 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:54.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.533+0000 7f471f7fe700 1 -- 172.21.15.156:0/4037223456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4720071040 msgr2=0x7f47200c2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.533+0000 7f471f7fe700 1 --2- 172.21.15.156:0/4037223456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4720071040 0x7f47200c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.533+0000 7f471f7fe700 1 -- 172.21.15.156:0/4037223456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4720160540 msgr2=0x7f47200aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.533+0000 7f471f7fe700 1 --2- 172.21.15.156:0/4037223456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4720160540 0x7f47200aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.533+0000 7f471f7fe700 1 -- 172.21.15.156:0/4037223456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47100094a0 con 0x7f472006dee0 2023-11-28T11:10:54.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.533+0000 7f471f7fe700 1 --2- 172.21.15.156:0/4037223456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f472006dee0 0x7f47200b2ec0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f471400c550 tx=0x7f471400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:54.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.533+0000 7f4707fff700 1 -- 172.21.15.156:0/4037223456 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714004c80 con 0x7f472006dee0 2023-11-28T11:10:54.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.533+0000 7f4707fff700 1 -- 172.21.15.156:0/4037223456 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4714004e00 con 0x7f472006dee0 2023-11-28T11:10:54.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.533+0000 7f4725c25700 1 -- 172.21.15.156:0/4037223456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472015ccd0 con 0x7f472006dee0 2023-11-28T11:10:54.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.533+0000 7f4707fff700 1 -- 172.21.15.156:0/4037223456 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714007310 con 0x7f472006dee0 2023-11-28T11:10:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.533+0000 7f4725c25700 1 -- 172.21.15.156:0/4037223456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472006e870 con 0x7f472006dee0 2023-11-28T11:10:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.534+0000 7f4725c25700 1 -- 172.21.15.156:0/4037223456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472006a270 con 0x7f472006dee0 2023-11-28T11:10:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.534+0000 7f4707fff700 1 -- 172.21.15.156:0/4037223456 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f471401a020 con 0x7f472006dee0 2023-11-28T11:10:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.535+0000 7f4707fff700 1 --2- 172.21.15.156:0/4037223456 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f470c0682f0 0x7f470c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.535+0000 7f471effd700 1 --2- 172.21.15.156:0/4037223456 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f470c0682f0 0x7f470c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.535+0000 7f4707fff700 1 -- 172.21.15.156:0/4037223456 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47140858e0 con 0x7f472006dee0 2023-11-28T11:10:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.535+0000 7f471effd700 1 --2- 172.21.15.156:0/4037223456 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f470c0682f0 0x7f470c06a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f47100090b0 tx=0x7f4710013040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.540+0000 7f4707fff700 1 -- 172.21.15.156:0/4037223456 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4714052170 con 0x7f472006dee0 2023-11-28T11:10:54.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:54.766+0000 7f4725c25700 1 -- 172.21.15.156:0/4037223456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4720141b60 con 0x7f472006dee0 2023-11-28T11:10:55.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.385+0000 7f4707fff700 1 -- 172.21.15.156:0/4037223456 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4714055790 con 0x7f472006dee0 2023-11-28T11:10:55.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.388+0000 7f4725c25700 1 -- 172.21.15.156:0/4037223456 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f470c0682f0 msgr2=0x7f470c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.388+0000 7f4725c25700 1 --2- 172.21.15.156:0/4037223456 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f470c0682f0 0x7f470c06a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f47100090b0 tx=0x7f4710013040 comp rx=0 tx=0).stop 2023-11-28T11:10:55.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.388+0000 7f4725c25700 1 -- 172.21.15.156:0/4037223456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f472006dee0 msgr2=0x7f47200b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.388+0000 7f4725c25700 1 --2- 172.21.15.156:0/4037223456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f472006dee0 0x7f47200b2ec0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f471400c550 tx=0x7f471400c910 comp rx=0 tx=0).stop 2023-11-28T11:10:55.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.388+0000 7f4725c25700 1 -- 172.21.15.156:0/4037223456 shutdown_connections 2023-11-28T11:10:55.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.388+0000 7f4725c25700 1 --2- 172.21.15.156:0/4037223456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4720160540 0x7f47200aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.388+0000 7f4725c25700 1 --2- 172.21.15.156:0/4037223456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f472006dee0 0x7f47200b2ec0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.388+0000 7f4725c25700 1 --2- 172.21.15.156:0/4037223456 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f470c0682f0 0x7f470c06a7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.388+0000 7f4725c25700 1 --2- 172.21.15.156:0/4037223456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4720071040 0x7f47200c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.388+0000 7f4725c25700 1 -- 172.21.15.156:0/4037223456 >> 172.21.15.156:0/4037223456 conn(0x7f47200d0000 msgr2=0x7f472015e190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:55.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.389+0000 7f4725c25700 1 -- 172.21.15.156:0/4037223456 shutdown_connections 2023-11-28T11:10:55.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.389+0000 7f4725c25700 1 -- 172.21.15.156:0/4037223456 wait complete. 2023-11-28T11:10:55.405 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:10:55.405 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:55.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.531+0000 7f088ce01700 1 Processor -- start 2023-11-28T11:10:55.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.531+0000 7f088ce01700 1 -- start start 2023-11-28T11:10:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088ce01700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f088815ace0 0x7f08880b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088ce01700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08880d0240 con 0x7f08880cd5a0 2023-11-28T11:10:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088ce01700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08880cdd60 con 0x7f08880cf1b0 2023-11-28T11:10:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088ce01700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08880ce040 con 0x7f088815ace0 2023-11-28T11:10:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f0885d9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f08880cd5a0 0x7f08880ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50520/0 (socket says 172.21.15.156:50520) 2023-11-28T11:10:55.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f0885d9b700 1 -- 172.21.15.156:0/3383340813 learned_addr learned my addr 172.21.15.156:0/3383340813 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656176321 0 0) 0x7f08880d0240 con 0x7f08880cd5a0 2023-11-28T11:10:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088659c700 1 --2- 172.21.15.156:0/3383340813 >> v2:172.21.15.186:3300/0 conn(0x7f088815ace0 0x7f08880b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08700035b0 con 0x7f08880cd5a0 2023-11-28T11:10:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 971589307 0 0) 0x7f08700035b0 con 0x7f08880cd5a0 2023-11-28T11:10:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 >> v2:172.21.15.186:3300/0 conn(0x7f088815ace0 msgr2=0x7f08880b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 --2- 172.21.15.156:0/3383340813 >> v2:172.21.15.186:3300/0 conn(0x7f088815ace0 0x7f08880b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 >> v1:172.21.15.186:6789/0 conn(0x7f08880cf1b0 legacy=0x7f08880d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08880d04e0 con 0x7f08880cd5a0 2023-11-28T11:10:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f087c001670 con 0x7f08880cd5a0 2023-11-28T11:10:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 >> v1:172.21.15.156:6789/0 conn(0x7f08880cd5a0 legacy=0x7f08880ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 --2- 172.21.15.156:0/3383340813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 0x7f0870003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 --2- 172.21.15.156:0/3383340813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0870004850 0x7f0870004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 --2- 172.21.15.156:0/3383340813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0870005200 0x7f087000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08880ce040 con 0x7f0870003a00 2023-11-28T11:10:55.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087000c080 con 0x7f0870005200 2023-11-28T11:10:55.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087000c240 con 0x7f0870004850 2023-11-28T11:10:55.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.532+0000 7f088659c700 1 --2- 172.21.15.156:0/3383340813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 0x7f0870003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f0885d9b700 1 --2- 172.21.15.156:0/3383340813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0870004850 0x7f0870004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f0886d9d700 1 --2- 172.21.15.156:0/3383340813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0870005200 0x7f087000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f088659c700 1 -- 172.21.15.156:0/3383340813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0870004850 msgr2=0x7f0870004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:55.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f088659c700 1 --2- 172.21.15.156:0/3383340813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0870004850 0x7f0870004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f088659c700 1 -- 172.21.15.156:0/3383340813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0870005200 msgr2=0x7f087000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:55.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f088659c700 1 --2- 172.21.15.156:0/3383340813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0870005200 0x7f087000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f088659c700 1 -- 172.21.15.156:0/3383340813 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08700035b0 con 0x7f0870003a00 2023-11-28T11:10:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f0886d9d700 1 --2- 172.21.15.156:0/3383340813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0870005200 0x7f087000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f0886d9d700 1 -- 172.21.15.156:0/3383340813 reap_dead start 2023-11-28T11:10:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f088659c700 1 --2- 172.21.15.156:0/3383340813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 0x7f0870003e10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0878009b60 tx=0x7f0870005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f300a19d943d14 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800e6c0 con 0x7f0870003a00 2023-11-28T11:10:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087800e820 con 0x7f0870003a00 2023-11-28T11:10:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.533+0000 7f088559a700 1 -- 172.21.15.156:0/3383340813 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800eb90 con 0x7f0870003a00 2023-11-28T11:10:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.534+0000 7f088ce01700 1 -- 172.21.15.156:0/3383340813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 msgr2=0x7f0870003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.534+0000 7f088ce01700 1 --2- 172.21.15.156:0/3383340813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 0x7f0870003e10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0878009b60 tx=0x7f0870005880 comp rx=0 tx=0).stop 2023-11-28T11:10:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.534+0000 7f088ce01700 1 -- 172.21.15.156:0/3383340813 shutdown_connections 2023-11-28T11:10:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.534+0000 7f088ce01700 1 --2- 172.21.15.156:0/3383340813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 0x7f0870003e10 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.534+0000 7f088ce01700 1 -- 172.21.15.156:0/3383340813 >> 172.21.15.156:0/3383340813 conn(0x7f088806a270 msgr2=0x7f088806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.534+0000 7f088ce01700 1 -- 172.21.15.156:0/3383340813 shutdown_connections 2023-11-28T11:10:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.534+0000 7f088ce01700 1 -- 172.21.15.156:0/3383340813 wait complete. 2023-11-28T11:10:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.535+0000 7f088ce01700 1 Processor -- start 2023-11-28T11:10:55.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.535+0000 7f088ce01700 1 -- start start 2023-11-28T11:10:55.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088ce01700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 0x7f08881e4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088ce01700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08881e4ee0 0x7f08881504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088ce01700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08880b6220 0x7f08880b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088ce01700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088806bd20 con 0x7f0870003a00 2023-11-28T11:10:55.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088ce01700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08880b6be0 con 0x7f08880b6220 2023-11-28T11:10:55.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088ce01700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08880b0420 con 0x7f08881e4ee0 2023-11-28T11:10:55.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 0x7f08881e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 0x7f08881e4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35654/0 (socket says 172.21.15.156:35654) 2023-11-28T11:10:55.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088659c700 1 -- 172.21.15.156:0/1173422521 learned_addr learned my addr 172.21.15.156:0/1173422521 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:55.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f0885d9b700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08881e4ee0 0x7f08881504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f0886d9d700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08880b6220 0x7f08880b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088659c700 1 -- 172.21.15.156:0/1173422521 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08881e4ee0 msgr2=0x7f08881504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088659c700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08881e4ee0 0x7f08881504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088659c700 1 -- 172.21.15.156:0/1173422521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08880b6220 msgr2=0x7f08880b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088659c700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08880b6220 0x7f08880b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088659c700 1 -- 172.21.15.156:0/1173422521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08880cda40 con 0x7f0870003a00 2023-11-28T11:10:55.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f0886d9d700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08880b6220 0x7f08880b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:55.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.536+0000 7f088659c700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 0x7f08881e4900 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f087000c050 tx=0x7f08780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:55.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.537+0000 7f0876ffd700 1 -- 172.21.15.156:0/1173422521 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800e6c0 con 0x7f0870003a00 2023-11-28T11:10:55.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.537+0000 7f0876ffd700 1 -- 172.21.15.156:0/1173422521 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0878003ad0 con 0x7f0870003a00 2023-11-28T11:10:55.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.537+0000 7f088ce01700 1 -- 172.21.15.156:0/1173422521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08880ce320 con 0x7f0870003a00 2023-11-28T11:10:55.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.537+0000 7f0876ffd700 1 -- 172.21.15.156:0/1173422521 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878003e80 con 0x7f0870003a00 2023-11-28T11:10:55.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.537+0000 7f088ce01700 1 -- 172.21.15.156:0/1173422521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08880b29c0 con 0x7f0870003a00 2023-11-28T11:10:55.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.537+0000 7f088ce01700 1 -- 172.21.15.156:0/1173422521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0888068690 con 0x7f0870003a00 2023-11-28T11:10:55.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.538+0000 7f0876ffd700 1 -- 172.21.15.156:0/1173422521 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0878019030 con 0x7f0870003a00 2023-11-28T11:10:55.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.538+0000 7f0876ffd700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f08680682f0 0x7f086806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.538+0000 7f0876ffd700 1 -- 172.21.15.156:0/1173422521 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0878087310 con 0x7f0870003a00 2023-11-28T11:10:55.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.538+0000 7f0885d9b700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f08680682f0 0x7f086806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.539+0000 7f0885d9b700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f08680682f0 0x7f086806a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f087c005b40 tx=0x7f087c007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:55.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.543+0000 7f0876ffd700 1 -- 172.21.15.156:0/1173422521 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0878053b70 con 0x7f0870003a00 2023-11-28T11:10:55.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.768+0000 7f088ce01700 1 -- 172.21.15.156:0/1173422521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08880d1650 con 0x7f0870003a00 2023-11-28T11:10:55.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.769+0000 7f0876ffd700 1 -- 172.21.15.156:0/1173422521 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f0878057190 con 0x7f0870003a00 2023-11-28T11:10:55.770 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:55.770 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6006,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1991065152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1991065152},{"type":"v1","addr":"172.21.15.156:6836","nonce":1991065152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/43543030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":43543030},{"type":"v1","addr":"172.21.15.156:6837","nonce":43543030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/2480189069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2480189069},{"type":"v1","addr":"172.21.15.186:6834","nonce":2480189069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:52.367543+0000","modified":"2023-11-28T11:10:55.380411+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6021":{"gid":6021,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3811791267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3811791267},{"type":"v1","addr":"172.21.15.186:6835","nonce":3811791267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:10:55.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.770+0000 7f088ce01700 1 -- 172.21.15.156:0/1173422521 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f08680682f0 msgr2=0x7f086806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.770+0000 7f088ce01700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f08680682f0 0x7f086806a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f087c005b40 tx=0x7f087c007040 comp rx=0 tx=0).stop 2023-11-28T11:10:55.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.770+0000 7f088ce01700 1 -- 172.21.15.156:0/1173422521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 msgr2=0x7f08881e4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.770+0000 7f088ce01700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 0x7f08881e4900 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f087000c050 tx=0x7f08780047c0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.771+0000 7f088ce01700 1 -- 172.21.15.156:0/1173422521 shutdown_connections 2023-11-28T11:10:55.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.771+0000 7f088ce01700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08880b6220 0x7f08880b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.771+0000 7f088ce01700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0870003a00 0x7f08881e4900 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.771+0000 7f088ce01700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f08680682f0 0x7f086806a7a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.771+0000 7f088ce01700 1 --2- 172.21.15.156:0/1173422521 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08881e4ee0 0x7f08881504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.771+0000 7f088ce01700 1 -- 172.21.15.156:0/1173422521 >> 172.21.15.156:0/1173422521 conn(0x7f088806a270 msgr2=0x7f08880d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:55.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.771+0000 7f088ce01700 1 -- 172.21.15.156:0/1173422521 shutdown_connections 2023-11-28T11:10:55.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.771+0000 7f088ce01700 1 -- 172.21.15.156:0/1173422521 wait complete. 2023-11-28T11:10:55.774 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 31 2023-11-28T11:10:55.785 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:55.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.903+0000 7fab4f3d2700 1 Processor -- start 2023-11-28T11:10:55.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.904+0000 7fab4f3d2700 1 -- start start 2023-11-28T11:10:55.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.904+0000 7fab4f3d2700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fab48154e70 0x7fab480cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.904+0000 7fab4f3d2700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab480cf360 0x7fab48071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.904+0000 7fab4f3d2700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fab48071580 0x7fab4815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.904+0000 7fab4f3d2700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab480719e0 con 0x7fab48154e70 2023-11-28T11:10:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.904+0000 7fab4f3d2700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab48155240 con 0x7fab48071580 2023-11-28T11:10:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.904+0000 7fab4f3d2700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4815b100 con 0x7fab480cf360 2023-11-28T11:10:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab4d16e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fab48154e70 0x7fab480cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab4d16e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fab48154e70 0x7fab480cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35662/0 (socket says 172.21.15.156:35662) 2023-11-28T11:10:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab4d16e700 1 -- 172.21.15.156:0/4097670214 learned_addr learned my addr 172.21.15.156:0/4097670214 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab4d96f700 1 --2- 172.21.15.156:0/4097670214 >> v2:172.21.15.186:3300/0 conn(0x7fab48071580 0x7fab4815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab4c96d700 1 --2- 172.21.15.156:0/4097670214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab480cf360 0x7fab48071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab4d16e700 1 -- 172.21.15.156:0/4097670214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab480cf360 msgr2=0x7fab48071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab4d16e700 1 --2- 172.21.15.156:0/4097670214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab480cf360 0x7fab48071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab4d16e700 1 -- 172.21.15.156:0/4097670214 >> v2:172.21.15.186:3300/0 conn(0x7fab48071580 msgr2=0x7fab4815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab4d16e700 1 --2- 172.21.15.156:0/4097670214 >> v2:172.21.15.186:3300/0 conn(0x7fab48071580 0x7fab4815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab4d16e700 1 -- 172.21.15.156:0/4097670214 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4815b3e0 con 0x7fab48154e70 2023-11-28T11:10:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab4d16e700 1 --2- 172.21.15.156:0/4097670214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab48154e70 0x7fab480cdbd0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fab480af0c0 tx=0x7fab4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3167310a15ed00dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab3ffff700 1 -- 172.21.15.156:0/4097670214 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4000e070 con 0x7fab48154e70 2023-11-28T11:10:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab3ffff700 1 -- 172.21.15.156:0/4097670214 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4000b9c0 con 0x7fab48154e70 2023-11-28T11:10:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.905+0000 7fab3ffff700 1 -- 172.21.15.156:0/4097670214 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4000bd70 con 0x7fab48154e70 2023-11-28T11:10:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.906+0000 7fab4f3d2700 1 -- 172.21.15.156:0/4097670214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab48154e70 msgr2=0x7fab480cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.906+0000 7fab4f3d2700 1 --2- 172.21.15.156:0/4097670214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab48154e70 0x7fab480cdbd0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fab480af0c0 tx=0x7fab4000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.906+0000 7fab4f3d2700 1 -- 172.21.15.156:0/4097670214 shutdown_connections 2023-11-28T11:10:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.906+0000 7fab4f3d2700 1 --2- 172.21.15.156:0/4097670214 >> v2:172.21.15.186:3300/0 conn(0x7fab48071580 0x7fab4815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.906+0000 7fab4f3d2700 1 --2- 172.21.15.156:0/4097670214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab48154e70 0x7fab480cdbd0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.906+0000 7fab4f3d2700 1 --2- 172.21.15.156:0/4097670214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab480cf360 0x7fab48071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.906+0000 7fab4f3d2700 1 -- 172.21.15.156:0/4097670214 >> 172.21.15.156:0/4097670214 conn(0x7fab480d0000 msgr2=0x7fab480d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:55.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.906+0000 7fab4f3d2700 1 -- 172.21.15.156:0/4097670214 shutdown_connections 2023-11-28T11:10:55.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.906+0000 7fab4f3d2700 1 -- 172.21.15.156:0/4097670214 wait complete. 2023-11-28T11:10:55.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.907+0000 7fab4f3d2700 1 Processor -- start 2023-11-28T11:10:55.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.907+0000 7fab4f3d2700 1 -- start start 2023-11-28T11:10:55.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.908+0000 7fab4f3d2700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48071580 0x7fab481f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.908+0000 7fab4f3d2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab480cf360 0x7fab48151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.908+0000 7fab4f3d2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab48154e70 0x7fab48159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.908+0000 7fab4f3d2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4815a6a0 con 0x7fab48154e70 2023-11-28T11:10:55.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.908+0000 7fab4f3d2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48152380 con 0x7fab480cf360 2023-11-28T11:10:55.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.908+0000 7fab4f3d2700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab480b47c0 con 0x7fab48071580 2023-11-28T11:10:55.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.908+0000 7fab4d96f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab48154e70 0x7fab48159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.908+0000 7fab4d96f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab48154e70 0x7fab48159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35672/0 (socket says 172.21.15.156:35672) 2023-11-28T11:10:55.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.908+0000 7fab4d96f700 1 -- 172.21.15.156:0/19807314 learned_addr learned my addr 172.21.15.156:0/19807314 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:55.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab4c96d700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab480cf360 0x7fab48151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab4d16e700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48071580 0x7fab481f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab4d96f700 1 -- 172.21.15.156:0/19807314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48071580 msgr2=0x7fab481f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab4d96f700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48071580 0x7fab481f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab4d96f700 1 -- 172.21.15.156:0/19807314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab480cf360 msgr2=0x7fab48151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab4d96f700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab480cf360 0x7fab48151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab4d96f700 1 -- 172.21.15.156:0/19807314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab40009b30 con 0x7fab48154e70 2023-11-28T11:10:55.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab4c96d700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab480cf360 0x7fab48151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:55.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab4d96f700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab48154e70 0x7fab48159a00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fab4400e810 tx=0x7fab4400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:55.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab3dffb700 1 -- 172.21.15.156:0/19807314 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4400c070 con 0x7fab48154e70 2023-11-28T11:10:55.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab3dffb700 1 -- 172.21.15.156:0/19807314 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab44013610 con 0x7fab48154e70 2023-11-28T11:10:55.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab4f3d2700 1 -- 172.21.15.156:0/19807314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4815b9a0 con 0x7fab48154e70 2023-11-28T11:10:55.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab3dffb700 1 -- 172.21.15.156:0/19807314 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab440139c0 con 0x7fab48154e70 2023-11-28T11:10:55.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.909+0000 7fab4f3d2700 1 -- 172.21.15.156:0/19807314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4815b6c0 con 0x7fab48154e70 2023-11-28T11:10:55.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.910+0000 7fab4f3d2700 1 -- 172.21.15.156:0/19807314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab480af520 con 0x7fab48154e70 2023-11-28T11:10:55.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.910+0000 7fab3dffb700 1 -- 172.21.15.156:0/19807314 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fab44023070 con 0x7fab48154e70 2023-11-28T11:10:55.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.911+0000 7fab3dffb700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fab2c0682f0 0x7fab2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.911+0000 7fab3dffb700 1 -- 172.21.15.156:0/19807314 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab44085990 con 0x7fab48154e70 2023-11-28T11:10:55.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.911+0000 7fab4d16e700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fab2c0682f0 0x7fab2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.911+0000 7fab4d16e700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fab2c0682f0 0x7fab2c06a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fab480ad720 tx=0x7fab40009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:55.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:55.915+0000 7fab3dffb700 1 -- 172.21.15.156:0/19807314 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fab440521f0 con 0x7fab48154e70 2023-11-28T11:10:56.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.097+0000 7fab4f3d2700 1 -- 172.21.15.156:0/19807314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab480b4930 con 0x7fab48154e70 2023-11-28T11:10:56.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.098+0000 7fab3dffb700 1 -- 172.21.15.156:0/19807314 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17969 (secure 0 0 0) 0x7fab4400f040 con 0x7fab48154e70 2023-11-28T11:10:56.101 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:56.101 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":63,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:10:53.369390+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:50.055098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:51.350508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:56.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.101+0000 7fab4f3d2700 1 -- 172.21.15.156:0/19807314 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fab2c0682f0 msgr2=0x7fab2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.101+0000 7fab4f3d2700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fab2c0682f0 0x7fab2c06a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fab480ad720 tx=0x7fab40009000 comp rx=0 tx=0).stop 2023-11-28T11:10:56.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.101+0000 7fab4f3d2700 1 -- 172.21.15.156:0/19807314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab48154e70 msgr2=0x7fab48159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.101+0000 7fab4f3d2700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab48154e70 0x7fab48159a00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fab4400e810 tx=0x7fab4400ebd0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.101+0000 7fab4f3d2700 1 -- 172.21.15.156:0/19807314 shutdown_connections 2023-11-28T11:10:56.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.101+0000 7fab4f3d2700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab48154e70 0x7fab48159a00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.101+0000 7fab4f3d2700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fab2c0682f0 0x7fab2c06a7a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.101+0000 7fab4f3d2700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48071580 0x7fab481f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.101+0000 7fab4f3d2700 1 --2- 172.21.15.156:0/19807314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab480cf360 0x7fab48151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.101+0000 7fab4f3d2700 1 -- 172.21.15.156:0/19807314 >> 172.21.15.156:0/19807314 conn(0x7fab480d0000 msgr2=0x7fab480add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:56.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.101+0000 7fab4f3d2700 1 -- 172.21.15.156:0/19807314 shutdown_connections 2023-11-28T11:10:56.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.101+0000 7fab4f3d2700 1 -- 172.21.15.156:0/19807314 wait complete. 2023-11-28T11:10:56.114 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:10:56.115 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:10:56.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.235+0000 7f29406ac700 1 Processor -- start 2023-11-28T11:10:56.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.235+0000 7f29406ac700 1 -- start start 2023-11-28T11:10:56.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.236+0000 7f29406ac700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f293806e9f0 0x7f293806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.236+0000 7f29406ac700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f293806b830 0x7f293806c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.236+0000 7f29406ac700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293806ae60 con 0x7f293806ced0 2023-11-28T11:10:56.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.236+0000 7f29406ac700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293806d330 con 0x7f293806b830 2023-11-28T11:10:56.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.236+0000 7f29406ac700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293806a270 con 0x7f293806e9f0 2023-11-28T11:10:56.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.236+0000 7f293ec49700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f293806ced0 0x7f29380ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50558/0 (socket says 172.21.15.156:50558) 2023-11-28T11:10:56.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.236+0000 7f293ec49700 1 -- 172.21.15.156:0/2463884354 learned_addr learned my addr 172.21.15.156:0/2463884354 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:56.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.236+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128536318 0 0) 0x7f293806ae60 con 0x7f293806ced0 2023-11-28T11:10:56.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.236+0000 7f293dc47700 1 --2- 172.21.15.156:0/2463884354 >> v2:172.21.15.186:3300/0 conn(0x7f293806b830 0x7f293806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.236+0000 7f293e448700 1 --2- 172.21.15.156:0/2463884354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f293806e9f0 0x7f293806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.236+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29240057c0 con 0x7f293806ced0 2023-11-28T11:10:56.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3228182018 0 0) 0x7f29240057c0 con 0x7f293806ced0 2023-11-28T11:10:56.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f293806e9f0 msgr2=0x7f293806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 --2- 172.21.15.156:0/2463884354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f293806e9f0 0x7f293806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 >> v2:172.21.15.186:3300/0 conn(0x7f293806b830 msgr2=0x7f293806c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 --2- 172.21.15.156:0/2463884354 >> v2:172.21.15.186:3300/0 conn(0x7f293806b830 0x7f293806c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293806a530 con 0x7f293806ced0 2023-11-28T11:10:56.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f29300019f0 con 0x7f293806ced0 2023-11-28T11:10:56.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 >> v1:172.21.15.156:6789/0 conn(0x7f293806ced0 legacy=0x7f29380ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 --2- 172.21.15.156:0/2463884354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f293806e9f0 0x7f2924005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 --2- 172.21.15.156:0/2463884354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2924006190 0x7f2924006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 --2- 172.21.15.156:0/2463884354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2924006fa0 0x7f2924007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293806d330 con 0x7f2924006190 2023-11-28T11:10:56.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293806a270 con 0x7f2924006fa0 2023-11-28T11:10:56.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292400dc50 con 0x7f293806e9f0 2023-11-28T11:10:56.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293dc47700 1 --2- 172.21.15.156:0/2463884354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2924006190 0x7f2924006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293e448700 1 --2- 172.21.15.156:0/2463884354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f293806e9f0 0x7f2924005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293ec49700 1 --2- 172.21.15.156:0/2463884354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2924006fa0 0x7f2924007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293dc47700 1 -- 172.21.15.156:0/2463884354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f293806e9f0 msgr2=0x7f2924005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293dc47700 1 --2- 172.21.15.156:0/2463884354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f293806e9f0 0x7f2924005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293dc47700 1 -- 172.21.15.156:0/2463884354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2924006fa0 msgr2=0x7f2924007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293dc47700 1 --2- 172.21.15.156:0/2463884354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2924006fa0 0x7f2924007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293dc47700 1 -- 172.21.15.156:0/2463884354 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29240057c0 con 0x7f2924006190 2023-11-28T11:10:56.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293dc47700 1 --2- 172.21.15.156:0/2463884354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2924006190 0x7f2924006a60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f29340075b0 tx=0x7f2924007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5149c21bbda7ca13 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934007be0 con 0x7f2924006190 2023-11-28T11:10:56.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293400f040 con 0x7f2924006190 2023-11-28T11:10:56.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.237+0000 7f293d446700 1 -- 172.21.15.156:0/2463884354 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293400e630 con 0x7f2924006190 2023-11-28T11:10:56.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.238+0000 7f29406ac700 1 -- 172.21.15.156:0/2463884354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2924006190 msgr2=0x7f2924006a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.238+0000 7f29406ac700 1 --2- 172.21.15.156:0/2463884354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2924006190 0x7f2924006a60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f29340075b0 tx=0x7f2924007a90 comp rx=0 tx=0).stop 2023-11-28T11:10:56.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.238+0000 7f293ec49700 1 -- 172.21.15.156:0/2463884354 reap_dead start 2023-11-28T11:10:56.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.238+0000 7f29406ac700 1 -- 172.21.15.156:0/2463884354 shutdown_connections 2023-11-28T11:10:56.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.238+0000 7f29406ac700 1 -- 172.21.15.156:0/2463884354 >> 172.21.15.156:0/2463884354 conn(0x7f29380b12b0 msgr2=0x7f29380b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:56.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.238+0000 7f29406ac700 1 -- 172.21.15.156:0/2463884354 shutdown_connections 2023-11-28T11:10:56.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.238+0000 7f29406ac700 1 -- 172.21.15.156:0/2463884354 wait complete. 2023-11-28T11:10:56.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.239+0000 7f29406ac700 1 Processor -- start 2023-11-28T11:10:56.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.239+0000 7f29406ac700 1 -- start start 2023-11-28T11:10:56.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.240+0000 7f29406ac700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2924006190 0x7f2938153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.240+0000 7f29406ac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29381deb00 0x7f29380c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.240+0000 7f29406ac700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29380c8fa0 0x7f29380c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.240+0000 7f29406ac700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380c9420 con 0x7f29380c8fa0 2023-11-28T11:10:56.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.240+0000 7f29406ac700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380c4be0 con 0x7f29381deb00 2023-11-28T11:10:56.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.240+0000 7f29406ac700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29381def70 con 0x7f2924006190 2023-11-28T11:10:56.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.240+0000 7f293ec49700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29380c8fa0 0x7f29380c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.240+0000 7f293ec49700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29380c8fa0 0x7f29380c2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35692/0 (socket says 172.21.15.156:35692) 2023-11-28T11:10:56.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.240+0000 7f293ec49700 1 -- 172.21.15.156:0/1812074107 learned_addr learned my addr 172.21.15.156:0/1812074107 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:56.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.240+0000 7f293e448700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2924006190 0x7f2938153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.240+0000 7f293dc47700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29381deb00 0x7f29380c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f293ec49700 1 -- 172.21.15.156:0/1812074107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2924006190 msgr2=0x7f2938153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f293ec49700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2924006190 0x7f2938153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f293ec49700 1 -- 172.21.15.156:0/1812074107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29381deb00 msgr2=0x7f29380c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f293ec49700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29381deb00 0x7f29380c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f293ec49700 1 -- 172.21.15.156:0/1812074107 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293806aaf0 con 0x7f29380c8fa0 2023-11-28T11:10:56.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f293e448700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2924006190 0x7f2938153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:56.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f293dc47700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29381deb00 0x7f29380c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:56.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f293ec49700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29380c8fa0 0x7f29380c2f90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2930001330 tx=0x7f293000a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f292effd700 1 -- 172.21.15.156:0/1812074107 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293000ae70 con 0x7f29380c8fa0 2023-11-28T11:10:56.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f292effd700 1 -- 172.21.15.156:0/1812074107 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29300133f0 con 0x7f29380c8fa0 2023-11-28T11:10:56.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f29406ac700 1 -- 172.21.15.156:0/1812074107 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293806a810 con 0x7f29380c8fa0 2023-11-28T11:10:56.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f292effd700 1 -- 172.21.15.156:0/1812074107 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29300137c0 con 0x7f29380c8fa0 2023-11-28T11:10:56.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.241+0000 7f29406ac700 1 -- 172.21.15.156:0/1812074107 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293806edc0 con 0x7f29380c8fa0 2023-11-28T11:10:56.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.242+0000 7f29406ac700 1 -- 172.21.15.156:0/1812074107 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29380cd5b0 con 0x7f29380c8fa0 2023-11-28T11:10:56.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.242+0000 7f292effd700 1 -- 172.21.15.156:0/1812074107 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2930023070 con 0x7f29380c8fa0 2023-11-28T11:10:56.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.242+0000 7f292effd700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f291c0682f0 0x7f291c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.243+0000 7f292effd700 1 -- 172.21.15.156:0/1812074107 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2930086770 con 0x7f29380c8fa0 2023-11-28T11:10:56.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.243+0000 7f293e448700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f291c0682f0 0x7f291c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.243+0000 7f293e448700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f291c0682f0 0x7f291c06a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2928009930 tx=0x7f2928008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.247+0000 7f292effd700 1 -- 172.21.15.156:0/1812074107 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2930052fd0 con 0x7f29380c8fa0 2023-11-28T11:10:56.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.427+0000 7f29406ac700 1 -- 172.21.15.156:0/1812074107 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2938066230 con 0x7f291c0682f0 2023-11-28T11:10:56.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.435+0000 7f292effd700 1 -- 172.21.15.156:0/1812074107 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300247 (secure 0 0 0) 0x7f2938066230 con 0x7f291c0682f0 2023-11-28T11:10:56.440 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:56.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.438+0000 7f29406ac700 1 -- 172.21.15.156:0/1812074107 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f291c0682f0 msgr2=0x7f291c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.438+0000 7f29406ac700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f291c0682f0 0x7f291c06a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2928009930 tx=0x7f2928008040 comp rx=0 tx=0).stop 2023-11-28T11:10:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.438+0000 7f29406ac700 1 -- 172.21.15.156:0/1812074107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29380c8fa0 msgr2=0x7f29380c2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.438+0000 7f29406ac700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29380c8fa0 0x7f29380c2f90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2930001330 tx=0x7f293000a640 comp rx=0 tx=0).stop 2023-11-28T11:10:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.438+0000 7f29406ac700 1 -- 172.21.15.156:0/1812074107 shutdown_connections 2023-11-28T11:10:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.438+0000 7f29406ac700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29380c8fa0 0x7f29380c2f90 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.438+0000 7f29406ac700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f291c0682f0 0x7f291c06a7a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.438+0000 7f29406ac700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2924006190 0x7f2938153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.438+0000 7f29406ac700 1 --2- 172.21.15.156:0/1812074107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29381deb00 0x7f29380c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.438+0000 7f29406ac700 1 -- 172.21.15.156:0/1812074107 >> 172.21.15.156:0/1812074107 conn(0x7f29380b12b0 msgr2=0x7f29380b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.438+0000 7f29406ac700 1 -- 172.21.15.156:0/1812074107 shutdown_connections 2023-11-28T11:10:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.438+0000 7f29406ac700 1 -- 172.21.15.156:0/1812074107 wait complete. 2023-11-28T11:10:56.444 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:10:56.452 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2023-11-28T11:10:54.559766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":77220,"kb_used_data":6756,"kb_used_omap":0,"kb_used_meta":70400,"kb_avail":754897500,"statfs":{"total":773094113280,"available":773015040000,"internally_reserved":0,"allocated":6918144,"data_stored":3670378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72089600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000650"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978200+0000","last_change":"2023-11-28T11:10:50.978200+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978200+0000","last_undegraded":"2023-11-28T11:10:50.978200+0000","last_fullsized":"2023-11-28T11:10:50.978200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:23.426647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.976341+0000","last_change":"2023-11-28T11:10:50.976341+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.976341+0000","last_undegraded":"2023-11-28T11:10:50.976341+0000","last_fullsized":"2023-11-28T11:10:50.976341+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:32.263075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.977231+0000","last_change":"2023-11-28T11:10:50.977231+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.977231+0000","last_undegraded":"2023-11-28T11:10:50.977231+0000","last_fullsized":"2023-11-28T11:10:50.977231+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:52.799489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.976599+0000","last_change":"2023-11-28T11:10:50.976599+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.976599+0000","last_undegraded":"2023-11-28T11:10:50.976599+0000","last_fullsized":"2023-11-28T11:10:50.976599+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:06.486410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.975467+0000","last_change":"2023-11-28T11:10:50.975467+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.975467+0000","last_undegraded":"2023-11-28T11:10:50.975467+0000","last_fullsized":"2023-11-28T11:10:50.975467+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:11.092714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.975443+0000","last_change":"2023-11-28T11:10:50.975443+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.975443+0000","last_undegraded":"2023-11-28T11:10:50.975443+0000","last_fullsized":"2023-11-28T11:10:50.975443+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:06.486410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.975443+0000","last_change":"2023-11-28T11:10:50.975443+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.975443+0000","last_undegraded":"2023-11-28T11:10:50.975443+0000","last_fullsized":"2023-11-28T11:10:50.975443+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:21.757386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978897+0000","last_change":"2023-11-28T11:10:50.978897+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978897+0000","last_undegraded":"2023-11-28T11:10:50.978897+0000","last_fullsized":"2023-11-28T11:10:50.978897+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:31:32.827101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.980516+0000","last_change":"2023-11-28T11:10:50.980516+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.980516+0000","last_undegraded":"2023-11-28T11:10:50.980516+0000","last_fullsized":"2023-11-28T11:10:50.980516+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:21:56.126127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.979461+0000","last_change":"2023-11-28T11:10:50.979461+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.979461+0000","last_undegraded":"2023-11-28T11:10:50.979461+0000","last_fullsized":"2023-11-28T11:10:50.979461+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:21.858555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.980439+0000","last_change":"2023-11-28T11:10:50.980439+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.980439+0000","last_undegraded":"2023-11-28T11:10:50.980439+0000","last_fullsized":"2023-11-28T11:10:50.980439+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:47.034924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978374+0000","last_change":"2023-11-28T11:10:50.978374+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978374+0000","last_undegraded":"2023-11-28T11:10:50.978374+0000","last_fullsized":"2023-11-28T11:10:50.978374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:41:07.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.980057+0000","last_change":"2023-11-28T11:10:50.980057+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.980057+0000","last_undegraded":"2023-11-28T11:10:50.980057+0000","last_fullsized":"2023-11-28T11:10:50.980057+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:27.337701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.981250+0000","last_change":"2023-11-28T11:10:50.981250+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.981250+0000","last_undegraded":"2023-11-28T11:10:50.981250+0000","last_fullsized":"2023-11-28T11:10:50.981250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:29.954884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.979512+0000","last_change":"2023-11-28T11:10:50.979512+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.979512+0000","last_undegraded":"2023-11-28T11:10:50.979512+0000","last_fullsized":"2023-11-28T11:10:50.979512+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:41:07.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.980082+0000","last_change":"2023-11-28T11:10:50.980082+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.980082+0000","last_undegraded":"2023-11-28T11:10:50.980082+0000","last_fullsized":"2023-11-28T11:10:50.980082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:38.465853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978466+0000","last_change":"2023-11-28T11:10:50.978466+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978466+0000","last_undegraded":"2023-11-28T11:10:50.978466+0000","last_fullsized":"2023-11-28T11:10:50.978466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:44.608724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978752+0000","last_change":"2023-11-28T11:10:50.978752+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978752+0000","last_undegraded":"2023-11-28T11:10:50.978752+0000","last_fullsized":"2023-11-28T11:10:50.978752+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:35.847896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.976239+0000","last_change":"2023-11-28T11:10:50.976239+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.976239+0000","last_undegraded":"2023-11-28T11:10:50.976239+0000","last_fullsized":"2023-11-28T11:10:50.976239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:56.382110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.977949+0000","last_change":"2023-11-28T11:10:50.977949+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.977949+0000","last_undegraded":"2023-11-28T11:10:50.977949+0000","last_fullsized":"2023-11-28T11:10:50.977949+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:11.021567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978612+0000","last_change":"2023-11-28T11:10:50.978612+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978612+0000","last_undegraded":"2023-11-28T11:10:50.978612+0000","last_fullsized":"2023-11-28T11:10:50.978612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:11.021567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.974281+0000","last_change":"2023-11-28T11:10:50.974281+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.974281+0000","last_undegraded":"2023-11-28T11:10:50.974281+0000","last_fullsized":"2023-11-28T11:10:50.974281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:05.195466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.974314+0000","last_change":"2023-11-28T11:10:50.974314+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.974314+0000","last_undegraded":"2023-11-28T11:10:50.974314+0000","last_fullsized":"2023-11-28T11:10:50.974314+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:35.988439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.977903+0000","last_change":"2023-11-28T11:10:50.977903+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.977903+0000","last_undegraded":"2023-11-28T11:10:50.977903+0000","last_fullsized":"2023-11-28T11:10:50.977903+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:37.815667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.979017+0000","last_change":"2023-11-28T11:10:50.979017+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.979017+0000","last_undegraded":"2023-11-28T11:10:50.979017+0000","last_fullsized":"2023-11-28T11:10:50.979017+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:00.060672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.027168+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:10:48.027168+0000","last_peered":"2023-11-28T11:10:48.027168+0000","last_clean":"2023-11-28T11:10:48.027168+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:10:48.027168+0000","last_undegraded":"2023-11-28T11:10:48.027168+0000","last_fullsized":"2023-11-28T11:10:48.027168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.976502+0000","last_change":"2023-11-28T11:10:50.976502+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.976502+0000","last_undegraded":"2023-11-28T11:10:50.976502+0000","last_fullsized":"2023-11-28T11:10:50.976502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:11:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:47.963710+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:10:47.963710+0000","last_peered":"2023-11-28T11:10:47.963710+0000","last_clean":"2023-11-28T11:10:47.963710+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:10:47.963710+0000","last_undegraded":"2023-11-28T11:10:47.963710+0000","last_fullsized":"2023-11-28T11:10:47.963710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978394+0000","last_change":"2023-11-28T11:10:50.978394+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978394+0000","last_undegraded":"2023-11-28T11:10:50.978394+0000","last_fullsized":"2023-11-28T11:10:50.978394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:38.465853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.026963+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:10:48.026963+0000","last_peered":"2023-11-28T11:10:48.026963+0000","last_clean":"2023-11-28T11:10:48.026963+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:10:48.026963+0000","last_undegraded":"2023-11-28T11:10:48.026963+0000","last_fullsized":"2023-11-28T11:10:48.026963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.974266+0000","last_change":"2023-11-28T11:10:50.974266+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.974266+0000","last_undegraded":"2023-11-28T11:10:50.974266+0000","last_fullsized":"2023-11-28T11:10:50.974266+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:05:42.312941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:47.963710+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:10:47.963710+0000","last_peered":"2023-11-28T11:10:47.963710+0000","last_clean":"2023-11-28T11:10:47.963710+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:10:47.963710+0000","last_undegraded":"2023-11-28T11:10:47.963710+0000","last_fullsized":"2023-11-28T11:10:47.963710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.026815+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:10:48.026815+0000","last_peered":"2023-11-28T11:10:48.026815+0000","last_clean":"2023-11-28T11:10:48.026815+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:10:48.026815+0000","last_undegraded":"2023-11-28T11:10:48.026815+0000","last_fullsized":"2023-11-28T11:10:48.026815+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.976217+0000","last_change":"2023-11-28T11:10:50.976217+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.976217+0000","last_undegraded":"2023-11-28T11:10:50.976217+0000","last_fullsized":"2023-11-28T11:10:50.976217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:35.988439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.026812+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:10:48.026812+0000","last_peered":"2023-11-28T11:10:48.026812+0000","last_clean":"2023-11-28T11:10:48.026812+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:10:48.026812+0000","last_undegraded":"2023-11-28T11:10:48.026812+0000","last_fullsized":"2023-11-28T11:10:48.026812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978847+0000","last_change":"2023-11-28T11:10:50.978847+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978847+0000","last_undegraded":"2023-11-28T11:10:50.978847+0000","last_fullsized":"2023-11-28T11:10:50.978847+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:37.815667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.026816+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:10:48.026816+0000","last_peered":"2023-11-28T11:10:48.026816+0000","last_clean":"2023-11-28T11:10:48.026816+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:10:48.026816+0000","last_undegraded":"2023-11-28T11:10:48.026816+0000","last_fullsized":"2023-11-28T11:10:48.026816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.979738+0000","last_change":"2023-11-28T11:10:50.979738+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.979738+0000","last_undegraded":"2023-11-28T11:10:50.979738+0000","last_fullsized":"2023-11-28T11:10:50.979738+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:23.426647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.027097+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:10:48.027097+0000","last_peered":"2023-11-28T11:10:48.027097+0000","last_clean":"2023-11-28T11:10:48.027097+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:10:48.027097+0000","last_undegraded":"2023-11-28T11:10:48.027097+0000","last_fullsized":"2023-11-28T11:10:48.027097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.974300+0000","last_change":"2023-11-28T11:10:50.974300+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.974300+0000","last_undegraded":"2023-11-28T11:10:50.974300+0000","last_fullsized":"2023-11-28T11:10:50.974300+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:56.382110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.026706+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:10:48.026706+0000","last_peered":"2023-11-28T11:10:48.026706+0000","last_clean":"2023-11-28T11:10:48.026706+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:10:48.026706+0000","last_undegraded":"2023-11-28T11:10:48.026706+0000","last_fullsized":"2023-11-28T11:10:48.026706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10636,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361204,"statfs":{"total":96636764160,"available":96625872896,"internally_reserved":0,"allocated":1314816,"data_stored":903887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92200000000000004,"5min":1.006,"15min":1.006},"last":0.54800000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.64600000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.67500000000000004,"15min":0.69999999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71099999999999997,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.747,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0349999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97199999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67200000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.33700000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.86699999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.35499999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92100000000000004,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.42899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.504},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.47299999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9160,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362680,"statfs":{"total":96636764160,"available":96627384320,"internally_reserved":0,"allocated":720896,"data_stored":313519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69499999999999995,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.70299999999999996,"15min":0.73399999999999999},"min":{"1min":0.36299999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76800000000000002,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.58199999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.81599999999999995,"15min":0.85999999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76500000000000001,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.77000000000000002,"15min":0.80500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79800000000000004,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.63500000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.78900000000000003,"15min":0.83599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67200000000000004,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.68899999999999995,"15min":0.71699999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68400000000000005,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.60499999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.755,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.65400000000000003,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65700000000000003,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.62}]},{"osd":4,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.70399999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.379,"5min":0.373,"15min":0.373},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.77500000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76900000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.71699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9288,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362552,"statfs":{"total":96636764160,"available":96627253248,"internally_reserved":0,"allocated":720896,"data_stored":313519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.59999999999999998,"15min":0.64600000000000002},"min":{"1min":0.221,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.504,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.377,"15min":0.38200000000000001},"min":{"1min":0.248,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.46100000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.378}]},{"osd":2,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.64200000000000002,"15min":0.68999999999999995},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56999999999999995,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51900000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.44600000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42899999999999999,"15min":0.435},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58899999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.375,"15min":0.375},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54200000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.3120000000000001,"15min":1.431},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79500000000000004,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.76100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78800000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":1.3440000000000001,"15min":1.462},"min":{"1min":0.47799999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93600000000000005,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.67900000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":1.3500000000000001,"15min":1.4670000000000001},"min":{"1min":0.498,"5min":0.439,"15min":0.439},"max":{"1min":0.95999999999999996,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.48599999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94399999999999995,"5min":1.018,"15min":1.018},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.44700000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10444,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361396,"statfs":{"total":96636764160,"available":96626069504,"internally_reserved":0,"allocated":1314816,"data_stored":903887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.48599999999999999,"5min":0.502,"15min":0.502},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.44400000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.246}]},{"osd":2,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53600000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.378,"15min":0.378},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.50900000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.29499999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47799999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.56699999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.309}]},{"osd":4,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58999999999999997,"15min":0.59299999999999997},"min":{"1min":0.38800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66700000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.46700000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.49399999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.434,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52900000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.47399999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.47499999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.62,"15min":0.62},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.81399999999999995,"5min":1.026,"15min":1.026},"last":0.436},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9352,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362488,"statfs":{"total":96636764160,"available":96627187712,"internally_reserved":0,"allocated":720896,"data_stored":313519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.32500000000000001,"15min":0.316},"min":{"1min":0.22,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.379},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.36299999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.34899999999999998,"15min":0.33500000000000002},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33700000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.36899999999999999},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.40999999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.61499999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.66000000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.622},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.68500000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9596,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362244,"statfs":{"total":96636764160,"available":96626937856,"internally_reserved":0,"allocated":708608,"data_stored":307349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.42999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.31900000000000001,"15min":0.317},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.502,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.42399999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.56000000000000005,"5min":1.365,"15min":1.365},"last":0.315}]},{"osd":2,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48499999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41599999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82999999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.45300000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81200000000000006,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85999999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.39500000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.48099999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57799999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9404,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362436,"statfs":{"total":96636764160,"available":96627134464,"internally_reserved":0,"allocated":708608,"data_stored":307349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69699999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.60099999999999998,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.9740000000000002,"5min":4.9740000000000002,"15min":4.9740000000000002},"last":0.41499999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":1.115,"15min":1.2070000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69699999999999995,"5min":12.263,"15min":12.263},"last":0.495}]},{"osd":2,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.437},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":1.236,"15min":1.236},"last":0.64600000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5600000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.51500000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":4.3710000000000004,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.59499999999999997,"15min":0.55000000000000004},"min":{"1min":0.40200000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":5.0220000000000002,"5min":5.0220000000000002,"15min":5.0220000000000002},"last":0.67000000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.72199999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.65600000000000003,"15min":0.64100000000000001},"min":{"1min":0.55700000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.57799999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9340,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362500,"statfs":{"total":96636764160,"available":96627200000,"internally_reserved":0,"allocated":708608,"data_stored":307349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.45700000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53100000000000003},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.504,"15min":0.505},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66300000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.438,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.432}]},{"osd":4,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.497,"15min":0.497},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72999999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.67500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72899999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.441,"5min":0.374,"15min":0.374},"max":{"1min":0.67700000000000005,"5min":0.753,"15min":0.753},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.747,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.59299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:10:56.458 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:10:56.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.613+0000 7f99a92d3700 1 Processor -- start 2023-11-28T11:10:56.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.613+0000 7f99a92d3700 1 -- start start 2023-11-28T11:10:56.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a92d3700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f99a415ca50 0x7f99a406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a92d3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f99a406b8c0 0x7f99a40cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a92d3700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a40cf5e0 con 0x7f99a40ccc40 2023-11-28T11:10:56.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a92d3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a406bd60 con 0x7f99a406b8c0 2023-11-28T11:10:56.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a92d3700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a40cd470 con 0x7f99a415ca50 2023-11-28T11:10:56.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a27fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f99a406b8c0 0x7f99a40cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a37fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f99a40ccc40 0x7f99a40cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50578/0 (socket says 172.21.15.156:50578) 2023-11-28T11:10:56.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a37fe700 1 -- 172.21.15.156:0/415096867 learned_addr learned my addr 172.21.15.156:0/415096867 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:56.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a2ffd700 1 --2- 172.21.15.156:0/415096867 >> v2:172.21.15.186:3300/0 conn(0x7f99a415ca50 0x7f99a406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a27fc700 1 -- 172.21.15.156:0/415096867 >> v2:172.21.15.186:3300/0 conn(0x7f99a415ca50 msgr2=0x7f99a406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a27fc700 1 --2- 172.21.15.156:0/415096867 >> v2:172.21.15.186:3300/0 conn(0x7f99a415ca50 0x7f99a406b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a27fc700 1 -- 172.21.15.156:0/415096867 >> v1:172.21.15.156:6789/0 conn(0x7f99a40ccc40 legacy=0x7f99a40cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a27fc700 1 -- 172.21.15.156:0/415096867 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a40cecb0 con 0x7f99a406b8c0 2023-11-28T11:10:56.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a1ffb700 1 -- 172.21.15.156:0/415096867 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441816014 0 0) 0x7f99a40cf5e0 con 0x7f99a40ccc40 2023-11-28T11:10:56.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a27fc700 1 --2- 172.21.15.156:0/415096867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99a406b8c0 0x7f99a40cc700 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f99a4071240 tx=0x7f9998009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e36ff9540edc0e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a1ffb700 1 -- 172.21.15.156:0/415096867 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998011830 con 0x7f99a406b8c0 2023-11-28T11:10:56.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a1ffb700 1 -- 172.21.15.156:0/415096867 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9998011990 con 0x7f99a406b8c0 2023-11-28T11:10:56.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.614+0000 7f99a1ffb700 1 -- 172.21.15.156:0/415096867 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998011d20 con 0x7f99a406b8c0 2023-11-28T11:10:56.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.615+0000 7f99a92d3700 1 -- 172.21.15.156:0/415096867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99a406b8c0 msgr2=0x7f99a40cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.615+0000 7f99a92d3700 1 --2- 172.21.15.156:0/415096867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99a406b8c0 0x7f99a40cc700 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f99a4071240 tx=0x7f9998009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:56.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.615+0000 7f99a92d3700 1 -- 172.21.15.156:0/415096867 shutdown_connections 2023-11-28T11:10:56.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.615+0000 7f99a92d3700 1 --2- 172.21.15.156:0/415096867 >> v2:172.21.15.186:3300/0 conn(0x7f99a415ca50 0x7f99a406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.615+0000 7f99a92d3700 1 --2- 172.21.15.156:0/415096867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99a406b8c0 0x7f99a40cc700 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.615+0000 7f99a92d3700 1 -- 172.21.15.156:0/415096867 >> 172.21.15.156:0/415096867 conn(0x7f99a409a230 msgr2=0x7f99a408afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:56.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.615+0000 7f99a92d3700 1 -- 172.21.15.156:0/415096867 shutdown_connections 2023-11-28T11:10:56.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.615+0000 7f99a92d3700 1 -- 172.21.15.156:0/415096867 wait complete. 2023-11-28T11:10:56.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.616+0000 7f99a92d3700 1 Processor -- start 2023-11-28T11:10:56.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.616+0000 7f99a92d3700 1 -- start start 2023-11-28T11:10:56.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a92d3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99a406b8c0 0x7f99a41544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a92d3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99a40ccc40 0x7f99a41592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a92d3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a415ca50 0x7f99a4159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a92d3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40c6af0 con 0x7f99a406b8c0 2023-11-28T11:10:56.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a92d3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40c6dd0 con 0x7f99a415ca50 2023-11-28T11:10:56.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a92d3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40c70b0 con 0x7f99a40ccc40 2023-11-28T11:10:56.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99a406b8c0 0x7f99a41544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99a406b8c0 0x7f99a41544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35702/0 (socket says 172.21.15.156:35702) 2023-11-28T11:10:56.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a2ffd700 1 -- 172.21.15.156:0/806660650 learned_addr learned my addr 172.21.15.156:0/806660650 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:56.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a37fe700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a415ca50 0x7f99a4159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a2ffd700 1 -- 172.21.15.156:0/806660650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99a40ccc40 msgr2=0x7f99a41592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a27fc700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99a40ccc40 0x7f99a41592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a2ffd700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99a40ccc40 0x7f99a41592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a2ffd700 1 -- 172.21.15.156:0/806660650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a415ca50 msgr2=0x7f99a4159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a2ffd700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a415ca50 0x7f99a4159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a2ffd700 1 -- 172.21.15.156:0/806660650 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99980094a0 con 0x7f99a406b8c0 2023-11-28T11:10:56.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.617+0000 7f99a37fe700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a415ca50 0x7f99a4159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:56.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.618+0000 7f99a2ffd700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99a406b8c0 0x7f99a41544c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f999400d800 tx=0x7f999400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.618+0000 7f998b7fe700 1 -- 172.21.15.156:0/806660650 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99940125f0 con 0x7f99a406b8c0 2023-11-28T11:10:56.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.618+0000 7f998b7fe700 1 -- 172.21.15.156:0/806660650 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9994012750 con 0x7f99a406b8c0 2023-11-28T11:10:56.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.618+0000 7f99a92d3700 1 -- 172.21.15.156:0/806660650 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a40cf270 con 0x7f99a406b8c0 2023-11-28T11:10:56.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.618+0000 7f998b7fe700 1 -- 172.21.15.156:0/806660650 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994012b00 con 0x7f99a406b8c0 2023-11-28T11:10:56.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.618+0000 7f99a92d3700 1 -- 172.21.15.156:0/806660650 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a40cef90 con 0x7f99a406b8c0 2023-11-28T11:10:56.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.619+0000 7f99a92d3700 1 -- 172.21.15.156:0/806660650 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a40cd0a0 con 0x7f99a406b8c0 2023-11-28T11:10:56.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.619+0000 7f998b7fe700 1 -- 172.21.15.156:0/806660650 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9994012c60 con 0x7f99a406b8c0 2023-11-28T11:10:56.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.620+0000 7f998b7fe700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f99900682f0 0x7f999006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.620+0000 7f998b7fe700 1 -- 172.21.15.156:0/806660650 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9994086270 con 0x7f99a406b8c0 2023-11-28T11:10:56.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.620+0000 7f99a27fc700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f99900682f0 0x7f999006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.620+0000 7f99a27fc700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f99900682f0 0x7f999006a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f99a41f5bf0 tx=0x7f9998008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.624+0000 7f998b7fe700 1 -- 172.21.15.156:0/806660650 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f99940529e0 con 0x7f99a406b8c0 2023-11-28T11:10:56.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.802+0000 7f99a92d3700 1 -- 172.21.15.156:0/806660650 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99a40745c0 con 0x7f99900682f0 2023-11-28T11:10:56.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.810+0000 7f998b7fe700 1 -- 172.21.15.156:0/806660650 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308201 (secure 0 0 0) 0x7f99a40745c0 con 0x7f99900682f0 2023-11-28T11:10:56.815 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:56.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.814+0000 7f99a92d3700 1 -- 172.21.15.156:0/806660650 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f99900682f0 msgr2=0x7f999006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.814+0000 7f99a92d3700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f99900682f0 0x7f999006a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f99a41f5bf0 tx=0x7f9998008030 comp rx=0 tx=0).stop 2023-11-28T11:10:56.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.814+0000 7f99a92d3700 1 -- 172.21.15.156:0/806660650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99a406b8c0 msgr2=0x7f99a41544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.814+0000 7f99a92d3700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99a406b8c0 0x7f99a41544c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f999400d800 tx=0x7f999400dbc0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.814+0000 7f99a92d3700 1 -- 172.21.15.156:0/806660650 shutdown_connections 2023-11-28T11:10:56.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.814+0000 7f99a92d3700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99a415ca50 0x7f99a4159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.814+0000 7f99a92d3700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99a406b8c0 0x7f99a41544c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.814+0000 7f99a92d3700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f99900682f0 0x7f999006a7a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.814+0000 7f99a92d3700 1 --2- 172.21.15.156:0/806660650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99a40ccc40 0x7f99a41592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.814+0000 7f99a92d3700 1 -- 172.21.15.156:0/806660650 >> 172.21.15.156:0/806660650 conn(0x7f99a409a230 msgr2=0x7f99a40cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.814+0000 7f99a92d3700 1 -- 172.21.15.156:0/806660650 shutdown_connections 2023-11-28T11:10:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.814+0000 7f99a92d3700 1 -- 172.21.15.156:0/806660650 wait complete. 2023-11-28T11:10:56.820 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:10:56.827 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2023-11-28T11:10:56.560054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590562,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82420,"kb_used_data":7028,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":754892300,"statfs":{"total":773094113280,"available":773009715200,"internally_reserved":0,"allocated":7196672,"data_stored":3807466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":194,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.588339"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848556+0000","last_change":"2023-11-28T11:10:52.399647+0000","last_active":"2023-11-28T11:10:53.848556+0000","last_peered":"2023-11-28T11:10:53.848556+0000","last_clean":"2023-11-28T11:10:53.848556+0000","last_became_active":"2023-11-28T11:10:52.399461+0000","last_became_peered":"2023-11-28T11:10:52.399461+0000","last_unstale":"2023-11-28T11:10:53.848556+0000","last_undegraded":"2023-11-28T11:10:53.848556+0000","last_fullsized":"2023-11-28T11:10:53.848556+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:36.282716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848853+0000","last_change":"2023-11-28T11:10:52.390854+0000","last_active":"2023-11-28T11:10:53.848853+0000","last_peered":"2023-11-28T11:10:53.848853+0000","last_clean":"2023-11-28T11:10:53.848853+0000","last_became_active":"2023-11-28T11:10:52.390541+0000","last_became_peered":"2023-11-28T11:10:52.390541+0000","last_unstale":"2023-11-28T11:10:53.848853+0000","last_undegraded":"2023-11-28T11:10:53.848853+0000","last_fullsized":"2023-11-28T11:10:53.848853+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:06.203049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377143+0000","last_change":"2023-11-28T11:10:52.394457+0000","last_active":"2023-11-28T11:10:53.377143+0000","last_peered":"2023-11-28T11:10:53.377143+0000","last_clean":"2023-11-28T11:10:53.377143+0000","last_became_active":"2023-11-28T11:10:52.394170+0000","last_became_peered":"2023-11-28T11:10:52.394170+0000","last_unstale":"2023-11-28T11:10:53.377143+0000","last_undegraded":"2023-11-28T11:10:53.377143+0000","last_fullsized":"2023-11-28T11:10:53.377143+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:12.029150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848451+0000","last_change":"2023-11-28T11:10:52.398045+0000","last_active":"2023-11-28T11:10:53.848451+0000","last_peered":"2023-11-28T11:10:53.848451+0000","last_clean":"2023-11-28T11:10:53.848451+0000","last_became_active":"2023-11-28T11:10:52.397753+0000","last_became_peered":"2023-11-28T11:10:52.397753+0000","last_unstale":"2023-11-28T11:10:53.848451+0000","last_undegraded":"2023-11-28T11:10:53.848451+0000","last_fullsized":"2023-11-28T11:10:53.848451+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:35.944815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847689+0000","last_change":"2023-11-28T11:10:52.397026+0000","last_active":"2023-11-28T11:10:53.847689+0000","last_peered":"2023-11-28T11:10:53.847689+0000","last_clean":"2023-11-28T11:10:53.847689+0000","last_became_active":"2023-11-28T11:10:52.396742+0000","last_became_peered":"2023-11-28T11:10:52.396742+0000","last_unstale":"2023-11-28T11:10:53.847689+0000","last_undegraded":"2023-11-28T11:10:53.847689+0000","last_fullsized":"2023-11-28T11:10:53.847689+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:00.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848795+0000","last_change":"2023-11-28T11:10:52.401622+0000","last_active":"2023-11-28T11:10:53.848795+0000","last_peered":"2023-11-28T11:10:53.848795+0000","last_clean":"2023-11-28T11:10:53.848795+0000","last_became_active":"2023-11-28T11:10:52.401310+0000","last_became_peered":"2023-11-28T11:10:52.401310+0000","last_unstale":"2023-11-28T11:10:53.848795+0000","last_undegraded":"2023-11-28T11:10:53.848795+0000","last_fullsized":"2023-11-28T11:10:53.848795+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:23.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848431+0000","last_change":"2023-11-28T11:10:52.398550+0000","last_active":"2023-11-28T11:10:53.848431+0000","last_peered":"2023-11-28T11:10:53.848431+0000","last_clean":"2023-11-28T11:10:53.848431+0000","last_became_active":"2023-11-28T11:10:52.398277+0000","last_became_peered":"2023-11-28T11:10:52.398277+0000","last_unstale":"2023-11-28T11:10:53.848431+0000","last_undegraded":"2023-11-28T11:10:53.848431+0000","last_fullsized":"2023-11-28T11:10:53.848431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:53.807072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848691+0000","last_change":"2023-11-28T11:10:52.396844+0000","last_active":"2023-11-28T11:10:53.848691+0000","last_peered":"2023-11-28T11:10:53.848691+0000","last_clean":"2023-11-28T11:10:53.848691+0000","last_became_active":"2023-11-28T11:10:52.393214+0000","last_became_peered":"2023-11-28T11:10:52.393214+0000","last_unstale":"2023-11-28T11:10:53.848691+0000","last_undegraded":"2023-11-28T11:10:53.848691+0000","last_fullsized":"2023-11-28T11:10:53.848691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:12.100298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848063+0000","last_change":"2023-11-28T11:10:52.397847+0000","last_active":"2023-11-28T11:10:53.848063+0000","last_peered":"2023-11-28T11:10:53.848063+0000","last_clean":"2023-11-28T11:10:53.848063+0000","last_became_active":"2023-11-28T11:10:52.397580+0000","last_became_peered":"2023-11-28T11:10:52.397580+0000","last_unstale":"2023-11-28T11:10:53.848063+0000","last_undegraded":"2023-11-28T11:10:53.848063+0000","last_fullsized":"2023-11-28T11:10:53.848063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:11:29.529381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848017+0000","last_change":"2023-11-28T11:10:52.397544+0000","last_active":"2023-11-28T11:10:53.848017+0000","last_peered":"2023-11-28T11:10:53.848017+0000","last_clean":"2023-11-28T11:10:53.848017+0000","last_became_active":"2023-11-28T11:10:52.397232+0000","last_became_peered":"2023-11-28T11:10:52.397232+0000","last_unstale":"2023-11-28T11:10:53.848017+0000","last_undegraded":"2023-11-28T11:10:53.848017+0000","last_fullsized":"2023-11-28T11:10:53.848017+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:33.270658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847532+0000","last_change":"2023-11-28T11:10:52.388564+0000","last_active":"2023-11-28T11:10:53.847532+0000","last_peered":"2023-11-28T11:10:53.847532+0000","last_clean":"2023-11-28T11:10:53.847532+0000","last_became_active":"2023-11-28T11:10:52.388307+0000","last_became_peered":"2023-11-28T11:10:52.388307+0000","last_unstale":"2023-11-28T11:10:53.847532+0000","last_undegraded":"2023-11-28T11:10:53.847532+0000","last_fullsized":"2023-11-28T11:10:53.847532+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:54.802666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848635+0000","last_change":"2023-11-28T11:10:52.391388+0000","last_active":"2023-11-28T11:10:53.848635+0000","last_peered":"2023-11-28T11:10:53.848635+0000","last_clean":"2023-11-28T11:10:53.848635+0000","last_became_active":"2023-11-28T11:10:52.391076+0000","last_became_peered":"2023-11-28T11:10:52.391076+0000","last_unstale":"2023-11-28T11:10:53.848635+0000","last_undegraded":"2023-11-28T11:10:53.848635+0000","last_fullsized":"2023-11-28T11:10:53.848635+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:05:43.320524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376913+0000","last_change":"2023-11-28T11:10:52.380958+0000","last_active":"2023-11-28T11:10:53.376913+0000","last_peered":"2023-11-28T11:10:53.376913+0000","last_clean":"2023-11-28T11:10:53.376913+0000","last_became_active":"2023-11-28T11:10:52.380688+0000","last_became_peered":"2023-11-28T11:10:52.380688+0000","last_unstale":"2023-11-28T11:10:53.376913+0000","last_undegraded":"2023-11-28T11:10:53.376913+0000","last_fullsized":"2023-11-28T11:10:53.376913+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:22.866138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375117+0000","last_change":"2023-11-28T11:10:52.387375+0000","last_active":"2023-11-28T11:10:53.375117+0000","last_peered":"2023-11-28T11:10:53.375117+0000","last_clean":"2023-11-28T11:10:53.375117+0000","last_became_active":"2023-11-28T11:10:52.387088+0000","last_became_peered":"2023-11-28T11:10:52.387088+0000","last_unstale":"2023-11-28T11:10:53.375117+0000","last_undegraded":"2023-11-28T11:10:53.375117+0000","last_fullsized":"2023-11-28T11:10:53.375117+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:54.810083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375344+0000","last_change":"2023-11-28T11:10:52.398710+0000","last_active":"2023-11-28T11:10:53.375344+0000","last_peered":"2023-11-28T11:10:53.375344+0000","last_clean":"2023-11-28T11:10:53.375344+0000","last_became_active":"2023-11-28T11:10:52.398388+0000","last_became_peered":"2023-11-28T11:10:52.398388+0000","last_unstale":"2023-11-28T11:10:53.375344+0000","last_undegraded":"2023-11-28T11:10:53.375344+0000","last_fullsized":"2023-11-28T11:10:53.375344+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:09.520497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.374950+0000","last_change":"2023-11-28T11:10:52.398644+0000","last_active":"2023-11-28T11:10:53.374950+0000","last_peered":"2023-11-28T11:10:53.374950+0000","last_clean":"2023-11-28T11:10:53.374950+0000","last_became_active":"2023-11-28T11:10:52.398342+0000","last_became_peered":"2023-11-28T11:10:52.398342+0000","last_unstale":"2023-11-28T11:10:53.374950+0000","last_undegraded":"2023-11-28T11:10:53.374950+0000","last_fullsized":"2023-11-28T11:10:53.374950+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:02.775343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848270+0000","last_change":"2023-11-28T11:10:52.400926+0000","last_active":"2023-11-28T11:10:53.848270+0000","last_peered":"2023-11-28T11:10:53.848270+0000","last_clean":"2023-11-28T11:10:53.848270+0000","last_became_active":"2023-11-28T11:10:52.400667+0000","last_became_peered":"2023-11-28T11:10:52.400667+0000","last_unstale":"2023-11-28T11:10:53.848270+0000","last_undegraded":"2023-11-28T11:10:53.848270+0000","last_fullsized":"2023-11-28T11:10:53.848270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:06.203049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848749+0000","last_change":"2023-11-28T11:10:52.390919+0000","last_active":"2023-11-28T11:10:53.848749+0000","last_peered":"2023-11-28T11:10:53.848749+0000","last_clean":"2023-11-28T11:10:53.848749+0000","last_became_active":"2023-11-28T11:10:52.390435+0000","last_became_peered":"2023-11-28T11:10:52.390435+0000","last_unstale":"2023-11-28T11:10:53.848749+0000","last_undegraded":"2023-11-28T11:10:53.848749+0000","last_fullsized":"2023-11-28T11:10:53.848749+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:48.226111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848092+0000","last_change":"2023-11-28T11:10:52.397318+0000","last_active":"2023-11-28T11:10:53.848092+0000","last_peered":"2023-11-28T11:10:53.848092+0000","last_clean":"2023-11-28T11:10:53.848092+0000","last_became_active":"2023-11-28T11:10:52.397039+0000","last_became_peered":"2023-11-28T11:10:52.397039+0000","last_unstale":"2023-11-28T11:10:53.848092+0000","last_undegraded":"2023-11-28T11:10:53.848092+0000","last_fullsized":"2023-11-28T11:10:53.848092+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:54.810083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847782+0000","last_change":"2023-11-28T11:10:52.394969+0000","last_active":"2023-11-28T11:10:53.847782+0000","last_peered":"2023-11-28T11:10:53.847782+0000","last_clean":"2023-11-28T11:10:53.847782+0000","last_became_active":"2023-11-28T11:10:52.394620+0000","last_became_peered":"2023-11-28T11:10:52.394620+0000","last_unstale":"2023-11-28T11:10:53.847782+0000","last_undegraded":"2023-11-28T11:10:53.847782+0000","last_fullsized":"2023-11-28T11:10:53.847782+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:28.345284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847322+0000","last_change":"2023-11-28T11:10:52.382159+0000","last_active":"2023-11-28T11:10:53.847322+0000","last_peered":"2023-11-28T11:10:53.847322+0000","last_clean":"2023-11-28T11:10:53.847322+0000","last_became_active":"2023-11-28T11:10:52.381757+0000","last_became_peered":"2023-11-28T11:10:52.381757+0000","last_unstale":"2023-11-28T11:10:53.847322+0000","last_undegraded":"2023-11-28T11:10:53.847322+0000","last_fullsized":"2023-11-28T11:10:53.847322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:21.448143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377057+0000","last_change":"2023-11-28T11:10:52.389365+0000","last_active":"2023-11-28T11:10:53.377057+0000","last_peered":"2023-11-28T11:10:53.377057+0000","last_clean":"2023-11-28T11:10:53.377057+0000","last_became_active":"2023-11-28T11:10:52.388931+0000","last_became_peered":"2023-11-28T11:10:52.388931+0000","last_unstale":"2023-11-28T11:10:53.377057+0000","last_undegraded":"2023-11-28T11:10:53.377057+0000","last_fullsized":"2023-11-28T11:10:53.377057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:30.962468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978200+0000","last_change":"2023-11-28T11:10:50.978200+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978200+0000","last_undegraded":"2023-11-28T11:10:50.978200+0000","last_fullsized":"2023-11-28T11:10:50.978200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:23.426647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848809+0000","last_change":"2023-11-28T11:10:52.390732+0000","last_active":"2023-11-28T11:10:53.848809+0000","last_peered":"2023-11-28T11:10:53.848809+0000","last_clean":"2023-11-28T11:10:53.848809+0000","last_became_active":"2023-11-28T11:10:52.390111+0000","last_became_peered":"2023-11-28T11:10:52.390111+0000","last_unstale":"2023-11-28T11:10:53.848809+0000","last_undegraded":"2023-11-28T11:10:53.848809+0000","last_fullsized":"2023-11-28T11:10:53.848809+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:18.651068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.378205+0000","last_change":"2023-11-28T11:10:51.995120+0000","last_active":"2023-11-28T11:10:52.378205+0000","last_peered":"2023-11-28T11:10:52.378205+0000","last_clean":"2023-11-28T11:10:52.378205+0000","last_became_active":"2023-11-28T11:10:51.994836+0000","last_became_peered":"2023-11-28T11:10:51.994836+0000","last_unstale":"2023-11-28T11:10:52.378205+0000","last_undegraded":"2023-11-28T11:10:52.378205+0000","last_fullsized":"2023-11-28T11:10:52.378205+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:32.263075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848457+0000","last_change":"2023-11-28T11:10:52.385151+0000","last_active":"2023-11-28T11:10:53.848457+0000","last_peered":"2023-11-28T11:10:53.848457+0000","last_clean":"2023-11-28T11:10:53.848457+0000","last_became_active":"2023-11-28T11:10:52.381849+0000","last_became_peered":"2023-11-28T11:10:52.381849+0000","last_unstale":"2023-11-28T11:10:53.848457+0000","last_undegraded":"2023-11-28T11:10:53.848457+0000","last_fullsized":"2023-11-28T11:10:53.848457+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:02.775343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.384672+0000","last_change":"2023-11-28T11:10:52.000037+0000","last_active":"2023-11-28T11:10:52.384672+0000","last_peered":"2023-11-28T11:10:52.384672+0000","last_clean":"2023-11-28T11:10:52.384672+0000","last_became_active":"2023-11-28T11:10:51.999788+0000","last_became_peered":"2023-11-28T11:10:51.999788+0000","last_unstale":"2023-11-28T11:10:52.384672+0000","last_undegraded":"2023-11-28T11:10:52.384672+0000","last_fullsized":"2023-11-28T11:10:52.384672+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:52.799489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848659+0000","last_change":"2023-11-28T11:10:52.399189+0000","last_active":"2023-11-28T11:10:53.848659+0000","last_peered":"2023-11-28T11:10:53.848659+0000","last_clean":"2023-11-28T11:10:53.848659+0000","last_became_active":"2023-11-28T11:10:52.398894+0000","last_became_peered":"2023-11-28T11:10:52.398894+0000","last_unstale":"2023-11-28T11:10:53.848659+0000","last_undegraded":"2023-11-28T11:10:53.848659+0000","last_fullsized":"2023-11-28T11:10:53.848659+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:09.926232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.381856+0000","last_change":"2023-11-28T11:10:51.990812+0000","last_active":"2023-11-28T11:10:52.381856+0000","last_peered":"2023-11-28T11:10:52.381856+0000","last_clean":"2023-11-28T11:10:52.381856+0000","last_became_active":"2023-11-28T11:10:51.990539+0000","last_became_peered":"2023-11-28T11:10:51.990539+0000","last_unstale":"2023-11-28T11:10:52.381856+0000","last_undegraded":"2023-11-28T11:10:52.381856+0000","last_fullsized":"2023-11-28T11:10:52.381856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:06.486410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847570+0000","last_change":"2023-11-28T11:10:52.381091+0000","last_active":"2023-11-28T11:10:53.847570+0000","last_peered":"2023-11-28T11:10:53.847570+0000","last_clean":"2023-11-28T11:10:53.847570+0000","last_became_active":"2023-11-28T11:10:52.380839+0000","last_became_peered":"2023-11-28T11:10:52.380839+0000","last_unstale":"2023-11-28T11:10:53.847570+0000","last_undegraded":"2023-11-28T11:10:53.847570+0000","last_fullsized":"2023-11-28T11:10:53.847570+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:29:30.173766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.375838+0000","last_change":"2023-11-28T11:10:51.985814+0000","last_active":"2023-11-28T11:10:52.375838+0000","last_peered":"2023-11-28T11:10:52.375838+0000","last_clean":"2023-11-28T11:10:52.375838+0000","last_became_active":"2023-11-28T11:10:51.985536+0000","last_became_peered":"2023-11-28T11:10:51.985536+0000","last_unstale":"2023-11-28T11:10:52.375838+0000","last_undegraded":"2023-11-28T11:10:52.375838+0000","last_fullsized":"2023-11-28T11:10:52.375838+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:11.092714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377014+0000","last_change":"2023-11-28T11:10:51.984690+0000","last_active":"2023-11-28T11:10:52.377014+0000","last_peered":"2023-11-28T11:10:52.377014+0000","last_clean":"2023-11-28T11:10:52.377014+0000","last_became_active":"2023-11-28T11:10:51.984385+0000","last_became_peered":"2023-11-28T11:10:51.984385+0000","last_unstale":"2023-11-28T11:10:52.377014+0000","last_undegraded":"2023-11-28T11:10:52.377014+0000","last_fullsized":"2023-11-28T11:10:52.377014+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:06.486410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376774+0000","last_change":"2023-11-28T11:10:52.390623+0000","last_active":"2023-11-28T11:10:53.376774+0000","last_peered":"2023-11-28T11:10:53.376774+0000","last_clean":"2023-11-28T11:10:53.376774+0000","last_became_active":"2023-11-28T11:10:52.390344+0000","last_became_peered":"2023-11-28T11:10:52.390344+0000","last_unstale":"2023-11-28T11:10:53.376774+0000","last_undegraded":"2023-11-28T11:10:53.376774+0000","last_fullsized":"2023-11-28T11:10:53.376774+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:45.616308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.370871+0000","last_change":"2023-11-28T11:10:51.991786+0000","last_active":"2023-11-28T11:10:52.370871+0000","last_peered":"2023-11-28T11:10:52.370871+0000","last_clean":"2023-11-28T11:10:52.370871+0000","last_became_active":"2023-11-28T11:10:51.991493+0000","last_became_peered":"2023-11-28T11:10:51.991493+0000","last_unstale":"2023-11-28T11:10:52.370871+0000","last_undegraded":"2023-11-28T11:10:52.370871+0000","last_fullsized":"2023-11-28T11:10:52.370871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:21.757386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375324+0000","last_change":"2023-11-28T11:10:52.398474+0000","last_active":"2023-11-28T11:10:53.375324+0000","last_peered":"2023-11-28T11:10:53.375324+0000","last_clean":"2023-11-28T11:10:53.375324+0000","last_became_active":"2023-11-28T11:10:52.398155+0000","last_became_peered":"2023-11-28T11:10:52.398155+0000","last_unstale":"2023-11-28T11:10:53.375324+0000","last_undegraded":"2023-11-28T11:10:53.375324+0000","last_fullsized":"2023-11-28T11:10:53.375324+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:23.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.383141+0000","last_change":"2023-11-28T11:10:51.995022+0000","last_active":"2023-11-28T11:10:52.383141+0000","last_peered":"2023-11-28T11:10:52.383141+0000","last_clean":"2023-11-28T11:10:52.383141+0000","last_became_active":"2023-11-28T11:10:51.994731+0000","last_became_peered":"2023-11-28T11:10:51.994731+0000","last_unstale":"2023-11-28T11:10:52.383141+0000","last_undegraded":"2023-11-28T11:10:52.383141+0000","last_fullsized":"2023-11-28T11:10:52.383141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:31:32.827101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.395098+0000","last_change":"2023-11-28T11:10:51.998518+0000","last_active":"2023-11-28T11:10:52.395098+0000","last_peered":"2023-11-28T11:10:52.395098+0000","last_clean":"2023-11-28T11:10:52.395098+0000","last_became_active":"2023-11-28T11:10:51.998086+0000","last_became_peered":"2023-11-28T11:10:51.998086+0000","last_unstale":"2023-11-28T11:10:52.395098+0000","last_undegraded":"2023-11-28T11:10:52.395098+0000","last_fullsized":"2023-11-28T11:10:52.395098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:21:56.126127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375245+0000","last_change":"2023-11-28T11:10:52.383474+0000","last_active":"2023-11-28T11:10:53.375245+0000","last_peered":"2023-11-28T11:10:53.375245+0000","last_clean":"2023-11-28T11:10:53.375245+0000","last_became_active":"2023-11-28T11:10:52.383211+0000","last_became_peered":"2023-11-28T11:10:52.383211+0000","last_unstale":"2023-11-28T11:10:53.375245+0000","last_undegraded":"2023-11-28T11:10:53.375245+0000","last_fullsized":"2023-11-28T11:10:53.375245+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:48.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.384146+0000","last_change":"2023-11-28T11:10:51.997551+0000","last_active":"2023-11-28T11:10:52.384146+0000","last_peered":"2023-11-28T11:10:52.384146+0000","last_clean":"2023-11-28T11:10:52.384146+0000","last_became_active":"2023-11-28T11:10:51.997278+0000","last_became_peered":"2023-11-28T11:10:51.997278+0000","last_unstale":"2023-11-28T11:10:52.384146+0000","last_undegraded":"2023-11-28T11:10:52.384146+0000","last_fullsized":"2023-11-28T11:10:52.384146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:21.858555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377081+0000","last_change":"2023-11-28T11:10:52.393694+0000","last_active":"2023-11-28T11:10:53.377081+0000","last_peered":"2023-11-28T11:10:53.377081+0000","last_clean":"2023-11-28T11:10:53.377081+0000","last_became_active":"2023-11-28T11:10:52.393435+0000","last_became_peered":"2023-11-28T11:10:52.393435+0000","last_unstale":"2023-11-28T11:10:53.377081+0000","last_undegraded":"2023-11-28T11:10:53.377081+0000","last_fullsized":"2023-11-28T11:10:53.377081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:05:25.141120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"62'3","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.399587+0000","last_change":"2023-11-28T11:10:51.997311+0000","last_active":"2023-11-28T11:10:52.399587+0000","last_peered":"2023-11-28T11:10:52.399587+0000","last_clean":"2023-11-28T11:10:52.399587+0000","last_became_active":"2023-11-28T11:10:51.997038+0000","last_became_peered":"2023-11-28T11:10:51.997038+0000","last_unstale":"2023-11-28T11:10:52.399587+0000","last_undegraded":"2023-11-28T11:10:52.399587+0000","last_fullsized":"2023-11-28T11:10:52.399587+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:47.034924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848569+0000","last_change":"2023-11-28T11:10:52.393378+0000","last_active":"2023-11-28T11:10:53.848569+0000","last_peered":"2023-11-28T11:10:53.848569+0000","last_clean":"2023-11-28T11:10:53.848569+0000","last_became_active":"2023-11-28T11:10:52.393089+0000","last_became_peered":"2023-11-28T11:10:52.393089+0000","last_unstale":"2023-11-28T11:10:53.848569+0000","last_undegraded":"2023-11-28T11:10:53.848569+0000","last_fullsized":"2023-11-28T11:10:53.848569+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:22.764970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978374+0000","last_change":"2023-11-28T11:10:50.978374+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978374+0000","last_undegraded":"2023-11-28T11:10:50.978374+0000","last_fullsized":"2023-11-28T11:10:50.978374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:41:07.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847717+0000","last_change":"2023-11-28T11:10:52.402741+0000","last_active":"2023-11-28T11:10:53.847717+0000","last_peered":"2023-11-28T11:10:53.847717+0000","last_clean":"2023-11-28T11:10:53.847717+0000","last_became_active":"2023-11-28T11:10:52.402537+0000","last_became_peered":"2023-11-28T11:10:52.402537+0000","last_unstale":"2023-11-28T11:10:53.847717+0000","last_undegraded":"2023-11-28T11:10:53.847717+0000","last_fullsized":"2023-11-28T11:10:53.847717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:48.042507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.403563+0000","last_change":"2023-11-28T11:10:51.999231+0000","last_active":"2023-11-28T11:10:52.403563+0000","last_peered":"2023-11-28T11:10:52.403563+0000","last_clean":"2023-11-28T11:10:52.403563+0000","last_became_active":"2023-11-28T11:10:51.998951+0000","last_became_peered":"2023-11-28T11:10:51.998951+0000","last_unstale":"2023-11-28T11:10:52.403563+0000","last_undegraded":"2023-11-28T11:10:52.403563+0000","last_fullsized":"2023-11-28T11:10:52.403563+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:27.337701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.981250+0000","last_change":"2023-11-28T11:10:50.981250+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.981250+0000","last_undegraded":"2023-11-28T11:10:50.981250+0000","last_fullsized":"2023-11-28T11:10:50.981250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:29.954884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377049+0000","last_change":"2023-11-28T11:10:52.383141+0000","last_active":"2023-11-28T11:10:53.377049+0000","last_peered":"2023-11-28T11:10:53.377049+0000","last_clean":"2023-11-28T11:10:53.377049+0000","last_became_active":"2023-11-28T11:10:52.382460+0000","last_became_peered":"2023-11-28T11:10:52.382460+0000","last_unstale":"2023-11-28T11:10:53.377049+0000","last_undegraded":"2023-11-28T11:10:53.377049+0000","last_fullsized":"2023-11-28T11:10:53.377049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:24.111335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.393434+0000","last_change":"2023-11-28T11:10:51.999127+0000","last_active":"2023-11-28T11:10:52.393434+0000","last_peered":"2023-11-28T11:10:52.393434+0000","last_clean":"2023-11-28T11:10:52.393434+0000","last_became_active":"2023-11-28T11:10:51.998846+0000","last_became_peered":"2023-11-28T11:10:51.998846+0000","last_unstale":"2023-11-28T11:10:52.393434+0000","last_undegraded":"2023-11-28T11:10:52.393434+0000","last_fullsized":"2023-11-28T11:10:52.393434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:41:07.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"62'8","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.402319+0000","last_change":"2023-11-28T11:10:51.996381+0000","last_active":"2023-11-28T11:10:52.402319+0000","last_peered":"2023-11-28T11:10:52.402319+0000","last_clean":"2023-11-28T11:10:52.402319+0000","last_became_active":"2023-11-28T11:10:51.996095+0000","last_became_peered":"2023-11-28T11:10:51.996095+0000","last_unstale":"2023-11-28T11:10:52.402319+0000","last_undegraded":"2023-11-28T11:10:52.402319+0000","last_fullsized":"2023-11-28T11:10:52.402319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:38.465853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847836+0000","last_change":"2023-11-28T11:10:52.394438+0000","last_active":"2023-11-28T11:10:53.847836+0000","last_peered":"2023-11-28T11:10:53.847836+0000","last_clean":"2023-11-28T11:10:53.847836+0000","last_became_active":"2023-11-28T11:10:52.394191+0000","last_became_peered":"2023-11-28T11:10:52.394191+0000","last_unstale":"2023-11-28T11:10:53.847836+0000","last_undegraded":"2023-11-28T11:10:53.847836+0000","last_fullsized":"2023-11-28T11:10:53.847836+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:35.306404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978466+0000","last_change":"2023-11-28T11:10:50.978466+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978466+0000","last_undegraded":"2023-11-28T11:10:50.978466+0000","last_fullsized":"2023-11-28T11:10:50.978466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:44.608724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848232+0000","last_change":"2023-11-28T11:10:52.384083+0000","last_active":"2023-11-28T11:10:53.848232+0000","last_peered":"2023-11-28T11:10:53.848232+0000","last_clean":"2023-11-28T11:10:53.848232+0000","last_became_active":"2023-11-28T11:10:52.383213+0000","last_became_peered":"2023-11-28T11:10:52.383213+0000","last_unstale":"2023-11-28T11:10:53.848232+0000","last_undegraded":"2023-11-28T11:10:53.848232+0000","last_fullsized":"2023-11-28T11:10:53.848232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:00.199905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978752+0000","last_change":"2023-11-28T11:10:50.978752+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978752+0000","last_undegraded":"2023-11-28T11:10:50.978752+0000","last_fullsized":"2023-11-28T11:10:50.978752+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:35.847896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.373540+0000","last_change":"2023-11-28T11:10:52.000024+0000","last_active":"2023-11-28T11:10:52.373540+0000","last_peered":"2023-11-28T11:10:52.373540+0000","last_clean":"2023-11-28T11:10:52.373540+0000","last_became_active":"2023-11-28T11:10:51.999765+0000","last_became_peered":"2023-11-28T11:10:51.999765+0000","last_unstale":"2023-11-28T11:10:52.373540+0000","last_undegraded":"2023-11-28T11:10:52.373540+0000","last_fullsized":"2023-11-28T11:10:52.373540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:56.382110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376866+0000","last_change":"2023-11-28T11:10:52.388835+0000","last_active":"2023-11-28T11:10:53.376866+0000","last_peered":"2023-11-28T11:10:53.376866+0000","last_clean":"2023-11-28T11:10:53.376866+0000","last_became_active":"2023-11-28T11:10:52.388560+0000","last_became_peered":"2023-11-28T11:10:52.388560+0000","last_unstale":"2023-11-28T11:10:53.376866+0000","last_undegraded":"2023-11-28T11:10:53.376866+0000","last_fullsized":"2023-11-28T11:10:53.376866+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:03:08.782678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.977949+0000","last_change":"2023-11-28T11:10:50.977949+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.977949+0000","last_undegraded":"2023-11-28T11:10:50.977949+0000","last_fullsized":"2023-11-28T11:10:50.977949+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:11.021567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376607+0000","last_change":"2023-11-28T11:10:52.387164+0000","last_active":"2023-11-28T11:10:53.376607+0000","last_peered":"2023-11-28T11:10:53.376607+0000","last_clean":"2023-11-28T11:10:53.376607+0000","last_became_active":"2023-11-28T11:10:52.386857+0000","last_became_peered":"2023-11-28T11:10:52.386857+0000","last_unstale":"2023-11-28T11:10:53.376607+0000","last_undegraded":"2023-11-28T11:10:53.376607+0000","last_fullsized":"2023-11-28T11:10:53.376607+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:15:03.899294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.389892+0000","last_change":"2023-11-28T11:10:51.999035+0000","last_active":"2023-11-28T11:10:52.389892+0000","last_peered":"2023-11-28T11:10:52.389892+0000","last_clean":"2023-11-28T11:10:52.389892+0000","last_became_active":"2023-11-28T11:10:51.998758+0000","last_became_peered":"2023-11-28T11:10:51.998758+0000","last_unstale":"2023-11-28T11:10:52.389892+0000","last_undegraded":"2023-11-28T11:10:52.389892+0000","last_fullsized":"2023-11-28T11:10:52.389892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:11.021567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848172+0000","last_change":"2023-11-28T11:10:52.401133+0000","last_active":"2023-11-28T11:10:53.848172+0000","last_peered":"2023-11-28T11:10:53.848172+0000","last_clean":"2023-11-28T11:10:53.848172+0000","last_became_active":"2023-11-28T11:10:52.400869+0000","last_became_peered":"2023-11-28T11:10:52.400869+0000","last_unstale":"2023-11-28T11:10:53.848172+0000","last_undegraded":"2023-11-28T11:10:53.848172+0000","last_fullsized":"2023-11-28T11:10:53.848172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:38.996786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.371070+0000","last_change":"2023-11-28T11:10:51.989318+0000","last_active":"2023-11-28T11:10:52.371070+0000","last_peered":"2023-11-28T11:10:52.371070+0000","last_clean":"2023-11-28T11:10:52.371070+0000","last_became_active":"2023-11-28T11:10:51.988981+0000","last_became_peered":"2023-11-28T11:10:51.988981+0000","last_unstale":"2023-11-28T11:10:52.371070+0000","last_undegraded":"2023-11-28T11:10:52.371070+0000","last_fullsized":"2023-11-28T11:10:52.371070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:05.195466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.370973+0000","last_change":"2023-11-28T11:10:51.997542+0000","last_active":"2023-11-28T11:10:52.370973+0000","last_peered":"2023-11-28T11:10:52.370973+0000","last_clean":"2023-11-28T11:10:52.370973+0000","last_became_active":"2023-11-28T11:10:51.997247+0000","last_became_peered":"2023-11-28T11:10:51.997247+0000","last_unstale":"2023-11-28T11:10:52.370973+0000","last_undegraded":"2023-11-28T11:10:52.370973+0000","last_fullsized":"2023-11-28T11:10:52.370973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:35.988439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847922+0000","last_change":"2023-11-28T11:10:52.388406+0000","last_active":"2023-11-28T11:10:53.847922+0000","last_peered":"2023-11-28T11:10:53.847922+0000","last_clean":"2023-11-28T11:10:53.847922+0000","last_became_active":"2023-11-28T11:10:52.388066+0000","last_became_peered":"2023-11-28T11:10:52.388066+0000","last_unstale":"2023-11-28T11:10:53.847922+0000","last_undegraded":"2023-11-28T11:10:53.847922+0000","last_fullsized":"2023-11-28T11:10:53.847922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:43.460660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.977903+0000","last_change":"2023-11-28T11:10:50.977903+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.977903+0000","last_undegraded":"2023-11-28T11:10:50.977903+0000","last_fullsized":"2023-11-28T11:10:50.977903+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:37.815667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848272+0000","last_change":"2023-11-28T11:10:52.398149+0000","last_active":"2023-11-28T11:10:53.848272+0000","last_peered":"2023-11-28T11:10:53.848272+0000","last_clean":"2023-11-28T11:10:53.848272+0000","last_became_active":"2023-11-28T11:10:52.397875+0000","last_became_peered":"2023-11-28T11:10:52.397875+0000","last_unstale":"2023-11-28T11:10:53.848272+0000","last_undegraded":"2023-11-28T11:10:53.848272+0000","last_fullsized":"2023-11-28T11:10:53.848272+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:00.423890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.979017+0000","last_change":"2023-11-28T11:10:50.979017+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.979017+0000","last_undegraded":"2023-11-28T11:10:50.979017+0000","last_fullsized":"2023-11-28T11:10:50.979017+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:00.060672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.027168+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:10:48.027168+0000","last_peered":"2023-11-28T11:10:48.027168+0000","last_clean":"2023-11-28T11:10:48.027168+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:10:48.027168+0000","last_undegraded":"2023-11-28T11:10:48.027168+0000","last_fullsized":"2023-11-28T11:10:48.027168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377047+0000","last_change":"2023-11-28T11:10:52.387511+0000","last_active":"2023-11-28T11:10:53.377047+0000","last_peered":"2023-11-28T11:10:53.377047+0000","last_clean":"2023-11-28T11:10:53.377047+0000","last_became_active":"2023-11-28T11:10:52.387143+0000","last_became_peered":"2023-11-28T11:10:52.387143+0000","last_unstale":"2023-11-28T11:10:53.377047+0000","last_undegraded":"2023-11-28T11:10:53.377047+0000","last_fullsized":"2023-11-28T11:10:53.377047+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:01.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.373304+0000","last_change":"2023-11-28T11:10:51.995232+0000","last_active":"2023-11-28T11:10:52.373304+0000","last_peered":"2023-11-28T11:10:52.373304+0000","last_clean":"2023-11-28T11:10:52.373304+0000","last_became_active":"2023-11-28T11:10:51.994941+0000","last_became_peered":"2023-11-28T11:10:51.994941+0000","last_unstale":"2023-11-28T11:10:52.373304+0000","last_undegraded":"2023-11-28T11:10:52.373304+0000","last_fullsized":"2023-11-28T11:10:52.373304+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:11:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:47.963710+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:10:47.963710+0000","last_peered":"2023-11-28T11:10:47.963710+0000","last_clean":"2023-11-28T11:10:47.963710+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:10:47.963710+0000","last_undegraded":"2023-11-28T11:10:47.963710+0000","last_fullsized":"2023-11-28T11:10:47.963710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978394+0000","last_change":"2023-11-28T11:10:50.978394+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978394+0000","last_undegraded":"2023-11-28T11:10:50.978394+0000","last_fullsized":"2023-11-28T11:10:50.978394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:38.465853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377553+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:10:52.377553+0000","last_peered":"2023-11-28T11:10:52.377553+0000","last_clean":"2023-11-28T11:10:52.377553+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:10:52.377553+0000","last_undegraded":"2023-11-28T11:10:52.377553+0000","last_fullsized":"2023-11-28T11:10:52.377553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377169+0000","last_change":"2023-11-28T11:10:52.387614+0000","last_active":"2023-11-28T11:10:53.377169+0000","last_peered":"2023-11-28T11:10:53.377169+0000","last_clean":"2023-11-28T11:10:53.377169+0000","last_became_active":"2023-11-28T11:10:52.387338+0000","last_became_peered":"2023-11-28T11:10:52.387338+0000","last_unstale":"2023-11-28T11:10:53.377169+0000","last_undegraded":"2023-11-28T11:10:53.377169+0000","last_fullsized":"2023-11-28T11:10:53.377169+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:36.855479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.370866+0000","last_change":"2023-11-28T11:10:51.985858+0000","last_active":"2023-11-28T11:10:52.370866+0000","last_peered":"2023-11-28T11:10:52.370866+0000","last_clean":"2023-11-28T11:10:52.370866+0000","last_became_active":"2023-11-28T11:10:51.985623+0000","last_became_peered":"2023-11-28T11:10:51.985623+0000","last_unstale":"2023-11-28T11:10:52.370866+0000","last_undegraded":"2023-11-28T11:10:52.370866+0000","last_fullsized":"2023-11-28T11:10:52.370866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:05:42.312941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:47.963710+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:10:47.963710+0000","last_peered":"2023-11-28T11:10:47.963710+0000","last_clean":"2023-11-28T11:10:47.963710+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:10:47.963710+0000","last_undegraded":"2023-11-28T11:10:47.963710+0000","last_fullsized":"2023-11-28T11:10:47.963710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":204,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.389688+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:10:52.389688+0000","last_peered":"2023-11-28T11:10:52.389688+0000","last_clean":"2023-11-28T11:10:52.389688+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:10:52.389688+0000","last_undegraded":"2023-11-28T11:10:52.389688+0000","last_fullsized":"2023-11-28T11:10:52.389688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847400+0000","last_change":"2023-11-28T11:10:52.381996+0000","last_active":"2023-11-28T11:10:53.847400+0000","last_peered":"2023-11-28T11:10:53.847400+0000","last_clean":"2023-11-28T11:10:53.847400+0000","last_became_active":"2023-11-28T11:10:52.381358+0000","last_became_peered":"2023-11-28T11:10:52.381358+0000","last_unstale":"2023-11-28T11:10:53.847400+0000","last_undegraded":"2023-11-28T11:10:53.847400+0000","last_fullsized":"2023-11-28T11:10:53.847400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:48.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.386941+0000","last_change":"2023-11-28T11:10:51.996342+0000","last_active":"2023-11-28T11:10:52.386941+0000","last_peered":"2023-11-28T11:10:52.386941+0000","last_clean":"2023-11-28T11:10:52.386941+0000","last_became_active":"2023-11-28T11:10:51.995997+0000","last_became_peered":"2023-11-28T11:10:51.995997+0000","last_unstale":"2023-11-28T11:10:52.386941+0000","last_undegraded":"2023-11-28T11:10:52.386941+0000","last_fullsized":"2023-11-28T11:10:52.386941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:35.988439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.389449+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:10:52.389449+0000","last_peered":"2023-11-28T11:10:52.389449+0000","last_clean":"2023-11-28T11:10:52.389449+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:10:52.389449+0000","last_undegraded":"2023-11-28T11:10:52.389449+0000","last_fullsized":"2023-11-28T11:10:52.389449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376952+0000","last_change":"2023-11-28T11:10:52.397798+0000","last_active":"2023-11-28T11:10:53.376952+0000","last_peered":"2023-11-28T11:10:53.376952+0000","last_clean":"2023-11-28T11:10:53.376952+0000","last_became_active":"2023-11-28T11:10:52.397532+0000","last_became_peered":"2023-11-28T11:10:52.397532+0000","last_unstale":"2023-11-28T11:10:53.376952+0000","last_undegraded":"2023-11-28T11:10:53.376952+0000","last_fullsized":"2023-11-28T11:10:53.376952+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:38.823250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.398098+0000","last_change":"2023-11-28T11:10:52.000553+0000","last_active":"2023-11-28T11:10:52.398098+0000","last_peered":"2023-11-28T11:10:52.398098+0000","last_clean":"2023-11-28T11:10:52.398098+0000","last_became_active":"2023-11-28T11:10:52.000223+0000","last_became_peered":"2023-11-28T11:10:52.000223+0000","last_unstale":"2023-11-28T11:10:52.398098+0000","last_undegraded":"2023-11-28T11:10:52.398098+0000","last_fullsized":"2023-11-28T11:10:52.398098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:37.815667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.391001+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:10:52.391001+0000","last_peered":"2023-11-28T11:10:52.391001+0000","last_clean":"2023-11-28T11:10:52.391001+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:10:52.391001+0000","last_undegraded":"2023-11-28T11:10:52.391001+0000","last_fullsized":"2023-11-28T11:10:52.391001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848738+0000","last_change":"2023-11-28T11:10:52.388031+0000","last_active":"2023-11-28T11:10:53.848738+0000","last_peered":"2023-11-28T11:10:53.848738+0000","last_clean":"2023-11-28T11:10:53.848738+0000","last_became_active":"2023-11-28T11:10:52.387685+0000","last_became_peered":"2023-11-28T11:10:52.387685+0000","last_unstale":"2023-11-28T11:10:53.848738+0000","last_undegraded":"2023-11-28T11:10:53.848738+0000","last_fullsized":"2023-11-28T11:10:53.848738+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:19:57.624643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.399510+0000","last_change":"2023-11-28T11:10:51.998733+0000","last_active":"2023-11-28T11:10:52.399510+0000","last_peered":"2023-11-28T11:10:52.399510+0000","last_clean":"2023-11-28T11:10:52.399510+0000","last_became_active":"2023-11-28T11:10:51.998123+0000","last_became_peered":"2023-11-28T11:10:51.998123+0000","last_unstale":"2023-11-28T11:10:52.399510+0000","last_undegraded":"2023-11-28T11:10:52.399510+0000","last_fullsized":"2023-11-28T11:10:52.399510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:23.426647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.027097+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:10:48.027097+0000","last_peered":"2023-11-28T11:10:48.027097+0000","last_clean":"2023-11-28T11:10:48.027097+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:10:48.027097+0000","last_undegraded":"2023-11-28T11:10:48.027097+0000","last_fullsized":"2023-11-28T11:10:48.027097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376982+0000","last_change":"2023-11-28T11:10:52.389489+0000","last_active":"2023-11-28T11:10:53.376982+0000","last_peered":"2023-11-28T11:10:53.376982+0000","last_clean":"2023-11-28T11:10:53.376982+0000","last_became_active":"2023-11-28T11:10:52.387877+0000","last_became_peered":"2023-11-28T11:10:52.387877+0000","last_unstale":"2023-11-28T11:10:53.376982+0000","last_undegraded":"2023-11-28T11:10:53.376982+0000","last_fullsized":"2023-11-28T11:10:53.376982+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:08:26.596352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.371082+0000","last_change":"2023-11-28T11:10:51.992409+0000","last_active":"2023-11-28T11:10:52.371082+0000","last_peered":"2023-11-28T11:10:52.371082+0000","last_clean":"2023-11-28T11:10:52.371082+0000","last_became_active":"2023-11-28T11:10:51.992031+0000","last_became_peered":"2023-11-28T11:10:51.992031+0000","last_unstale":"2023-11-28T11:10:52.371082+0000","last_undegraded":"2023-11-28T11:10:52.371082+0000","last_fullsized":"2023-11-28T11:10:52.371082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:56.382110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.380028+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:10:52.380028+0000","last_peered":"2023-11-28T11:10:52.380028+0000","last_clean":"2023-11-28T11:10:52.380028+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:10:52.380028+0000","last_undegraded":"2023-11-28T11:10:52.380028+0000","last_fullsized":"2023-11-28T11:10:52.380028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":194,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10880,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360960,"statfs":{"total":96636764160,"available":96625623040,"internally_reserved":0,"allocated":1368064,"data_stored":925048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92200000000000004,"5min":1.006,"15min":1.006},"last":0.27800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.67500000000000004,"15min":0.69999999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71099999999999997,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.747,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.35199999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0349999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.432},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97199999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.39700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.33700000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.86699999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.38}]},{"osd":5,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92100000000000004,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.37},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.443}]},{"osd":6,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.47299999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.52000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9328,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362512,"statfs":{"total":96636764160,"available":96627212288,"internally_reserved":0,"allocated":761856,"data_stored":333954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69499999999999995,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.70299999999999996,"15min":0.73399999999999999},"min":{"1min":0.36299999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76800000000000002,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.69099999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.81599999999999995,"15min":0.85999999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76500000000000001,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.77000000000000002,"15min":0.80500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79800000000000004,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.78900000000000003,"15min":0.83599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67200000000000004,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.68899999999999995,"15min":0.71699999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68400000000000005,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.76800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.755,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.65400000000000003,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65700000000000003,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.755}]},{"osd":4,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.85099999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.379,"5min":0.373,"15min":0.373},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.88800000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76900000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.86099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9288,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362552,"statfs":{"total":96636764160,"available":96627253248,"internally_reserved":0,"allocated":720896,"data_stored":313519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.59999999999999998,"15min":0.64600000000000002},"min":{"1min":0.221,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.504,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.377,"15min":0.38200000000000001},"min":{"1min":0.248,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.46100000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.378}]},{"osd":2,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.64200000000000002,"15min":0.68999999999999995},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56999999999999995,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51900000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.44600000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42899999999999999,"15min":0.435},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58899999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.375,"15min":0.375},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54200000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.3120000000000001,"15min":1.431},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79500000000000004,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.76100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78800000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":1.3440000000000001,"15min":1.462},"min":{"1min":0.47799999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93600000000000005,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.67900000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":1.3500000000000001,"15min":1.4670000000000001},"min":{"1min":0.498,"5min":0.439,"15min":0.439},"max":{"1min":0.95999999999999996,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.48599999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94399999999999995,"5min":1.018,"15min":1.018},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.44700000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14768,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94357072,"statfs":{"total":96636764160,"available":96621641728,"internally_reserved":0,"allocated":1351680,"data_stored":924288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.48599999999999999,"5min":0.502,"15min":0.502},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.44400000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.26100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53600000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.378,"15min":0.378},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.50900000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.29699999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47799999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.56699999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.311}]},{"osd":4,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58999999999999997,"15min":0.59299999999999997},"min":{"1min":0.38800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66700000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.46700000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.35899999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.434,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.39600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.432},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.47399999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.62,"15min":0.62},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.81399999999999995,"5min":1.026,"15min":1.026},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.41699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9452,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362388,"statfs":{"total":96636764160,"available":96627085312,"internally_reserved":0,"allocated":757760,"data_stored":333920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.32500000000000001,"15min":0.316},"min":{"1min":0.22,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.33900000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.34899999999999998,"15min":0.33500000000000002},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33700000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.29299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.312},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.36899999999999999},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.35299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.437}]},{"osd":6,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.503},{"interface":"front","average":{"1min":0.63,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.44600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9772,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362068,"statfs":{"total":96636764160,"available":96626757632,"internally_reserved":0,"allocated":757760,"data_stored":333920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.42999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.31900000000000001,"15min":0.317},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.502,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.35399999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.56000000000000005,"5min":1.365,"15min":1.365},"last":0.33200000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48499999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.31}]},{"osd":4,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.495},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42899999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82999999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46000000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81200000000000006,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85999999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.48099999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9592,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362248,"statfs":{"total":96636764160,"available":96626941952,"internally_reserved":0,"allocated":770048,"data_stored":335468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69699999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.60099999999999998,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.9740000000000002,"5min":4.9740000000000002,"15min":4.9740000000000002},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":1.115,"15min":1.2070000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69699999999999995,"5min":12.263,"15min":12.263},"last":0.44700000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.505},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":1.236,"15min":1.236},"last":0.41399999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5600000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":4.3710000000000004,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.59499999999999997,"15min":0.55000000000000004},"min":{"1min":0.40200000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":5.0220000000000002,"5min":5.0220000000000002,"15min":5.0220000000000002},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.52000000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.65600000000000003,"15min":0.64100000000000001},"min":{"1min":0.55700000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.60199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9340,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362500,"statfs":{"total":96636764160,"available":96627200000,"internally_reserved":0,"allocated":708608,"data_stored":307349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.45700000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53100000000000003},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.504,"15min":0.505},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66300000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.438,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.432}]},{"osd":4,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.497,"15min":0.497},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72999999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.67500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72899999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.441,"5min":0.374,"15min":0.374},"max":{"1min":0.67700000000000005,"5min":0.753,"15min":0.753},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.747,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.59299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:10:56.832 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:10:56.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.986+0000 7f9a22982700 1 Processor -- start 2023-11-28T11:10:56.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.986+0000 7f9a22982700 1 -- start start 2023-11-28T11:10:56.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.986+0000 7f9a22982700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a1c071040 0x7f9a1c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a22982700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9a1c160540 0x7f9a1c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a22982700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c071950 con 0x7f9a1c06dee0 2023-11-28T11:10:56.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a22982700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c15d060 con 0x7f9a1c160540 2023-11-28T11:10:56.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a22982700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c15d340 con 0x7f9a1c071040 2023-11-28T11:10:56.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a20f1f700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9a1c160540 0x7f9a1c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a1bfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a1c071040 0x7f9a1c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a1b7fe700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f9a1c06dee0 0x7f9a1c06e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:60948/0 (socket says 172.21.15.156:60948) 2023-11-28T11:10:56.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a1b7fe700 1 -- 172.21.15.156:0/4292115259 learned_addr learned my addr 172.21.15.156:0/4292115259 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:56.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a20f1f700 1 -- 172.21.15.156:0/4292115259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a1c071040 msgr2=0x7f9a1c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a20f1f700 1 --2- 172.21.15.156:0/4292115259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a1c071040 0x7f9a1c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a20f1f700 1 -- 172.21.15.156:0/4292115259 >> v1:172.21.15.186:6789/0 conn(0x7f9a1c06dee0 legacy=0x7f9a1c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a20f1f700 1 -- 172.21.15.156:0/4292115259 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c15d620 con 0x7f9a1c160540 2023-11-28T11:10:56.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a1affd700 1 -- 172.21.15.156:0/4292115259 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862440965 0 0) 0x7f9a1c071950 con 0x7f9a1c06dee0 2023-11-28T11:10:56.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a1bfff700 1 --2- 172.21.15.156:0/4292115259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a1c071040 0x7f9a1c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:56.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.987+0000 7f9a20f1f700 1 --2- 172.21.15.156:0/4292115259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a1c160540 0x7f9a1c1609c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009d90 tx=0x7f9a1400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ee2d97b62de67b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.988+0000 7f9a1affd700 1 -- 172.21.15.156:0/4292115259 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400e070 con 0x7f9a1c160540 2023-11-28T11:10:56.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.988+0000 7f9a1affd700 1 -- 172.21.15.156:0/4292115259 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1400bb20 con 0x7f9a1c160540 2023-11-28T11:10:56.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.988+0000 7f9a1affd700 1 -- 172.21.15.156:0/4292115259 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14012430 con 0x7f9a1c160540 2023-11-28T11:10:56.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.988+0000 7f9a22982700 1 -- 172.21.15.156:0/4292115259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a1c160540 msgr2=0x7f9a1c1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.988+0000 7f9a22982700 1 --2- 172.21.15.156:0/4292115259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a1c160540 0x7f9a1c1609c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009d90 tx=0x7f9a1400b330 comp rx=0 tx=0).stop 2023-11-28T11:10:56.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.988+0000 7f9a22982700 1 -- 172.21.15.156:0/4292115259 shutdown_connections 2023-11-28T11:10:56.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.988+0000 7f9a22982700 1 --2- 172.21.15.156:0/4292115259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a1c160540 0x7f9a1c1609c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.988+0000 7f9a22982700 1 --2- 172.21.15.156:0/4292115259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a1c071040 0x7f9a1c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.988+0000 7f9a22982700 1 -- 172.21.15.156:0/4292115259 >> 172.21.15.156:0/4292115259 conn(0x7f9a1c0d0000 msgr2=0x7f9a1c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:56.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.988+0000 7f9a22982700 1 -- 172.21.15.156:0/4292115259 shutdown_connections 2023-11-28T11:10:56.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.988+0000 7f9a22982700 1 -- 172.21.15.156:0/4292115259 wait complete. 2023-11-28T11:10:56.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.989+0000 7f9a22982700 1 Processor -- start 2023-11-28T11:10:56.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.990+0000 7f9a22982700 1 -- start start 2023-11-28T11:10:56.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.990+0000 7f9a22982700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1c06dee0 0x7f9a1c0b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.990+0000 7f9a22982700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a1c071040 0x7f9a1c0ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.990+0000 7f9a22982700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a1c160540 0x7f9a1c0ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.990+0000 7f9a22982700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c1f3570 con 0x7f9a1c06dee0 2023-11-28T11:10:56.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.990+0000 7f9a22982700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c0c69b0 con 0x7f9a1c071040 2023-11-28T11:10:56.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.990+0000 7f9a22982700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c1f63b0 con 0x7f9a1c160540 2023-11-28T11:10:56.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.990+0000 7f9a1bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1c06dee0 0x7f9a1c0b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.990+0000 7f9a1bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1c06dee0 0x7f9a1c0b5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35714/0 (socket says 172.21.15.156:35714) 2023-11-28T11:10:56.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.990+0000 7f9a1bfff700 1 -- 172.21.15.156:0/4097608228 learned_addr learned my addr 172.21.15.156:0/4097608228 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:56.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a1b7fe700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a1c071040 0x7f9a1c0ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a20f1f700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a1c160540 0x7f9a1c0ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a1bfff700 1 -- 172.21.15.156:0/4097608228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a1c160540 msgr2=0x7f9a1c0ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a1bfff700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a1c160540 0x7f9a1c0ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a1bfff700 1 -- 172.21.15.156:0/4097608228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a1c071040 msgr2=0x7f9a1c0ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a1bfff700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a1c071040 0x7f9a1c0ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a1bfff700 1 -- 172.21.15.156:0/4097608228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0094a0 con 0x7f9a1c06dee0 2023-11-28T11:10:56.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a1b7fe700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a1c071040 0x7f9a1c0ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:56.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a20f1f700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a1c160540 0x7f9a1c0ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:56.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a1bfff700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1c06dee0 0x7f9a1c0b5920 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00ebe0 tx=0x7f9a0c009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a18ff9700 1 -- 172.21.15.156:0/4097608228 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00c8b0 con 0x7f9a1c06dee0 2023-11-28T11:10:56.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a18ff9700 1 -- 172.21.15.156:0/4097608228 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c00ca10 con 0x7f9a1c06dee0 2023-11-28T11:10:56.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a22982700 1 -- 172.21.15.156:0/4097608228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a14009a70 con 0x7f9a1c06dee0 2023-11-28T11:10:56.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a18ff9700 1 -- 172.21.15.156:0/4097608228 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00cda0 con 0x7f9a1c06dee0 2023-11-28T11:10:56.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.991+0000 7f9a22982700 1 -- 172.21.15.156:0/4097608228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c15ccd0 con 0x7f9a1c06dee0 2023-11-28T11:10:56.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.992+0000 7f9a22982700 1 -- 172.21.15.156:0/4097608228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c15d900 con 0x7f9a1c06dee0 2023-11-28T11:10:56.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.992+0000 7f9a18ff9700 1 -- 172.21.15.156:0/4097608228 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9a0c019030 con 0x7f9a1c06dee0 2023-11-28T11:10:56.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.993+0000 7f9a18ff9700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9a080682f0 0x7f9a0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.993+0000 7f9a18ff9700 1 -- 172.21.15.156:0/4097608228 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a0c085ca0 con 0x7f9a1c06dee0 2023-11-28T11:10:56.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.993+0000 7f9a1b7fe700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9a080682f0 0x7f9a0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.993+0000 7f9a1b7fe700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9a080682f0 0x7f9a0806a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9a10007fd0 tx=0x7f9a10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:56.997+0000 7f9a18ff9700 1 -- 172.21.15.156:0/4097608228 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9a0c08a070 con 0x7f9a1c06dee0 2023-11-28T11:10:57.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.174+0000 7f9a22982700 1 -- 172.21.15.156:0/4097608228 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a1c1f6690 con 0x7f9a080682f0 2023-11-28T11:10:57.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.182+0000 7f9a18ff9700 1 -- 172.21.15.156:0/4097608228 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308201 (secure 0 0 0) 0x7f9a1c1f6690 con 0x7f9a080682f0 2023-11-28T11:10:57.187 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:57.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.185+0000 7f9a22982700 1 -- 172.21.15.156:0/4097608228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9a080682f0 msgr2=0x7f9a0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.185+0000 7f9a22982700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9a080682f0 0x7f9a0806a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9a10007fd0 tx=0x7f9a10009040 comp rx=0 tx=0).stop 2023-11-28T11:10:57.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.185+0000 7f9a22982700 1 -- 172.21.15.156:0/4097608228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1c06dee0 msgr2=0x7f9a1c0b5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.185+0000 7f9a22982700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1c06dee0 0x7f9a1c0b5920 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00ebe0 tx=0x7f9a0c009e20 comp rx=0 tx=0).stop 2023-11-28T11:10:57.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.185+0000 7f9a22982700 1 -- 172.21.15.156:0/4097608228 shutdown_connections 2023-11-28T11:10:57.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.185+0000 7f9a22982700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9a080682f0 0x7f9a0806a7a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.185+0000 7f9a22982700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a1c160540 0x7f9a1c0ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.185+0000 7f9a22982700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1c06dee0 0x7f9a1c0b5920 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.185+0000 7f9a22982700 1 --2- 172.21.15.156:0/4097608228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a1c071040 0x7f9a1c0ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.185+0000 7f9a22982700 1 -- 172.21.15.156:0/4097608228 >> 172.21.15.156:0/4097608228 conn(0x7f9a1c0d0000 msgr2=0x7f9a1c0cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:57.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.186+0000 7f9a22982700 1 -- 172.21.15.156:0/4097608228 shutdown_connections 2023-11-28T11:10:57.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.186+0000 7f9a22982700 1 -- 172.21.15.156:0/4097608228 wait complete. 2023-11-28T11:10:57.191 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:10:57.199 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2023-11-28T11:10:56.560054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590562,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82420,"kb_used_data":7028,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":754892300,"statfs":{"total":773094113280,"available":773009715200,"internally_reserved":0,"allocated":7196672,"data_stored":3807466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":194,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.588339"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848556+0000","last_change":"2023-11-28T11:10:52.399647+0000","last_active":"2023-11-28T11:10:53.848556+0000","last_peered":"2023-11-28T11:10:53.848556+0000","last_clean":"2023-11-28T11:10:53.848556+0000","last_became_active":"2023-11-28T11:10:52.399461+0000","last_became_peered":"2023-11-28T11:10:52.399461+0000","last_unstale":"2023-11-28T11:10:53.848556+0000","last_undegraded":"2023-11-28T11:10:53.848556+0000","last_fullsized":"2023-11-28T11:10:53.848556+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:36.282716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848853+0000","last_change":"2023-11-28T11:10:52.390854+0000","last_active":"2023-11-28T11:10:53.848853+0000","last_peered":"2023-11-28T11:10:53.848853+0000","last_clean":"2023-11-28T11:10:53.848853+0000","last_became_active":"2023-11-28T11:10:52.390541+0000","last_became_peered":"2023-11-28T11:10:52.390541+0000","last_unstale":"2023-11-28T11:10:53.848853+0000","last_undegraded":"2023-11-28T11:10:53.848853+0000","last_fullsized":"2023-11-28T11:10:53.848853+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:06.203049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377143+0000","last_change":"2023-11-28T11:10:52.394457+0000","last_active":"2023-11-28T11:10:53.377143+0000","last_peered":"2023-11-28T11:10:53.377143+0000","last_clean":"2023-11-28T11:10:53.377143+0000","last_became_active":"2023-11-28T11:10:52.394170+0000","last_became_peered":"2023-11-28T11:10:52.394170+0000","last_unstale":"2023-11-28T11:10:53.377143+0000","last_undegraded":"2023-11-28T11:10:53.377143+0000","last_fullsized":"2023-11-28T11:10:53.377143+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:12.029150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848451+0000","last_change":"2023-11-28T11:10:52.398045+0000","last_active":"2023-11-28T11:10:53.848451+0000","last_peered":"2023-11-28T11:10:53.848451+0000","last_clean":"2023-11-28T11:10:53.848451+0000","last_became_active":"2023-11-28T11:10:52.397753+0000","last_became_peered":"2023-11-28T11:10:52.397753+0000","last_unstale":"2023-11-28T11:10:53.848451+0000","last_undegraded":"2023-11-28T11:10:53.848451+0000","last_fullsized":"2023-11-28T11:10:53.848451+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:35.944815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847689+0000","last_change":"2023-11-28T11:10:52.397026+0000","last_active":"2023-11-28T11:10:53.847689+0000","last_peered":"2023-11-28T11:10:53.847689+0000","last_clean":"2023-11-28T11:10:53.847689+0000","last_became_active":"2023-11-28T11:10:52.396742+0000","last_became_peered":"2023-11-28T11:10:52.396742+0000","last_unstale":"2023-11-28T11:10:53.847689+0000","last_undegraded":"2023-11-28T11:10:53.847689+0000","last_fullsized":"2023-11-28T11:10:53.847689+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:00.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848795+0000","last_change":"2023-11-28T11:10:52.401622+0000","last_active":"2023-11-28T11:10:53.848795+0000","last_peered":"2023-11-28T11:10:53.848795+0000","last_clean":"2023-11-28T11:10:53.848795+0000","last_became_active":"2023-11-28T11:10:52.401310+0000","last_became_peered":"2023-11-28T11:10:52.401310+0000","last_unstale":"2023-11-28T11:10:53.848795+0000","last_undegraded":"2023-11-28T11:10:53.848795+0000","last_fullsized":"2023-11-28T11:10:53.848795+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:23.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848431+0000","last_change":"2023-11-28T11:10:52.398550+0000","last_active":"2023-11-28T11:10:53.848431+0000","last_peered":"2023-11-28T11:10:53.848431+0000","last_clean":"2023-11-28T11:10:53.848431+0000","last_became_active":"2023-11-28T11:10:52.398277+0000","last_became_peered":"2023-11-28T11:10:52.398277+0000","last_unstale":"2023-11-28T11:10:53.848431+0000","last_undegraded":"2023-11-28T11:10:53.848431+0000","last_fullsized":"2023-11-28T11:10:53.848431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:53.807072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848691+0000","last_change":"2023-11-28T11:10:52.396844+0000","last_active":"2023-11-28T11:10:53.848691+0000","last_peered":"2023-11-28T11:10:53.848691+0000","last_clean":"2023-11-28T11:10:53.848691+0000","last_became_active":"2023-11-28T11:10:52.393214+0000","last_became_peered":"2023-11-28T11:10:52.393214+0000","last_unstale":"2023-11-28T11:10:53.848691+0000","last_undegraded":"2023-11-28T11:10:53.848691+0000","last_fullsized":"2023-11-28T11:10:53.848691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:12.100298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848063+0000","last_change":"2023-11-28T11:10:52.397847+0000","last_active":"2023-11-28T11:10:53.848063+0000","last_peered":"2023-11-28T11:10:53.848063+0000","last_clean":"2023-11-28T11:10:53.848063+0000","last_became_active":"2023-11-28T11:10:52.397580+0000","last_became_peered":"2023-11-28T11:10:52.397580+0000","last_unstale":"2023-11-28T11:10:53.848063+0000","last_undegraded":"2023-11-28T11:10:53.848063+0000","last_fullsized":"2023-11-28T11:10:53.848063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:11:29.529381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848017+0000","last_change":"2023-11-28T11:10:52.397544+0000","last_active":"2023-11-28T11:10:53.848017+0000","last_peered":"2023-11-28T11:10:53.848017+0000","last_clean":"2023-11-28T11:10:53.848017+0000","last_became_active":"2023-11-28T11:10:52.397232+0000","last_became_peered":"2023-11-28T11:10:52.397232+0000","last_unstale":"2023-11-28T11:10:53.848017+0000","last_undegraded":"2023-11-28T11:10:53.848017+0000","last_fullsized":"2023-11-28T11:10:53.848017+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:33.270658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847532+0000","last_change":"2023-11-28T11:10:52.388564+0000","last_active":"2023-11-28T11:10:53.847532+0000","last_peered":"2023-11-28T11:10:53.847532+0000","last_clean":"2023-11-28T11:10:53.847532+0000","last_became_active":"2023-11-28T11:10:52.388307+0000","last_became_peered":"2023-11-28T11:10:52.388307+0000","last_unstale":"2023-11-28T11:10:53.847532+0000","last_undegraded":"2023-11-28T11:10:53.847532+0000","last_fullsized":"2023-11-28T11:10:53.847532+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:54.802666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848635+0000","last_change":"2023-11-28T11:10:52.391388+0000","last_active":"2023-11-28T11:10:53.848635+0000","last_peered":"2023-11-28T11:10:53.848635+0000","last_clean":"2023-11-28T11:10:53.848635+0000","last_became_active":"2023-11-28T11:10:52.391076+0000","last_became_peered":"2023-11-28T11:10:52.391076+0000","last_unstale":"2023-11-28T11:10:53.848635+0000","last_undegraded":"2023-11-28T11:10:53.848635+0000","last_fullsized":"2023-11-28T11:10:53.848635+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:05:43.320524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376913+0000","last_change":"2023-11-28T11:10:52.380958+0000","last_active":"2023-11-28T11:10:53.376913+0000","last_peered":"2023-11-28T11:10:53.376913+0000","last_clean":"2023-11-28T11:10:53.376913+0000","last_became_active":"2023-11-28T11:10:52.380688+0000","last_became_peered":"2023-11-28T11:10:52.380688+0000","last_unstale":"2023-11-28T11:10:53.376913+0000","last_undegraded":"2023-11-28T11:10:53.376913+0000","last_fullsized":"2023-11-28T11:10:53.376913+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:22.866138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375117+0000","last_change":"2023-11-28T11:10:52.387375+0000","last_active":"2023-11-28T11:10:53.375117+0000","last_peered":"2023-11-28T11:10:53.375117+0000","last_clean":"2023-11-28T11:10:53.375117+0000","last_became_active":"2023-11-28T11:10:52.387088+0000","last_became_peered":"2023-11-28T11:10:52.387088+0000","last_unstale":"2023-11-28T11:10:53.375117+0000","last_undegraded":"2023-11-28T11:10:53.375117+0000","last_fullsized":"2023-11-28T11:10:53.375117+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:54.810083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375344+0000","last_change":"2023-11-28T11:10:52.398710+0000","last_active":"2023-11-28T11:10:53.375344+0000","last_peered":"2023-11-28T11:10:53.375344+0000","last_clean":"2023-11-28T11:10:53.375344+0000","last_became_active":"2023-11-28T11:10:52.398388+0000","last_became_peered":"2023-11-28T11:10:52.398388+0000","last_unstale":"2023-11-28T11:10:53.375344+0000","last_undegraded":"2023-11-28T11:10:53.375344+0000","last_fullsized":"2023-11-28T11:10:53.375344+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:09.520497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.374950+0000","last_change":"2023-11-28T11:10:52.398644+0000","last_active":"2023-11-28T11:10:53.374950+0000","last_peered":"2023-11-28T11:10:53.374950+0000","last_clean":"2023-11-28T11:10:53.374950+0000","last_became_active":"2023-11-28T11:10:52.398342+0000","last_became_peered":"2023-11-28T11:10:52.398342+0000","last_unstale":"2023-11-28T11:10:53.374950+0000","last_undegraded":"2023-11-28T11:10:53.374950+0000","last_fullsized":"2023-11-28T11:10:53.374950+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:02.775343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848270+0000","last_change":"2023-11-28T11:10:52.400926+0000","last_active":"2023-11-28T11:10:53.848270+0000","last_peered":"2023-11-28T11:10:53.848270+0000","last_clean":"2023-11-28T11:10:53.848270+0000","last_became_active":"2023-11-28T11:10:52.400667+0000","last_became_peered":"2023-11-28T11:10:52.400667+0000","last_unstale":"2023-11-28T11:10:53.848270+0000","last_undegraded":"2023-11-28T11:10:53.848270+0000","last_fullsized":"2023-11-28T11:10:53.848270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:06.203049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848749+0000","last_change":"2023-11-28T11:10:52.390919+0000","last_active":"2023-11-28T11:10:53.848749+0000","last_peered":"2023-11-28T11:10:53.848749+0000","last_clean":"2023-11-28T11:10:53.848749+0000","last_became_active":"2023-11-28T11:10:52.390435+0000","last_became_peered":"2023-11-28T11:10:52.390435+0000","last_unstale":"2023-11-28T11:10:53.848749+0000","last_undegraded":"2023-11-28T11:10:53.848749+0000","last_fullsized":"2023-11-28T11:10:53.848749+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:48.226111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848092+0000","last_change":"2023-11-28T11:10:52.397318+0000","last_active":"2023-11-28T11:10:53.848092+0000","last_peered":"2023-11-28T11:10:53.848092+0000","last_clean":"2023-11-28T11:10:53.848092+0000","last_became_active":"2023-11-28T11:10:52.397039+0000","last_became_peered":"2023-11-28T11:10:52.397039+0000","last_unstale":"2023-11-28T11:10:53.848092+0000","last_undegraded":"2023-11-28T11:10:53.848092+0000","last_fullsized":"2023-11-28T11:10:53.848092+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:54.810083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847782+0000","last_change":"2023-11-28T11:10:52.394969+0000","last_active":"2023-11-28T11:10:53.847782+0000","last_peered":"2023-11-28T11:10:53.847782+0000","last_clean":"2023-11-28T11:10:53.847782+0000","last_became_active":"2023-11-28T11:10:52.394620+0000","last_became_peered":"2023-11-28T11:10:52.394620+0000","last_unstale":"2023-11-28T11:10:53.847782+0000","last_undegraded":"2023-11-28T11:10:53.847782+0000","last_fullsized":"2023-11-28T11:10:53.847782+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:28.345284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847322+0000","last_change":"2023-11-28T11:10:52.382159+0000","last_active":"2023-11-28T11:10:53.847322+0000","last_peered":"2023-11-28T11:10:53.847322+0000","last_clean":"2023-11-28T11:10:53.847322+0000","last_became_active":"2023-11-28T11:10:52.381757+0000","last_became_peered":"2023-11-28T11:10:52.381757+0000","last_unstale":"2023-11-28T11:10:53.847322+0000","last_undegraded":"2023-11-28T11:10:53.847322+0000","last_fullsized":"2023-11-28T11:10:53.847322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:21.448143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377057+0000","last_change":"2023-11-28T11:10:52.389365+0000","last_active":"2023-11-28T11:10:53.377057+0000","last_peered":"2023-11-28T11:10:53.377057+0000","last_clean":"2023-11-28T11:10:53.377057+0000","last_became_active":"2023-11-28T11:10:52.388931+0000","last_became_peered":"2023-11-28T11:10:52.388931+0000","last_unstale":"2023-11-28T11:10:53.377057+0000","last_undegraded":"2023-11-28T11:10:53.377057+0000","last_fullsized":"2023-11-28T11:10:53.377057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:30.962468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978200+0000","last_change":"2023-11-28T11:10:50.978200+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978200+0000","last_undegraded":"2023-11-28T11:10:50.978200+0000","last_fullsized":"2023-11-28T11:10:50.978200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:23.426647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848809+0000","last_change":"2023-11-28T11:10:52.390732+0000","last_active":"2023-11-28T11:10:53.848809+0000","last_peered":"2023-11-28T11:10:53.848809+0000","last_clean":"2023-11-28T11:10:53.848809+0000","last_became_active":"2023-11-28T11:10:52.390111+0000","last_became_peered":"2023-11-28T11:10:52.390111+0000","last_unstale":"2023-11-28T11:10:53.848809+0000","last_undegraded":"2023-11-28T11:10:53.848809+0000","last_fullsized":"2023-11-28T11:10:53.848809+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:18.651068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.378205+0000","last_change":"2023-11-28T11:10:51.995120+0000","last_active":"2023-11-28T11:10:52.378205+0000","last_peered":"2023-11-28T11:10:52.378205+0000","last_clean":"2023-11-28T11:10:52.378205+0000","last_became_active":"2023-11-28T11:10:51.994836+0000","last_became_peered":"2023-11-28T11:10:51.994836+0000","last_unstale":"2023-11-28T11:10:52.378205+0000","last_undegraded":"2023-11-28T11:10:52.378205+0000","last_fullsized":"2023-11-28T11:10:52.378205+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:32.263075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848457+0000","last_change":"2023-11-28T11:10:52.385151+0000","last_active":"2023-11-28T11:10:53.848457+0000","last_peered":"2023-11-28T11:10:53.848457+0000","last_clean":"2023-11-28T11:10:53.848457+0000","last_became_active":"2023-11-28T11:10:52.381849+0000","last_became_peered":"2023-11-28T11:10:52.381849+0000","last_unstale":"2023-11-28T11:10:53.848457+0000","last_undegraded":"2023-11-28T11:10:53.848457+0000","last_fullsized":"2023-11-28T11:10:53.848457+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:02.775343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.384672+0000","last_change":"2023-11-28T11:10:52.000037+0000","last_active":"2023-11-28T11:10:52.384672+0000","last_peered":"2023-11-28T11:10:52.384672+0000","last_clean":"2023-11-28T11:10:52.384672+0000","last_became_active":"2023-11-28T11:10:51.999788+0000","last_became_peered":"2023-11-28T11:10:51.999788+0000","last_unstale":"2023-11-28T11:10:52.384672+0000","last_undegraded":"2023-11-28T11:10:52.384672+0000","last_fullsized":"2023-11-28T11:10:52.384672+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:52.799489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848659+0000","last_change":"2023-11-28T11:10:52.399189+0000","last_active":"2023-11-28T11:10:53.848659+0000","last_peered":"2023-11-28T11:10:53.848659+0000","last_clean":"2023-11-28T11:10:53.848659+0000","last_became_active":"2023-11-28T11:10:52.398894+0000","last_became_peered":"2023-11-28T11:10:52.398894+0000","last_unstale":"2023-11-28T11:10:53.848659+0000","last_undegraded":"2023-11-28T11:10:53.848659+0000","last_fullsized":"2023-11-28T11:10:53.848659+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:09.926232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.381856+0000","last_change":"2023-11-28T11:10:51.990812+0000","last_active":"2023-11-28T11:10:52.381856+0000","last_peered":"2023-11-28T11:10:52.381856+0000","last_clean":"2023-11-28T11:10:52.381856+0000","last_became_active":"2023-11-28T11:10:51.990539+0000","last_became_peered":"2023-11-28T11:10:51.990539+0000","last_unstale":"2023-11-28T11:10:52.381856+0000","last_undegraded":"2023-11-28T11:10:52.381856+0000","last_fullsized":"2023-11-28T11:10:52.381856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:06.486410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847570+0000","last_change":"2023-11-28T11:10:52.381091+0000","last_active":"2023-11-28T11:10:53.847570+0000","last_peered":"2023-11-28T11:10:53.847570+0000","last_clean":"2023-11-28T11:10:53.847570+0000","last_became_active":"2023-11-28T11:10:52.380839+0000","last_became_peered":"2023-11-28T11:10:52.380839+0000","last_unstale":"2023-11-28T11:10:53.847570+0000","last_undegraded":"2023-11-28T11:10:53.847570+0000","last_fullsized":"2023-11-28T11:10:53.847570+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:29:30.173766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.375838+0000","last_change":"2023-11-28T11:10:51.985814+0000","last_active":"2023-11-28T11:10:52.375838+0000","last_peered":"2023-11-28T11:10:52.375838+0000","last_clean":"2023-11-28T11:10:52.375838+0000","last_became_active":"2023-11-28T11:10:51.985536+0000","last_became_peered":"2023-11-28T11:10:51.985536+0000","last_unstale":"2023-11-28T11:10:52.375838+0000","last_undegraded":"2023-11-28T11:10:52.375838+0000","last_fullsized":"2023-11-28T11:10:52.375838+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:11.092714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377014+0000","last_change":"2023-11-28T11:10:51.984690+0000","last_active":"2023-11-28T11:10:52.377014+0000","last_peered":"2023-11-28T11:10:52.377014+0000","last_clean":"2023-11-28T11:10:52.377014+0000","last_became_active":"2023-11-28T11:10:51.984385+0000","last_became_peered":"2023-11-28T11:10:51.984385+0000","last_unstale":"2023-11-28T11:10:52.377014+0000","last_undegraded":"2023-11-28T11:10:52.377014+0000","last_fullsized":"2023-11-28T11:10:52.377014+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:06.486410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376774+0000","last_change":"2023-11-28T11:10:52.390623+0000","last_active":"2023-11-28T11:10:53.376774+0000","last_peered":"2023-11-28T11:10:53.376774+0000","last_clean":"2023-11-28T11:10:53.376774+0000","last_became_active":"2023-11-28T11:10:52.390344+0000","last_became_peered":"2023-11-28T11:10:52.390344+0000","last_unstale":"2023-11-28T11:10:53.376774+0000","last_undegraded":"2023-11-28T11:10:53.376774+0000","last_fullsized":"2023-11-28T11:10:53.376774+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:45.616308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.370871+0000","last_change":"2023-11-28T11:10:51.991786+0000","last_active":"2023-11-28T11:10:52.370871+0000","last_peered":"2023-11-28T11:10:52.370871+0000","last_clean":"2023-11-28T11:10:52.370871+0000","last_became_active":"2023-11-28T11:10:51.991493+0000","last_became_peered":"2023-11-28T11:10:51.991493+0000","last_unstale":"2023-11-28T11:10:52.370871+0000","last_undegraded":"2023-11-28T11:10:52.370871+0000","last_fullsized":"2023-11-28T11:10:52.370871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:21.757386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375324+0000","last_change":"2023-11-28T11:10:52.398474+0000","last_active":"2023-11-28T11:10:53.375324+0000","last_peered":"2023-11-28T11:10:53.375324+0000","last_clean":"2023-11-28T11:10:53.375324+0000","last_became_active":"2023-11-28T11:10:52.398155+0000","last_became_peered":"2023-11-28T11:10:52.398155+0000","last_unstale":"2023-11-28T11:10:53.375324+0000","last_undegraded":"2023-11-28T11:10:53.375324+0000","last_fullsized":"2023-11-28T11:10:53.375324+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:23.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.383141+0000","last_change":"2023-11-28T11:10:51.995022+0000","last_active":"2023-11-28T11:10:52.383141+0000","last_peered":"2023-11-28T11:10:52.383141+0000","last_clean":"2023-11-28T11:10:52.383141+0000","last_became_active":"2023-11-28T11:10:51.994731+0000","last_became_peered":"2023-11-28T11:10:51.994731+0000","last_unstale":"2023-11-28T11:10:52.383141+0000","last_undegraded":"2023-11-28T11:10:52.383141+0000","last_fullsized":"2023-11-28T11:10:52.383141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:31:32.827101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.395098+0000","last_change":"2023-11-28T11:10:51.998518+0000","last_active":"2023-11-28T11:10:52.395098+0000","last_peered":"2023-11-28T11:10:52.395098+0000","last_clean":"2023-11-28T11:10:52.395098+0000","last_became_active":"2023-11-28T11:10:51.998086+0000","last_became_peered":"2023-11-28T11:10:51.998086+0000","last_unstale":"2023-11-28T11:10:52.395098+0000","last_undegraded":"2023-11-28T11:10:52.395098+0000","last_fullsized":"2023-11-28T11:10:52.395098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:21:56.126127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375245+0000","last_change":"2023-11-28T11:10:52.383474+0000","last_active":"2023-11-28T11:10:53.375245+0000","last_peered":"2023-11-28T11:10:53.375245+0000","last_clean":"2023-11-28T11:10:53.375245+0000","last_became_active":"2023-11-28T11:10:52.383211+0000","last_became_peered":"2023-11-28T11:10:52.383211+0000","last_unstale":"2023-11-28T11:10:53.375245+0000","last_undegraded":"2023-11-28T11:10:53.375245+0000","last_fullsized":"2023-11-28T11:10:53.375245+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:48.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.384146+0000","last_change":"2023-11-28T11:10:51.997551+0000","last_active":"2023-11-28T11:10:52.384146+0000","last_peered":"2023-11-28T11:10:52.384146+0000","last_clean":"2023-11-28T11:10:52.384146+0000","last_became_active":"2023-11-28T11:10:51.997278+0000","last_became_peered":"2023-11-28T11:10:51.997278+0000","last_unstale":"2023-11-28T11:10:52.384146+0000","last_undegraded":"2023-11-28T11:10:52.384146+0000","last_fullsized":"2023-11-28T11:10:52.384146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:21.858555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377081+0000","last_change":"2023-11-28T11:10:52.393694+0000","last_active":"2023-11-28T11:10:53.377081+0000","last_peered":"2023-11-28T11:10:53.377081+0000","last_clean":"2023-11-28T11:10:53.377081+0000","last_became_active":"2023-11-28T11:10:52.393435+0000","last_became_peered":"2023-11-28T11:10:52.393435+0000","last_unstale":"2023-11-28T11:10:53.377081+0000","last_undegraded":"2023-11-28T11:10:53.377081+0000","last_fullsized":"2023-11-28T11:10:53.377081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:05:25.141120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"62'3","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.399587+0000","last_change":"2023-11-28T11:10:51.997311+0000","last_active":"2023-11-28T11:10:52.399587+0000","last_peered":"2023-11-28T11:10:52.399587+0000","last_clean":"2023-11-28T11:10:52.399587+0000","last_became_active":"2023-11-28T11:10:51.997038+0000","last_became_peered":"2023-11-28T11:10:51.997038+0000","last_unstale":"2023-11-28T11:10:52.399587+0000","last_undegraded":"2023-11-28T11:10:52.399587+0000","last_fullsized":"2023-11-28T11:10:52.399587+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:47.034924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848569+0000","last_change":"2023-11-28T11:10:52.393378+0000","last_active":"2023-11-28T11:10:53.848569+0000","last_peered":"2023-11-28T11:10:53.848569+0000","last_clean":"2023-11-28T11:10:53.848569+0000","last_became_active":"2023-11-28T11:10:52.393089+0000","last_became_peered":"2023-11-28T11:10:52.393089+0000","last_unstale":"2023-11-28T11:10:53.848569+0000","last_undegraded":"2023-11-28T11:10:53.848569+0000","last_fullsized":"2023-11-28T11:10:53.848569+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:22.764970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978374+0000","last_change":"2023-11-28T11:10:50.978374+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978374+0000","last_undegraded":"2023-11-28T11:10:50.978374+0000","last_fullsized":"2023-11-28T11:10:50.978374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:41:07.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847717+0000","last_change":"2023-11-28T11:10:52.402741+0000","last_active":"2023-11-28T11:10:53.847717+0000","last_peered":"2023-11-28T11:10:53.847717+0000","last_clean":"2023-11-28T11:10:53.847717+0000","last_became_active":"2023-11-28T11:10:52.402537+0000","last_became_peered":"2023-11-28T11:10:52.402537+0000","last_unstale":"2023-11-28T11:10:53.847717+0000","last_undegraded":"2023-11-28T11:10:53.847717+0000","last_fullsized":"2023-11-28T11:10:53.847717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:48.042507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.403563+0000","last_change":"2023-11-28T11:10:51.999231+0000","last_active":"2023-11-28T11:10:52.403563+0000","last_peered":"2023-11-28T11:10:52.403563+0000","last_clean":"2023-11-28T11:10:52.403563+0000","last_became_active":"2023-11-28T11:10:51.998951+0000","last_became_peered":"2023-11-28T11:10:51.998951+0000","last_unstale":"2023-11-28T11:10:52.403563+0000","last_undegraded":"2023-11-28T11:10:52.403563+0000","last_fullsized":"2023-11-28T11:10:52.403563+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:27.337701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.981250+0000","last_change":"2023-11-28T11:10:50.981250+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.981250+0000","last_undegraded":"2023-11-28T11:10:50.981250+0000","last_fullsized":"2023-11-28T11:10:50.981250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:29.954884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377049+0000","last_change":"2023-11-28T11:10:52.383141+0000","last_active":"2023-11-28T11:10:53.377049+0000","last_peered":"2023-11-28T11:10:53.377049+0000","last_clean":"2023-11-28T11:10:53.377049+0000","last_became_active":"2023-11-28T11:10:52.382460+0000","last_became_peered":"2023-11-28T11:10:52.382460+0000","last_unstale":"2023-11-28T11:10:53.377049+0000","last_undegraded":"2023-11-28T11:10:53.377049+0000","last_fullsized":"2023-11-28T11:10:53.377049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:24.111335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.393434+0000","last_change":"2023-11-28T11:10:51.999127+0000","last_active":"2023-11-28T11:10:52.393434+0000","last_peered":"2023-11-28T11:10:52.393434+0000","last_clean":"2023-11-28T11:10:52.393434+0000","last_became_active":"2023-11-28T11:10:51.998846+0000","last_became_peered":"2023-11-28T11:10:51.998846+0000","last_unstale":"2023-11-28T11:10:52.393434+0000","last_undegraded":"2023-11-28T11:10:52.393434+0000","last_fullsized":"2023-11-28T11:10:52.393434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:41:07.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"62'8","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.402319+0000","last_change":"2023-11-28T11:10:51.996381+0000","last_active":"2023-11-28T11:10:52.402319+0000","last_peered":"2023-11-28T11:10:52.402319+0000","last_clean":"2023-11-28T11:10:52.402319+0000","last_became_active":"2023-11-28T11:10:51.996095+0000","last_became_peered":"2023-11-28T11:10:51.996095+0000","last_unstale":"2023-11-28T11:10:52.402319+0000","last_undegraded":"2023-11-28T11:10:52.402319+0000","last_fullsized":"2023-11-28T11:10:52.402319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:38.465853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847836+0000","last_change":"2023-11-28T11:10:52.394438+0000","last_active":"2023-11-28T11:10:53.847836+0000","last_peered":"2023-11-28T11:10:53.847836+0000","last_clean":"2023-11-28T11:10:53.847836+0000","last_became_active":"2023-11-28T11:10:52.394191+0000","last_became_peered":"2023-11-28T11:10:52.394191+0000","last_unstale":"2023-11-28T11:10:53.847836+0000","last_undegraded":"2023-11-28T11:10:53.847836+0000","last_fullsized":"2023-11-28T11:10:53.847836+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:35.306404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978466+0000","last_change":"2023-11-28T11:10:50.978466+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978466+0000","last_undegraded":"2023-11-28T11:10:50.978466+0000","last_fullsized":"2023-11-28T11:10:50.978466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:44.608724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848232+0000","last_change":"2023-11-28T11:10:52.384083+0000","last_active":"2023-11-28T11:10:53.848232+0000","last_peered":"2023-11-28T11:10:53.848232+0000","last_clean":"2023-11-28T11:10:53.848232+0000","last_became_active":"2023-11-28T11:10:52.383213+0000","last_became_peered":"2023-11-28T11:10:52.383213+0000","last_unstale":"2023-11-28T11:10:53.848232+0000","last_undegraded":"2023-11-28T11:10:53.848232+0000","last_fullsized":"2023-11-28T11:10:53.848232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:00.199905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978752+0000","last_change":"2023-11-28T11:10:50.978752+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978752+0000","last_undegraded":"2023-11-28T11:10:50.978752+0000","last_fullsized":"2023-11-28T11:10:50.978752+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:35.847896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.373540+0000","last_change":"2023-11-28T11:10:52.000024+0000","last_active":"2023-11-28T11:10:52.373540+0000","last_peered":"2023-11-28T11:10:52.373540+0000","last_clean":"2023-11-28T11:10:52.373540+0000","last_became_active":"2023-11-28T11:10:51.999765+0000","last_became_peered":"2023-11-28T11:10:51.999765+0000","last_unstale":"2023-11-28T11:10:52.373540+0000","last_undegraded":"2023-11-28T11:10:52.373540+0000","last_fullsized":"2023-11-28T11:10:52.373540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:56.382110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376866+0000","last_change":"2023-11-28T11:10:52.388835+0000","last_active":"2023-11-28T11:10:53.376866+0000","last_peered":"2023-11-28T11:10:53.376866+0000","last_clean":"2023-11-28T11:10:53.376866+0000","last_became_active":"2023-11-28T11:10:52.388560+0000","last_became_peered":"2023-11-28T11:10:52.388560+0000","last_unstale":"2023-11-28T11:10:53.376866+0000","last_undegraded":"2023-11-28T11:10:53.376866+0000","last_fullsized":"2023-11-28T11:10:53.376866+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:03:08.782678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.977949+0000","last_change":"2023-11-28T11:10:50.977949+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.977949+0000","last_undegraded":"2023-11-28T11:10:50.977949+0000","last_fullsized":"2023-11-28T11:10:50.977949+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:11.021567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376607+0000","last_change":"2023-11-28T11:10:52.387164+0000","last_active":"2023-11-28T11:10:53.376607+0000","last_peered":"2023-11-28T11:10:53.376607+0000","last_clean":"2023-11-28T11:10:53.376607+0000","last_became_active":"2023-11-28T11:10:52.386857+0000","last_became_peered":"2023-11-28T11:10:52.386857+0000","last_unstale":"2023-11-28T11:10:53.376607+0000","last_undegraded":"2023-11-28T11:10:53.376607+0000","last_fullsized":"2023-11-28T11:10:53.376607+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:15:03.899294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.389892+0000","last_change":"2023-11-28T11:10:51.999035+0000","last_active":"2023-11-28T11:10:52.389892+0000","last_peered":"2023-11-28T11:10:52.389892+0000","last_clean":"2023-11-28T11:10:52.389892+0000","last_became_active":"2023-11-28T11:10:51.998758+0000","last_became_peered":"2023-11-28T11:10:51.998758+0000","last_unstale":"2023-11-28T11:10:52.389892+0000","last_undegraded":"2023-11-28T11:10:52.389892+0000","last_fullsized":"2023-11-28T11:10:52.389892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:11.021567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848172+0000","last_change":"2023-11-28T11:10:52.401133+0000","last_active":"2023-11-28T11:10:53.848172+0000","last_peered":"2023-11-28T11:10:53.848172+0000","last_clean":"2023-11-28T11:10:53.848172+0000","last_became_active":"2023-11-28T11:10:52.400869+0000","last_became_peered":"2023-11-28T11:10:52.400869+0000","last_unstale":"2023-11-28T11:10:53.848172+0000","last_undegraded":"2023-11-28T11:10:53.848172+0000","last_fullsized":"2023-11-28T11:10:53.848172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:38.996786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.371070+0000","last_change":"2023-11-28T11:10:51.989318+0000","last_active":"2023-11-28T11:10:52.371070+0000","last_peered":"2023-11-28T11:10:52.371070+0000","last_clean":"2023-11-28T11:10:52.371070+0000","last_became_active":"2023-11-28T11:10:51.988981+0000","last_became_peered":"2023-11-28T11:10:51.988981+0000","last_unstale":"2023-11-28T11:10:52.371070+0000","last_undegraded":"2023-11-28T11:10:52.371070+0000","last_fullsized":"2023-11-28T11:10:52.371070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:05.195466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.370973+0000","last_change":"2023-11-28T11:10:51.997542+0000","last_active":"2023-11-28T11:10:52.370973+0000","last_peered":"2023-11-28T11:10:52.370973+0000","last_clean":"2023-11-28T11:10:52.370973+0000","last_became_active":"2023-11-28T11:10:51.997247+0000","last_became_peered":"2023-11-28T11:10:51.997247+0000","last_unstale":"2023-11-28T11:10:52.370973+0000","last_undegraded":"2023-11-28T11:10:52.370973+0000","last_fullsized":"2023-11-28T11:10:52.370973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:35.988439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847922+0000","last_change":"2023-11-28T11:10:52.388406+0000","last_active":"2023-11-28T11:10:53.847922+0000","last_peered":"2023-11-28T11:10:53.847922+0000","last_clean":"2023-11-28T11:10:53.847922+0000","last_became_active":"2023-11-28T11:10:52.388066+0000","last_became_peered":"2023-11-28T11:10:52.388066+0000","last_unstale":"2023-11-28T11:10:53.847922+0000","last_undegraded":"2023-11-28T11:10:53.847922+0000","last_fullsized":"2023-11-28T11:10:53.847922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:43.460660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.977903+0000","last_change":"2023-11-28T11:10:50.977903+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.977903+0000","last_undegraded":"2023-11-28T11:10:50.977903+0000","last_fullsized":"2023-11-28T11:10:50.977903+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:37.815667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848272+0000","last_change":"2023-11-28T11:10:52.398149+0000","last_active":"2023-11-28T11:10:53.848272+0000","last_peered":"2023-11-28T11:10:53.848272+0000","last_clean":"2023-11-28T11:10:53.848272+0000","last_became_active":"2023-11-28T11:10:52.397875+0000","last_became_peered":"2023-11-28T11:10:52.397875+0000","last_unstale":"2023-11-28T11:10:53.848272+0000","last_undegraded":"2023-11-28T11:10:53.848272+0000","last_fullsized":"2023-11-28T11:10:53.848272+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:00.423890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.979017+0000","last_change":"2023-11-28T11:10:50.979017+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.979017+0000","last_undegraded":"2023-11-28T11:10:50.979017+0000","last_fullsized":"2023-11-28T11:10:50.979017+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:00.060672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.027168+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:10:48.027168+0000","last_peered":"2023-11-28T11:10:48.027168+0000","last_clean":"2023-11-28T11:10:48.027168+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:10:48.027168+0000","last_undegraded":"2023-11-28T11:10:48.027168+0000","last_fullsized":"2023-11-28T11:10:48.027168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377047+0000","last_change":"2023-11-28T11:10:52.387511+0000","last_active":"2023-11-28T11:10:53.377047+0000","last_peered":"2023-11-28T11:10:53.377047+0000","last_clean":"2023-11-28T11:10:53.377047+0000","last_became_active":"2023-11-28T11:10:52.387143+0000","last_became_peered":"2023-11-28T11:10:52.387143+0000","last_unstale":"2023-11-28T11:10:53.377047+0000","last_undegraded":"2023-11-28T11:10:53.377047+0000","last_fullsized":"2023-11-28T11:10:53.377047+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:01.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.373304+0000","last_change":"2023-11-28T11:10:51.995232+0000","last_active":"2023-11-28T11:10:52.373304+0000","last_peered":"2023-11-28T11:10:52.373304+0000","last_clean":"2023-11-28T11:10:52.373304+0000","last_became_active":"2023-11-28T11:10:51.994941+0000","last_became_peered":"2023-11-28T11:10:51.994941+0000","last_unstale":"2023-11-28T11:10:52.373304+0000","last_undegraded":"2023-11-28T11:10:52.373304+0000","last_fullsized":"2023-11-28T11:10:52.373304+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:11:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:47.963710+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:10:47.963710+0000","last_peered":"2023-11-28T11:10:47.963710+0000","last_clean":"2023-11-28T11:10:47.963710+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:10:47.963710+0000","last_undegraded":"2023-11-28T11:10:47.963710+0000","last_fullsized":"2023-11-28T11:10:47.963710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978394+0000","last_change":"2023-11-28T11:10:50.978394+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978394+0000","last_undegraded":"2023-11-28T11:10:50.978394+0000","last_fullsized":"2023-11-28T11:10:50.978394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:38.465853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377553+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:10:52.377553+0000","last_peered":"2023-11-28T11:10:52.377553+0000","last_clean":"2023-11-28T11:10:52.377553+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:10:52.377553+0000","last_undegraded":"2023-11-28T11:10:52.377553+0000","last_fullsized":"2023-11-28T11:10:52.377553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377169+0000","last_change":"2023-11-28T11:10:52.387614+0000","last_active":"2023-11-28T11:10:53.377169+0000","last_peered":"2023-11-28T11:10:53.377169+0000","last_clean":"2023-11-28T11:10:53.377169+0000","last_became_active":"2023-11-28T11:10:52.387338+0000","last_became_peered":"2023-11-28T11:10:52.387338+0000","last_unstale":"2023-11-28T11:10:53.377169+0000","last_undegraded":"2023-11-28T11:10:53.377169+0000","last_fullsized":"2023-11-28T11:10:53.377169+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:36.855479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.370866+0000","last_change":"2023-11-28T11:10:51.985858+0000","last_active":"2023-11-28T11:10:52.370866+0000","last_peered":"2023-11-28T11:10:52.370866+0000","last_clean":"2023-11-28T11:10:52.370866+0000","last_became_active":"2023-11-28T11:10:51.985623+0000","last_became_peered":"2023-11-28T11:10:51.985623+0000","last_unstale":"2023-11-28T11:10:52.370866+0000","last_undegraded":"2023-11-28T11:10:52.370866+0000","last_fullsized":"2023-11-28T11:10:52.370866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:05:42.312941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:47.963710+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:10:47.963710+0000","last_peered":"2023-11-28T11:10:47.963710+0000","last_clean":"2023-11-28T11:10:47.963710+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:10:47.963710+0000","last_undegraded":"2023-11-28T11:10:47.963710+0000","last_fullsized":"2023-11-28T11:10:47.963710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":204,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.389688+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:10:52.389688+0000","last_peered":"2023-11-28T11:10:52.389688+0000","last_clean":"2023-11-28T11:10:52.389688+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:10:52.389688+0000","last_undegraded":"2023-11-28T11:10:52.389688+0000","last_fullsized":"2023-11-28T11:10:52.389688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847400+0000","last_change":"2023-11-28T11:10:52.381996+0000","last_active":"2023-11-28T11:10:53.847400+0000","last_peered":"2023-11-28T11:10:53.847400+0000","last_clean":"2023-11-28T11:10:53.847400+0000","last_became_active":"2023-11-28T11:10:52.381358+0000","last_became_peered":"2023-11-28T11:10:52.381358+0000","last_unstale":"2023-11-28T11:10:53.847400+0000","last_undegraded":"2023-11-28T11:10:53.847400+0000","last_fullsized":"2023-11-28T11:10:53.847400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:48.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.386941+0000","last_change":"2023-11-28T11:10:51.996342+0000","last_active":"2023-11-28T11:10:52.386941+0000","last_peered":"2023-11-28T11:10:52.386941+0000","last_clean":"2023-11-28T11:10:52.386941+0000","last_became_active":"2023-11-28T11:10:51.995997+0000","last_became_peered":"2023-11-28T11:10:51.995997+0000","last_unstale":"2023-11-28T11:10:52.386941+0000","last_undegraded":"2023-11-28T11:10:52.386941+0000","last_fullsized":"2023-11-28T11:10:52.386941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:35.988439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.389449+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:10:52.389449+0000","last_peered":"2023-11-28T11:10:52.389449+0000","last_clean":"2023-11-28T11:10:52.389449+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:10:52.389449+0000","last_undegraded":"2023-11-28T11:10:52.389449+0000","last_fullsized":"2023-11-28T11:10:52.389449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376952+0000","last_change":"2023-11-28T11:10:52.397798+0000","last_active":"2023-11-28T11:10:53.376952+0000","last_peered":"2023-11-28T11:10:53.376952+0000","last_clean":"2023-11-28T11:10:53.376952+0000","last_became_active":"2023-11-28T11:10:52.397532+0000","last_became_peered":"2023-11-28T11:10:52.397532+0000","last_unstale":"2023-11-28T11:10:53.376952+0000","last_undegraded":"2023-11-28T11:10:53.376952+0000","last_fullsized":"2023-11-28T11:10:53.376952+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:38.823250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.398098+0000","last_change":"2023-11-28T11:10:52.000553+0000","last_active":"2023-11-28T11:10:52.398098+0000","last_peered":"2023-11-28T11:10:52.398098+0000","last_clean":"2023-11-28T11:10:52.398098+0000","last_became_active":"2023-11-28T11:10:52.000223+0000","last_became_peered":"2023-11-28T11:10:52.000223+0000","last_unstale":"2023-11-28T11:10:52.398098+0000","last_undegraded":"2023-11-28T11:10:52.398098+0000","last_fullsized":"2023-11-28T11:10:52.398098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:37.815667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.391001+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:10:52.391001+0000","last_peered":"2023-11-28T11:10:52.391001+0000","last_clean":"2023-11-28T11:10:52.391001+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:10:52.391001+0000","last_undegraded":"2023-11-28T11:10:52.391001+0000","last_fullsized":"2023-11-28T11:10:52.391001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848738+0000","last_change":"2023-11-28T11:10:52.388031+0000","last_active":"2023-11-28T11:10:53.848738+0000","last_peered":"2023-11-28T11:10:53.848738+0000","last_clean":"2023-11-28T11:10:53.848738+0000","last_became_active":"2023-11-28T11:10:52.387685+0000","last_became_peered":"2023-11-28T11:10:52.387685+0000","last_unstale":"2023-11-28T11:10:53.848738+0000","last_undegraded":"2023-11-28T11:10:53.848738+0000","last_fullsized":"2023-11-28T11:10:53.848738+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:19:57.624643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.399510+0000","last_change":"2023-11-28T11:10:51.998733+0000","last_active":"2023-11-28T11:10:52.399510+0000","last_peered":"2023-11-28T11:10:52.399510+0000","last_clean":"2023-11-28T11:10:52.399510+0000","last_became_active":"2023-11-28T11:10:51.998123+0000","last_became_peered":"2023-11-28T11:10:51.998123+0000","last_unstale":"2023-11-28T11:10:52.399510+0000","last_undegraded":"2023-11-28T11:10:52.399510+0000","last_fullsized":"2023-11-28T11:10:52.399510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:23.426647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.027097+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:10:48.027097+0000","last_peered":"2023-11-28T11:10:48.027097+0000","last_clean":"2023-11-28T11:10:48.027097+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:10:48.027097+0000","last_undegraded":"2023-11-28T11:10:48.027097+0000","last_fullsized":"2023-11-28T11:10:48.027097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376982+0000","last_change":"2023-11-28T11:10:52.389489+0000","last_active":"2023-11-28T11:10:53.376982+0000","last_peered":"2023-11-28T11:10:53.376982+0000","last_clean":"2023-11-28T11:10:53.376982+0000","last_became_active":"2023-11-28T11:10:52.387877+0000","last_became_peered":"2023-11-28T11:10:52.387877+0000","last_unstale":"2023-11-28T11:10:53.376982+0000","last_undegraded":"2023-11-28T11:10:53.376982+0000","last_fullsized":"2023-11-28T11:10:53.376982+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:08:26.596352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.371082+0000","last_change":"2023-11-28T11:10:51.992409+0000","last_active":"2023-11-28T11:10:52.371082+0000","last_peered":"2023-11-28T11:10:52.371082+0000","last_clean":"2023-11-28T11:10:52.371082+0000","last_became_active":"2023-11-28T11:10:51.992031+0000","last_became_peered":"2023-11-28T11:10:51.992031+0000","last_unstale":"2023-11-28T11:10:52.371082+0000","last_undegraded":"2023-11-28T11:10:52.371082+0000","last_fullsized":"2023-11-28T11:10:52.371082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:56.382110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.380028+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:10:52.380028+0000","last_peered":"2023-11-28T11:10:52.380028+0000","last_clean":"2023-11-28T11:10:52.380028+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:10:52.380028+0000","last_undegraded":"2023-11-28T11:10:52.380028+0000","last_fullsized":"2023-11-28T11:10:52.380028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":194,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10880,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360960,"statfs":{"total":96636764160,"available":96625623040,"internally_reserved":0,"allocated":1368064,"data_stored":925048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92200000000000004,"5min":1.006,"15min":1.006},"last":0.27800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.67500000000000004,"15min":0.69999999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71099999999999997,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.747,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.35199999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0349999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.432},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97199999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.39700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.33700000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.86699999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.38}]},{"osd":5,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92100000000000004,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.37},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.443}]},{"osd":6,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.47299999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.52000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9328,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362512,"statfs":{"total":96636764160,"available":96627212288,"internally_reserved":0,"allocated":761856,"data_stored":333954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69499999999999995,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.70299999999999996,"15min":0.73399999999999999},"min":{"1min":0.36299999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76800000000000002,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.69099999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.81599999999999995,"15min":0.85999999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76500000000000001,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.77000000000000002,"15min":0.80500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79800000000000004,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.78900000000000003,"15min":0.83599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67200000000000004,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.68899999999999995,"15min":0.71699999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68400000000000005,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.76800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.755,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.65400000000000003,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65700000000000003,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.755}]},{"osd":4,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.85099999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.379,"5min":0.373,"15min":0.373},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.88800000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76900000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.86099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9288,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362552,"statfs":{"total":96636764160,"available":96627253248,"internally_reserved":0,"allocated":720896,"data_stored":313519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.59999999999999998,"15min":0.64600000000000002},"min":{"1min":0.221,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.504,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.377,"15min":0.38200000000000001},"min":{"1min":0.248,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.46100000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.378}]},{"osd":2,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.64200000000000002,"15min":0.68999999999999995},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56999999999999995,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51900000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.44600000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42899999999999999,"15min":0.435},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58899999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.375,"15min":0.375},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54200000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.3120000000000001,"15min":1.431},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79500000000000004,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.76100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78800000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":1.3440000000000001,"15min":1.462},"min":{"1min":0.47799999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93600000000000005,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.67900000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":1.3500000000000001,"15min":1.4670000000000001},"min":{"1min":0.498,"5min":0.439,"15min":0.439},"max":{"1min":0.95999999999999996,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.48599999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94399999999999995,"5min":1.018,"15min":1.018},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.44700000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14768,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94357072,"statfs":{"total":96636764160,"available":96621641728,"internally_reserved":0,"allocated":1351680,"data_stored":924288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.48599999999999999,"5min":0.502,"15min":0.502},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.44400000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.26100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53600000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.378,"15min":0.378},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.50900000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.29699999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47799999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.56699999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.311}]},{"osd":4,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58999999999999997,"15min":0.59299999999999997},"min":{"1min":0.38800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66700000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.46700000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.35899999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.434,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.39600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.432},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.47399999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.62,"15min":0.62},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.81399999999999995,"5min":1.026,"15min":1.026},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.41699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9452,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362388,"statfs":{"total":96636764160,"available":96627085312,"internally_reserved":0,"allocated":757760,"data_stored":333920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.32500000000000001,"15min":0.316},"min":{"1min":0.22,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.33900000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.34899999999999998,"15min":0.33500000000000002},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33700000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.29299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.312},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.36899999999999999},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.35299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.437}]},{"osd":6,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.503},{"interface":"front","average":{"1min":0.63,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.44600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9772,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362068,"statfs":{"total":96636764160,"available":96626757632,"internally_reserved":0,"allocated":757760,"data_stored":333920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.42999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.31900000000000001,"15min":0.317},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.502,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.35399999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.56000000000000005,"5min":1.365,"15min":1.365},"last":0.33200000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48499999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.31}]},{"osd":4,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.495},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42899999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82999999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46000000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81200000000000006,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85999999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.48099999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9592,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362248,"statfs":{"total":96636764160,"available":96626941952,"internally_reserved":0,"allocated":770048,"data_stored":335468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69699999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.60099999999999998,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.9740000000000002,"5min":4.9740000000000002,"15min":4.9740000000000002},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":1.115,"15min":1.2070000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69699999999999995,"5min":12.263,"15min":12.263},"last":0.44700000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.505},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":1.236,"15min":1.236},"last":0.41399999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5600000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":4.3710000000000004,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.59499999999999997,"15min":0.55000000000000004},"min":{"1min":0.40200000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":5.0220000000000002,"5min":5.0220000000000002,"15min":5.0220000000000002},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.52000000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.65600000000000003,"15min":0.64100000000000001},"min":{"1min":0.55700000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.60199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9340,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362500,"statfs":{"total":96636764160,"available":96627200000,"internally_reserved":0,"allocated":708608,"data_stored":307349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.45700000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53100000000000003},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.504,"15min":0.505},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66300000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.438,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.432}]},{"osd":4,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.497,"15min":0.497},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72999999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.67500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72899999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.441,"5min":0.374,"15min":0.374},"max":{"1min":0.67700000000000005,"5min":0.753,"15min":0.753},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.747,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.59299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:10:57.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2023-11-28T11:10:57.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2023-11-28T11:10:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2023-11-28T11:10:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2023-11-28T11:10:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2023-11-28T11:10:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2023-11-28T11:10:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2023-11-28T11:10:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2023-11-28T11:10:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2023-11-28T11:10:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2023-11-28T11:10:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2023-11-28T11:10:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-11-28T11:10:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:10:50.978200+0000', 'last_change': '2023-11-28T11:10:50.978200+0000', 'last_active': '2023-11-28T11:10:50.964045+0000', 'last_peered': '2023-11-28T11:10:50.964045+0000', 'last_clean': '2023-11-28T11:10:50.964045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:50.978200+0000', 'last_undegraded': '2023-11-28T11:10:50.978200+0000', 'last_fullsized': '2023-11-28T11:10:50.978200+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:53:23.426647+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:10:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-11-28T11:10:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-11-28T11:10:57.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-11-28T11:10:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:10:50.978374+0000', 'last_change': '2023-11-28T11:10:50.978374+0000', 'last_active': '2023-11-28T11:10:50.964045+0000', 'last_peered': '2023-11-28T11:10:50.964045+0000', 'last_clean': '2023-11-28T11:10:50.964045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:50.978374+0000', 'last_undegraded': '2023-11-28T11:10:50.978374+0000', 'last_fullsized': '2023-11-28T11:10:50.978374+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:41:07.858489+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:10:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-11-28T11:10:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-11-28T11:10:57.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:10:50.981250+0000', 'last_change': '2023-11-28T11:10:50.981250+0000', 'last_active': '2023-11-28T11:10:50.964045+0000', 'last_peered': '2023-11-28T11:10:50.964045+0000', 'last_clean': '2023-11-28T11:10:50.964045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:50.981250+0000', 'last_undegraded': '2023-11-28T11:10:50.981250+0000', 'last_fullsized': '2023-11-28T11:10:50.981250+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:40:29.954884+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:10:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-11-28T11:10:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-11-28T11:10:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:10:50.978466+0000', 'last_change': '2023-11-28T11:10:50.978466+0000', 'last_active': '2023-11-28T11:10:50.964045+0000', 'last_peered': '2023-11-28T11:10:50.964045+0000', 'last_clean': '2023-11-28T11:10:50.964045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:50.978466+0000', 'last_undegraded': '2023-11-28T11:10:50.978466+0000', 'last_fullsized': '2023-11-28T11:10:50.978466+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T12:22:44.608724+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:10:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-11-28T11:10:57.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:10:50.978752+0000', 'last_change': '2023-11-28T11:10:50.978752+0000', 'last_active': '2023-11-28T11:10:50.964045+0000', 'last_peered': '2023-11-28T11:10:50.964045+0000', 'last_clean': '2023-11-28T11:10:50.964045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:50.978752+0000', 'last_undegraded': '2023-11-28T11:10:50.978752+0000', 'last_fullsized': '2023-11-28T11:10:50.978752+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:13:35.847896+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:10:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-11-28T11:10:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-11-28T11:10:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:10:50.977949+0000', 'last_change': '2023-11-28T11:10:50.977949+0000', 'last_active': '2023-11-28T11:10:50.964045+0000', 'last_peered': '2023-11-28T11:10:50.964045+0000', 'last_clean': '2023-11-28T11:10:50.964045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:50.977949+0000', 'last_undegraded': '2023-11-28T11:10:50.977949+0000', 'last_fullsized': '2023-11-28T11:10:50.977949+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T11:12:11.021567+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:10:57.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-11-28T11:10:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-11-28T11:10:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:10:50.977903+0000', 'last_change': '2023-11-28T11:10:50.977903+0000', 'last_active': '2023-11-28T11:10:50.964045+0000', 'last_peered': '2023-11-28T11:10:50.964045+0000', 'last_clean': '2023-11-28T11:10:50.964045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:50.977903+0000', 'last_undegraded': '2023-11-28T11:10:50.977903+0000', 'last_fullsized': '2023-11-28T11:10:50.977903+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T12:43:37.815667+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:10:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-11-28T11:10:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:10:50.979017+0000', 'last_change': '2023-11-28T11:10:50.979017+0000', 'last_active': '2023-11-28T11:10:50.964045+0000', 'last_peered': '2023-11-28T11:10:50.964045+0000', 'last_clean': '2023-11-28T11:10:50.964045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:50.979017+0000', 'last_undegraded': '2023-11-28T11:10:50.979017+0000', 'last_fullsized': '2023-11-28T11:10:50.979017+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T18:06:00.060672+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:10:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-11-28T11:10:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:51.971628+0000', 'last_change': '2023-11-28T11:10:51.971628+0000', 'last_active': '2023-11-28T11:10:51.971628+0000', 'last_peered': '2023-11-28T11:10:51.971628+0000', 'last_clean': '2023-11-28T11:10:51.971628+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:51.971628+0000', 'last_undegraded': '2023-11-28T11:10:51.971628+0000', 'last_fullsized': '2023-11-28T11:10:51.971628+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:51.971628+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:10:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-11-28T11:10:57.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:10:50.978394+0000', 'last_change': '2023-11-28T11:10:50.978394+0000', 'last_active': '2023-11-28T11:10:50.964045+0000', 'last_peered': '2023-11-28T11:10:50.964045+0000', 'last_clean': '2023-11-28T11:10:50.964045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:50.978394+0000', 'last_undegraded': '2023-11-28T11:10:50.978394+0000', 'last_fullsized': '2023-11-28T11:10:50.978394+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:50.964045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:45:38.465853+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:10:57.213 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T11:10:57.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.358+0000 7fe4d08a4700 1 Processor -- start 2023-11-28T11:10:57.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.358+0000 7fe4d08a4700 1 -- start start 2023-11-28T11:10:57.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.359+0000 7fe4d08a4700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe4cc06b060 0x7fe4cc06b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.359+0000 7fe4d08a4700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4cc0c9e10 con 0x7fe4cc160b10 2023-11-28T11:10:57.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.359+0000 7fe4d08a4700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4cc06efc0 con 0x7fe4cc160740 2023-11-28T11:10:57.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.359+0000 7fe4d08a4700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4cc0ca720 con 0x7fe4cc06b060 2023-11-28T11:10:57.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.359+0000 7fe4cad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe4cc06b060 0x7fe4cc06b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.359+0000 7fe4c9d9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe4cc160b10 0x7fe4cc0cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50600/0 (socket says 172.21.15.156:50600) 2023-11-28T11:10:57.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.359+0000 7fe4c9d9b700 1 -- 172.21.15.156:0/946001050 learned_addr learned my addr 172.21.15.156:0/946001050 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:10:57.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.359+0000 7fe4cad9d700 1 -- 172.21.15.156:0/946001050 >> v1:172.21.15.186:6789/0 conn(0x7fe4cc160740 legacy=0x7fe4cc0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:57.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.359+0000 7fe4cad9d700 1 -- 172.21.15.156:0/946001050 >> v1:172.21.15.156:6789/0 conn(0x7fe4cc160b10 legacy=0x7fe4cc0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.359+0000 7fe4cad9d700 1 -- 172.21.15.156:0/946001050 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc0ca0d0 con 0x7fe4cc06b060 2023-11-28T11:10:57.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.359+0000 7fe4cad9d700 1 --2- 172.21.15.156:0/946001050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4cc06b060 0x7fe4cc06b4c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009d90 tx=0x7fe4c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d37964103ffe71 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.360+0000 7fe4c959a700 1 -- 172.21.15.156:0/946001050 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c400e070 con 0x7fe4cc06b060 2023-11-28T11:10:57.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.360+0000 7fe4c959a700 1 -- 172.21.15.156:0/946001050 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c400bb90 con 0x7fe4cc06b060 2023-11-28T11:10:57.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.360+0000 7fe4c959a700 1 -- 172.21.15.156:0/946001050 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4003a20 con 0x7fe4cc06b060 2023-11-28T11:10:57.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.360+0000 7fe4d08a4700 1 -- 172.21.15.156:0/946001050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4cc06b060 msgr2=0x7fe4cc06b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.360+0000 7fe4d08a4700 1 --2- 172.21.15.156:0/946001050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4cc06b060 0x7fe4cc06b4c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009d90 tx=0x7fe4c400b330 comp rx=0 tx=0).stop 2023-11-28T11:10:57.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.360+0000 7fe4d08a4700 1 -- 172.21.15.156:0/946001050 shutdown_connections 2023-11-28T11:10:57.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.360+0000 7fe4d08a4700 1 --2- 172.21.15.156:0/946001050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4cc06b060 0x7fe4cc06b4c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.360+0000 7fe4d08a4700 1 -- 172.21.15.156:0/946001050 >> 172.21.15.156:0/946001050 conn(0x7fe4cc0b12b0 msgr2=0x7fe4cc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:57.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.360+0000 7fe4d08a4700 1 -- 172.21.15.156:0/946001050 shutdown_connections 2023-11-28T11:10:57.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.360+0000 7fe4d08a4700 1 -- 172.21.15.156:0/946001050 wait complete. 2023-11-28T11:10:57.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.362+0000 7fe4d08a4700 1 Processor -- start 2023-11-28T11:10:57.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.362+0000 7fe4d08a4700 1 -- start start 2023-11-28T11:10:57.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.362+0000 7fe4d08a4700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4cc06b060 0x7fe4cc1588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.362+0000 7fe4d08a4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4cc160740 0x7fe4cc14aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.362+0000 7fe4d08a4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4cc160b10 0x7fe4cc15a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.362+0000 7fe4d08a4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc14b1e0 con 0x7fe4cc160b10 2023-11-28T11:10:57.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.362+0000 7fe4d08a4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc15aa50 con 0x7fe4cc160740 2023-11-28T11:10:57.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.362+0000 7fe4d08a4700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc158f10 con 0x7fe4cc06b060 2023-11-28T11:10:57.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4cad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4cc160b10 0x7fe4cc15a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4cad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4cc160b10 0x7fe4cc15a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35732/0 (socket says 172.21.15.156:35732) 2023-11-28T11:10:57.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4cad9d700 1 -- 172.21.15.156:0/716561995 learned_addr learned my addr 172.21.15.156:0/716561995 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:57.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4c9d9b700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4cc160740 0x7fe4cc14aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4ca59c700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4cc06b060 0x7fe4cc1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4cad9d700 1 -- 172.21.15.156:0/716561995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4cc06b060 msgr2=0x7fe4cc1588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4cad9d700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4cc06b060 0x7fe4cc1588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4cad9d700 1 -- 172.21.15.156:0/716561995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4cc160740 msgr2=0x7fe4cc14aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4cad9d700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4cc160740 0x7fe4cc14aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4cad9d700 1 -- 172.21.15.156:0/716561995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c4009a70 con 0x7fe4cc160b10 2023-11-28T11:10:57.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4cad9d700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4cc160b10 0x7fe4cc15a510 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0704f0 tx=0x7fe4c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4baffd700 1 -- 172.21.15.156:0/716561995 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4012630 con 0x7fe4cc160b10 2023-11-28T11:10:57.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4baffd700 1 -- 172.21.15.156:0/716561995 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c40127b0 con 0x7fe4cc160b10 2023-11-28T11:10:57.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4d08a4700 1 -- 172.21.15.156:0/716561995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4cc160f60 con 0x7fe4cc160b10 2023-11-28T11:10:57.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4baffd700 1 -- 172.21.15.156:0/716561995 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4012b60 con 0x7fe4cc160b10 2023-11-28T11:10:57.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.363+0000 7fe4d08a4700 1 -- 172.21.15.156:0/716561995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4cc0ca3b0 con 0x7fe4cc160b10 2023-11-28T11:10:57.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.364+0000 7fe4d08a4700 1 -- 172.21.15.156:0/716561995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4cc06ec00 con 0x7fe4cc160b10 2023-11-28T11:10:57.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.364+0000 7fe4baffd700 1 -- 172.21.15.156:0/716561995 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe4c4012cc0 con 0x7fe4cc160b10 2023-11-28T11:10:57.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.365+0000 7fe4baffd700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe4ac0682f0 0x7fe4ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.365+0000 7fe4ca59c700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe4ac0682f0 0x7fe4ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.365+0000 7fe4baffd700 1 -- 172.21.15.156:0/716561995 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4c4088250 con 0x7fe4cc160b10 2023-11-28T11:10:57.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.365+0000 7fe4ca59c700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe4ac0682f0 0x7fe4ac06a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc003e60 tx=0x7fe4bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.369+0000 7fe4baffd700 1 -- 172.21.15.156:0/716561995 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe4c4054b70 con 0x7fe4cc160b10 2023-11-28T11:10:57.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.631+0000 7fe4d08a4700 1 -- 172.21.15.156:0/716561995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe4cc066110 con 0x7fe4cc160b10 2023-11-28T11:10:57.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.631+0000 7fe4baffd700 1 -- 172.21.15.156:0/716561995 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1722 (secure 0 0 0) 0x7fe4c4058190 con 0x7fe4cc160b10 2023-11-28T11:10:57.633 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:57.633 INFO:teuthology.orchestra.run.smithi156.stdout:{"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":182,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":63,"num_osds":8,"num_up_osds":8,"osd_up_since":1701169685,"num_in_osds":8,"osd_in_since":1701169677,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":78},{"state_name":"unknown","count":18},{"state_name":"creating+peering","count":9}],"num_pgs":105,"num_pools":4,"num_objects":15,"data_bytes":590562,"bytes_used":84398080,"bytes_avail":773009715200,"bytes_total":773094113280,"unknown_pgs_ratio":0.17142857611179352,"inactive_pgs_ratio":0.085714288055896759,"write_bytes_sec":1649,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":31,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":6021}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-11-28T11:09:08.536349+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-28T11:10:57.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.633+0000 7fe4d08a4700 1 -- 172.21.15.156:0/716561995 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe4ac0682f0 msgr2=0x7fe4ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.633+0000 7fe4d08a4700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe4ac0682f0 0x7fe4ac06a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc003e60 tx=0x7fe4bc009040 comp rx=0 tx=0).stop 2023-11-28T11:10:57.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.633+0000 7fe4d08a4700 1 -- 172.21.15.156:0/716561995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4cc160b10 msgr2=0x7fe4cc15a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.633+0000 7fe4d08a4700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4cc160b10 0x7fe4cc15a510 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0704f0 tx=0x7fe4c4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.633+0000 7fe4d08a4700 1 -- 172.21.15.156:0/716561995 shutdown_connections 2023-11-28T11:10:57.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.633+0000 7fe4d08a4700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4cc160b10 0x7fe4cc15a510 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.633+0000 7fe4d08a4700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe4ac0682f0 0x7fe4ac06a7a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.633+0000 7fe4d08a4700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4cc06b060 0x7fe4cc1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.633+0000 7fe4d08a4700 1 --2- 172.21.15.156:0/716561995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4cc160740 0x7fe4cc14aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.633+0000 7fe4d08a4700 1 -- 172.21.15.156:0/716561995 >> 172.21.15.156:0/716561995 conn(0x7fe4cc0b12b0 msgr2=0x7fe4cc06b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:57.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.633+0000 7fe4d08a4700 1 -- 172.21.15.156:0/716561995 shutdown_connections 2023-11-28T11:10:57.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.633+0000 7fe4d08a4700 1 -- 172.21.15.156:0/716561995 wait complete. 2023-11-28T11:10:57.647 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T11:10:57.647 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:10:57.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.767+0000 7f0e2a265700 1 Processor -- start 2023-11-28T11:10:57.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.767+0000 7f0e2a265700 1 -- start start 2023-11-28T11:10:57.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e2a265700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0e2415cef0 0x7f0e240af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e2a265700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e240cbdf0 0x7f0e2406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e2a265700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240cc5f0 con 0x7f0e2406b8c0 2023-11-28T11:10:57.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e2a265700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2406bd40 con 0x7f0e2415cef0 2023-11-28T11:10:57.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e2a265700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2415d470 con 0x7f0e240cbdf0 2023-11-28T11:10:57.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e237fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e240cbdf0 0x7f0e2406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e23fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0e2415cef0 0x7f0e240af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e237fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e240cbdf0 0x7f0e2406b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:39602/0 (socket says 172.21.15.156:39602) 2023-11-28T11:10:57.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e23fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0e2415cef0 0x7f0e240af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:46750/0 (socket says 172.21.15.156:46750) 2023-11-28T11:10:57.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e237fe700 1 -- 172.21.15.156:0/3936079146 learned_addr learned my addr 172.21.15.156:0/3936079146 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:57.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e28802700 1 --1- 172.21.15.156:0/3936079146 >> v1:172.21.15.186:6789/0 conn(0x7f0e2406b8c0 0x7f0e24071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:60978/0 (socket says 172.21.15.156:60978) 2023-11-28T11:10:57.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e22ffd700 1 -- 172.21.15.156:0/3936079146 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406017325 0 0) 0x7f0e240cc5f0 con 0x7f0e2406b8c0 2023-11-28T11:10:57.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e22ffd700 1 -- 172.21.15.156:0/3936079146 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e0c0035b0 con 0x7f0e2406b8c0 2023-11-28T11:10:57.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e237fe700 1 -- 172.21.15.156:0/3936079146 >> v2:172.21.15.186:3300/0 conn(0x7f0e2415cef0 msgr2=0x7f0e240af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:57.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e237fe700 1 --2- 172.21.15.156:0/3936079146 >> v2:172.21.15.186:3300/0 conn(0x7f0e2415cef0 0x7f0e240af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e237fe700 1 -- 172.21.15.156:0/3936079146 >> v1:172.21.15.186:6789/0 conn(0x7f0e2406b8c0 legacy=0x7f0e24071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.768+0000 7f0e237fe700 1 -- 172.21.15.156:0/3936079146 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2415d750 con 0x7f0e240cbdf0 2023-11-28T11:10:57.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.769+0000 7f0e23fff700 1 --2- 172.21.15.156:0/3936079146 >> v2:172.21.15.186:3300/0 conn(0x7f0e2415cef0 0x7f0e240af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:57.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.769+0000 7f0e237fe700 1 --2- 172.21.15.156:0/3936079146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e240cbdf0 0x7f0e2406b380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e140098f0 tx=0x7f0e14009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=228d8ee76d0c2a83 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.769+0000 7f0e22ffd700 1 -- 172.21.15.156:0/3936079146 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14011830 con 0x7f0e240cbdf0 2023-11-28T11:10:57.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.769+0000 7f0e22ffd700 1 -- 172.21.15.156:0/3936079146 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e14011990 con 0x7f0e240cbdf0 2023-11-28T11:10:57.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.769+0000 7f0e22ffd700 1 -- 172.21.15.156:0/3936079146 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14011d20 con 0x7f0e240cbdf0 2023-11-28T11:10:57.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.769+0000 7f0e2a265700 1 -- 172.21.15.156:0/3936079146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e240cbdf0 msgr2=0x7f0e2406b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.769+0000 7f0e2a265700 1 --2- 172.21.15.156:0/3936079146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e240cbdf0 0x7f0e2406b380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e140098f0 tx=0x7f0e14009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:57.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.769+0000 7f0e2a265700 1 -- 172.21.15.156:0/3936079146 shutdown_connections 2023-11-28T11:10:57.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.769+0000 7f0e2a265700 1 --2- 172.21.15.156:0/3936079146 >> v2:172.21.15.186:3300/0 conn(0x7f0e2415cef0 0x7f0e240af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.769+0000 7f0e2a265700 1 --2- 172.21.15.156:0/3936079146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e240cbdf0 0x7f0e2406b380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.769+0000 7f0e2a265700 1 -- 172.21.15.156:0/3936079146 >> 172.21.15.156:0/3936079146 conn(0x7f0e240d0000 msgr2=0x7f0e240d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:57.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.770+0000 7f0e2a265700 1 -- 172.21.15.156:0/3936079146 shutdown_connections 2023-11-28T11:10:57.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.770+0000 7f0e2a265700 1 -- 172.21.15.156:0/3936079146 wait complete. 2023-11-28T11:10:57.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.771+0000 7f0e2a265700 1 Processor -- start 2023-11-28T11:10:57.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.771+0000 7f0e2a265700 1 -- start start 2023-11-28T11:10:57.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e2a265700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e2406b8c0 0x7f0e240ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e2a265700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e240cbdf0 0x7f0e2415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e2a265700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e2415cef0 0x7f0e2406dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e2a265700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e241f34a0 con 0x7f0e240cbdf0 2023-11-28T11:10:57.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e2a265700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e240ca810 con 0x7f0e2406b8c0 2023-11-28T11:10:57.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e2a265700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e241f2920 con 0x7f0e2415cef0 2023-11-28T11:10:57.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e237fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e240cbdf0 0x7f0e2415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e237fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e240cbdf0 0x7f0e2415b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35744/0 (socket says 172.21.15.156:35744) 2023-11-28T11:10:57.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e237fe700 1 -- 172.21.15.156:0/2644019165 learned_addr learned my addr 172.21.15.156:0/2644019165 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:10:57.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e23fff700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e2406b8c0 0x7f0e240ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e28802700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e2415cef0 0x7f0e2406dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e237fe700 1 -- 172.21.15.156:0/2644019165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e2415cef0 msgr2=0x7f0e2406dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e237fe700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e2415cef0 0x7f0e2406dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e237fe700 1 -- 172.21.15.156:0/2644019165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e2406b8c0 msgr2=0x7f0e240ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e237fe700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e2406b8c0 0x7f0e240ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e237fe700 1 -- 172.21.15.156:0/2644019165 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180094a0 con 0x7f0e240cbdf0 2023-11-28T11:10:57.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e237fe700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e240cbdf0 0x7f0e2415b390 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0e14000c00 tx=0x7f0e14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e20ff9700 1 -- 172.21.15.156:0/2644019165 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14004300 con 0x7f0e240cbdf0 2023-11-28T11:10:57.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e20ff9700 1 -- 172.21.15.156:0/2644019165 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1401f070 con 0x7f0e240cbdf0 2023-11-28T11:10:57.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e2a265700 1 -- 172.21.15.156:0/2644019165 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e14009550 con 0x7f0e240cbdf0 2023-11-28T11:10:57.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e20ff9700 1 -- 172.21.15.156:0/2644019165 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140036a0 con 0x7f0e240cbdf0 2023-11-28T11:10:57.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.772+0000 7f0e2a265700 1 -- 172.21.15.156:0/2644019165 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2415dd10 con 0x7f0e240cbdf0 2023-11-28T11:10:57.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.773+0000 7f0e2a265700 1 -- 172.21.15.156:0/2644019165 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2415da30 con 0x7f0e240cbdf0 2023-11-28T11:10:57.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.774+0000 7f0e20ff9700 1 -- 172.21.15.156:0/2644019165 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0e1402a020 con 0x7f0e240cbdf0 2023-11-28T11:10:57.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.774+0000 7f0e20ff9700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0e100682f0 0x7f0e1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.774+0000 7f0e20ff9700 1 -- 172.21.15.156:0/2644019165 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e14087200 con 0x7f0e240cbdf0 2023-11-28T11:10:57.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.774+0000 7f0e23fff700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0e100682f0 0x7f0e1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.775+0000 7f0e23fff700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0e100682f0 0x7f0e1006a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009cd0 tx=0x7f0e18009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.779+0000 7f0e20ff9700 1 -- 172.21.15.156:0/2644019165 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0e14053980 con 0x7f0e240cbdf0 2023-11-28T11:10:57.866 INFO:tasks.ceph.mds.d.smithi186.stderr:2023-11-28T11:10:57.865+0000 7f634dda2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:10:57.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.956+0000 7f0e2a265700 1 -- 172.21.15.156:0/2644019165 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e2415dff0 con 0x7f0e100682f0 2023-11-28T11:10:57.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.964+0000 7f0e20ff9700 1 -- 172.21.15.156:0/2644019165 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308201 (secure 0 0 0) 0x7f0e2415dff0 con 0x7f0e100682f0 2023-11-28T11:10:57.969 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:10:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.967+0000 7f0e2a265700 1 -- 172.21.15.156:0/2644019165 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0e100682f0 msgr2=0x7f0e1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.967+0000 7f0e2a265700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0e100682f0 0x7f0e1006a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009cd0 tx=0x7f0e18009210 comp rx=0 tx=0).stop 2023-11-28T11:10:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.967+0000 7f0e2a265700 1 -- 172.21.15.156:0/2644019165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e240cbdf0 msgr2=0x7f0e2415b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.967+0000 7f0e2a265700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e240cbdf0 0x7f0e2415b390 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0e14000c00 tx=0x7f0e14004750 comp rx=0 tx=0).stop 2023-11-28T11:10:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.967+0000 7f0e2a265700 1 -- 172.21.15.156:0/2644019165 shutdown_connections 2023-11-28T11:10:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.967+0000 7f0e2a265700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f0e100682f0 0x7f0e1006a7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.967+0000 7f0e2a265700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e2415cef0 0x7f0e2406dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.967+0000 7f0e2a265700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e2406b8c0 0x7f0e240ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.967+0000 7f0e2a265700 1 --2- 172.21.15.156:0/2644019165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e240cbdf0 0x7f0e2415b390 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.967+0000 7f0e2a265700 1 -- 172.21.15.156:0/2644019165 >> 172.21.15.156:0/2644019165 conn(0x7f0e240d0000 msgr2=0x7f0e240cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:57.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.968+0000 7f0e2a265700 1 -- 172.21.15.156:0/2644019165 shutdown_connections 2023-11-28T11:10:57.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:10:57.968+0000 7f0e2a265700 1 -- 172.21.15.156:0/2644019165 wait complete. 2023-11-28T11:10:57.973 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:10:57.981 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2023-11-28T11:10:56.560054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590562,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82420,"kb_used_data":7028,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":754892300,"statfs":{"total":773094113280,"available":773009715200,"internally_reserved":0,"allocated":7196672,"data_stored":3807466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":194,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.588339"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848556+0000","last_change":"2023-11-28T11:10:52.399647+0000","last_active":"2023-11-28T11:10:53.848556+0000","last_peered":"2023-11-28T11:10:53.848556+0000","last_clean":"2023-11-28T11:10:53.848556+0000","last_became_active":"2023-11-28T11:10:52.399461+0000","last_became_peered":"2023-11-28T11:10:52.399461+0000","last_unstale":"2023-11-28T11:10:53.848556+0000","last_undegraded":"2023-11-28T11:10:53.848556+0000","last_fullsized":"2023-11-28T11:10:53.848556+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:36.282716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848853+0000","last_change":"2023-11-28T11:10:52.390854+0000","last_active":"2023-11-28T11:10:53.848853+0000","last_peered":"2023-11-28T11:10:53.848853+0000","last_clean":"2023-11-28T11:10:53.848853+0000","last_became_active":"2023-11-28T11:10:52.390541+0000","last_became_peered":"2023-11-28T11:10:52.390541+0000","last_unstale":"2023-11-28T11:10:53.848853+0000","last_undegraded":"2023-11-28T11:10:53.848853+0000","last_fullsized":"2023-11-28T11:10:53.848853+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:06.203049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377143+0000","last_change":"2023-11-28T11:10:52.394457+0000","last_active":"2023-11-28T11:10:53.377143+0000","last_peered":"2023-11-28T11:10:53.377143+0000","last_clean":"2023-11-28T11:10:53.377143+0000","last_became_active":"2023-11-28T11:10:52.394170+0000","last_became_peered":"2023-11-28T11:10:52.394170+0000","last_unstale":"2023-11-28T11:10:53.377143+0000","last_undegraded":"2023-11-28T11:10:53.377143+0000","last_fullsized":"2023-11-28T11:10:53.377143+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:12.029150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848451+0000","last_change":"2023-11-28T11:10:52.398045+0000","last_active":"2023-11-28T11:10:53.848451+0000","last_peered":"2023-11-28T11:10:53.848451+0000","last_clean":"2023-11-28T11:10:53.848451+0000","last_became_active":"2023-11-28T11:10:52.397753+0000","last_became_peered":"2023-11-28T11:10:52.397753+0000","last_unstale":"2023-11-28T11:10:53.848451+0000","last_undegraded":"2023-11-28T11:10:53.848451+0000","last_fullsized":"2023-11-28T11:10:53.848451+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:35.944815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847689+0000","last_change":"2023-11-28T11:10:52.397026+0000","last_active":"2023-11-28T11:10:53.847689+0000","last_peered":"2023-11-28T11:10:53.847689+0000","last_clean":"2023-11-28T11:10:53.847689+0000","last_became_active":"2023-11-28T11:10:52.396742+0000","last_became_peered":"2023-11-28T11:10:52.396742+0000","last_unstale":"2023-11-28T11:10:53.847689+0000","last_undegraded":"2023-11-28T11:10:53.847689+0000","last_fullsized":"2023-11-28T11:10:53.847689+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:00.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848795+0000","last_change":"2023-11-28T11:10:52.401622+0000","last_active":"2023-11-28T11:10:53.848795+0000","last_peered":"2023-11-28T11:10:53.848795+0000","last_clean":"2023-11-28T11:10:53.848795+0000","last_became_active":"2023-11-28T11:10:52.401310+0000","last_became_peered":"2023-11-28T11:10:52.401310+0000","last_unstale":"2023-11-28T11:10:53.848795+0000","last_undegraded":"2023-11-28T11:10:53.848795+0000","last_fullsized":"2023-11-28T11:10:53.848795+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:23.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848431+0000","last_change":"2023-11-28T11:10:52.398550+0000","last_active":"2023-11-28T11:10:53.848431+0000","last_peered":"2023-11-28T11:10:53.848431+0000","last_clean":"2023-11-28T11:10:53.848431+0000","last_became_active":"2023-11-28T11:10:52.398277+0000","last_became_peered":"2023-11-28T11:10:52.398277+0000","last_unstale":"2023-11-28T11:10:53.848431+0000","last_undegraded":"2023-11-28T11:10:53.848431+0000","last_fullsized":"2023-11-28T11:10:53.848431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:53.807072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848691+0000","last_change":"2023-11-28T11:10:52.396844+0000","last_active":"2023-11-28T11:10:53.848691+0000","last_peered":"2023-11-28T11:10:53.848691+0000","last_clean":"2023-11-28T11:10:53.848691+0000","last_became_active":"2023-11-28T11:10:52.393214+0000","last_became_peered":"2023-11-28T11:10:52.393214+0000","last_unstale":"2023-11-28T11:10:53.848691+0000","last_undegraded":"2023-11-28T11:10:53.848691+0000","last_fullsized":"2023-11-28T11:10:53.848691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:12.100298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848063+0000","last_change":"2023-11-28T11:10:52.397847+0000","last_active":"2023-11-28T11:10:53.848063+0000","last_peered":"2023-11-28T11:10:53.848063+0000","last_clean":"2023-11-28T11:10:53.848063+0000","last_became_active":"2023-11-28T11:10:52.397580+0000","last_became_peered":"2023-11-28T11:10:52.397580+0000","last_unstale":"2023-11-28T11:10:53.848063+0000","last_undegraded":"2023-11-28T11:10:53.848063+0000","last_fullsized":"2023-11-28T11:10:53.848063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:11:29.529381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848017+0000","last_change":"2023-11-28T11:10:52.397544+0000","last_active":"2023-11-28T11:10:53.848017+0000","last_peered":"2023-11-28T11:10:53.848017+0000","last_clean":"2023-11-28T11:10:53.848017+0000","last_became_active":"2023-11-28T11:10:52.397232+0000","last_became_peered":"2023-11-28T11:10:52.397232+0000","last_unstale":"2023-11-28T11:10:53.848017+0000","last_undegraded":"2023-11-28T11:10:53.848017+0000","last_fullsized":"2023-11-28T11:10:53.848017+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:33.270658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847532+0000","last_change":"2023-11-28T11:10:52.388564+0000","last_active":"2023-11-28T11:10:53.847532+0000","last_peered":"2023-11-28T11:10:53.847532+0000","last_clean":"2023-11-28T11:10:53.847532+0000","last_became_active":"2023-11-28T11:10:52.388307+0000","last_became_peered":"2023-11-28T11:10:52.388307+0000","last_unstale":"2023-11-28T11:10:53.847532+0000","last_undegraded":"2023-11-28T11:10:53.847532+0000","last_fullsized":"2023-11-28T11:10:53.847532+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:54.802666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848635+0000","last_change":"2023-11-28T11:10:52.391388+0000","last_active":"2023-11-28T11:10:53.848635+0000","last_peered":"2023-11-28T11:10:53.848635+0000","last_clean":"2023-11-28T11:10:53.848635+0000","last_became_active":"2023-11-28T11:10:52.391076+0000","last_became_peered":"2023-11-28T11:10:52.391076+0000","last_unstale":"2023-11-28T11:10:53.848635+0000","last_undegraded":"2023-11-28T11:10:53.848635+0000","last_fullsized":"2023-11-28T11:10:53.848635+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:05:43.320524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376913+0000","last_change":"2023-11-28T11:10:52.380958+0000","last_active":"2023-11-28T11:10:53.376913+0000","last_peered":"2023-11-28T11:10:53.376913+0000","last_clean":"2023-11-28T11:10:53.376913+0000","last_became_active":"2023-11-28T11:10:52.380688+0000","last_became_peered":"2023-11-28T11:10:52.380688+0000","last_unstale":"2023-11-28T11:10:53.376913+0000","last_undegraded":"2023-11-28T11:10:53.376913+0000","last_fullsized":"2023-11-28T11:10:53.376913+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:22.866138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375117+0000","last_change":"2023-11-28T11:10:52.387375+0000","last_active":"2023-11-28T11:10:53.375117+0000","last_peered":"2023-11-28T11:10:53.375117+0000","last_clean":"2023-11-28T11:10:53.375117+0000","last_became_active":"2023-11-28T11:10:52.387088+0000","last_became_peered":"2023-11-28T11:10:52.387088+0000","last_unstale":"2023-11-28T11:10:53.375117+0000","last_undegraded":"2023-11-28T11:10:53.375117+0000","last_fullsized":"2023-11-28T11:10:53.375117+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:54.810083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375344+0000","last_change":"2023-11-28T11:10:52.398710+0000","last_active":"2023-11-28T11:10:53.375344+0000","last_peered":"2023-11-28T11:10:53.375344+0000","last_clean":"2023-11-28T11:10:53.375344+0000","last_became_active":"2023-11-28T11:10:52.398388+0000","last_became_peered":"2023-11-28T11:10:52.398388+0000","last_unstale":"2023-11-28T11:10:53.375344+0000","last_undegraded":"2023-11-28T11:10:53.375344+0000","last_fullsized":"2023-11-28T11:10:53.375344+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:09.520497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.374950+0000","last_change":"2023-11-28T11:10:52.398644+0000","last_active":"2023-11-28T11:10:53.374950+0000","last_peered":"2023-11-28T11:10:53.374950+0000","last_clean":"2023-11-28T11:10:53.374950+0000","last_became_active":"2023-11-28T11:10:52.398342+0000","last_became_peered":"2023-11-28T11:10:52.398342+0000","last_unstale":"2023-11-28T11:10:53.374950+0000","last_undegraded":"2023-11-28T11:10:53.374950+0000","last_fullsized":"2023-11-28T11:10:53.374950+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:02.775343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848270+0000","last_change":"2023-11-28T11:10:52.400926+0000","last_active":"2023-11-28T11:10:53.848270+0000","last_peered":"2023-11-28T11:10:53.848270+0000","last_clean":"2023-11-28T11:10:53.848270+0000","last_became_active":"2023-11-28T11:10:52.400667+0000","last_became_peered":"2023-11-28T11:10:52.400667+0000","last_unstale":"2023-11-28T11:10:53.848270+0000","last_undegraded":"2023-11-28T11:10:53.848270+0000","last_fullsized":"2023-11-28T11:10:53.848270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:06.203049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848749+0000","last_change":"2023-11-28T11:10:52.390919+0000","last_active":"2023-11-28T11:10:53.848749+0000","last_peered":"2023-11-28T11:10:53.848749+0000","last_clean":"2023-11-28T11:10:53.848749+0000","last_became_active":"2023-11-28T11:10:52.390435+0000","last_became_peered":"2023-11-28T11:10:52.390435+0000","last_unstale":"2023-11-28T11:10:53.848749+0000","last_undegraded":"2023-11-28T11:10:53.848749+0000","last_fullsized":"2023-11-28T11:10:53.848749+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:48.226111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848092+0000","last_change":"2023-11-28T11:10:52.397318+0000","last_active":"2023-11-28T11:10:53.848092+0000","last_peered":"2023-11-28T11:10:53.848092+0000","last_clean":"2023-11-28T11:10:53.848092+0000","last_became_active":"2023-11-28T11:10:52.397039+0000","last_became_peered":"2023-11-28T11:10:52.397039+0000","last_unstale":"2023-11-28T11:10:53.848092+0000","last_undegraded":"2023-11-28T11:10:53.848092+0000","last_fullsized":"2023-11-28T11:10:53.848092+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:54.810083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847782+0000","last_change":"2023-11-28T11:10:52.394969+0000","last_active":"2023-11-28T11:10:53.847782+0000","last_peered":"2023-11-28T11:10:53.847782+0000","last_clean":"2023-11-28T11:10:53.847782+0000","last_became_active":"2023-11-28T11:10:52.394620+0000","last_became_peered":"2023-11-28T11:10:52.394620+0000","last_unstale":"2023-11-28T11:10:53.847782+0000","last_undegraded":"2023-11-28T11:10:53.847782+0000","last_fullsized":"2023-11-28T11:10:53.847782+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:28.345284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847322+0000","last_change":"2023-11-28T11:10:52.382159+0000","last_active":"2023-11-28T11:10:53.847322+0000","last_peered":"2023-11-28T11:10:53.847322+0000","last_clean":"2023-11-28T11:10:53.847322+0000","last_became_active":"2023-11-28T11:10:52.381757+0000","last_became_peered":"2023-11-28T11:10:52.381757+0000","last_unstale":"2023-11-28T11:10:53.847322+0000","last_undegraded":"2023-11-28T11:10:53.847322+0000","last_fullsized":"2023-11-28T11:10:53.847322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:21.448143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377057+0000","last_change":"2023-11-28T11:10:52.389365+0000","last_active":"2023-11-28T11:10:53.377057+0000","last_peered":"2023-11-28T11:10:53.377057+0000","last_clean":"2023-11-28T11:10:53.377057+0000","last_became_active":"2023-11-28T11:10:52.388931+0000","last_became_peered":"2023-11-28T11:10:52.388931+0000","last_unstale":"2023-11-28T11:10:53.377057+0000","last_undegraded":"2023-11-28T11:10:53.377057+0000","last_fullsized":"2023-11-28T11:10:53.377057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:30.962468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978200+0000","last_change":"2023-11-28T11:10:50.978200+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978200+0000","last_undegraded":"2023-11-28T11:10:50.978200+0000","last_fullsized":"2023-11-28T11:10:50.978200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:23.426647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848809+0000","last_change":"2023-11-28T11:10:52.390732+0000","last_active":"2023-11-28T11:10:53.848809+0000","last_peered":"2023-11-28T11:10:53.848809+0000","last_clean":"2023-11-28T11:10:53.848809+0000","last_became_active":"2023-11-28T11:10:52.390111+0000","last_became_peered":"2023-11-28T11:10:52.390111+0000","last_unstale":"2023-11-28T11:10:53.848809+0000","last_undegraded":"2023-11-28T11:10:53.848809+0000","last_fullsized":"2023-11-28T11:10:53.848809+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:18.651068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.378205+0000","last_change":"2023-11-28T11:10:51.995120+0000","last_active":"2023-11-28T11:10:52.378205+0000","last_peered":"2023-11-28T11:10:52.378205+0000","last_clean":"2023-11-28T11:10:52.378205+0000","last_became_active":"2023-11-28T11:10:51.994836+0000","last_became_peered":"2023-11-28T11:10:51.994836+0000","last_unstale":"2023-11-28T11:10:52.378205+0000","last_undegraded":"2023-11-28T11:10:52.378205+0000","last_fullsized":"2023-11-28T11:10:52.378205+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:32.263075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848457+0000","last_change":"2023-11-28T11:10:52.385151+0000","last_active":"2023-11-28T11:10:53.848457+0000","last_peered":"2023-11-28T11:10:53.848457+0000","last_clean":"2023-11-28T11:10:53.848457+0000","last_became_active":"2023-11-28T11:10:52.381849+0000","last_became_peered":"2023-11-28T11:10:52.381849+0000","last_unstale":"2023-11-28T11:10:53.848457+0000","last_undegraded":"2023-11-28T11:10:53.848457+0000","last_fullsized":"2023-11-28T11:10:53.848457+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:02.775343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.384672+0000","last_change":"2023-11-28T11:10:52.000037+0000","last_active":"2023-11-28T11:10:52.384672+0000","last_peered":"2023-11-28T11:10:52.384672+0000","last_clean":"2023-11-28T11:10:52.384672+0000","last_became_active":"2023-11-28T11:10:51.999788+0000","last_became_peered":"2023-11-28T11:10:51.999788+0000","last_unstale":"2023-11-28T11:10:52.384672+0000","last_undegraded":"2023-11-28T11:10:52.384672+0000","last_fullsized":"2023-11-28T11:10:52.384672+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:52.799489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848659+0000","last_change":"2023-11-28T11:10:52.399189+0000","last_active":"2023-11-28T11:10:53.848659+0000","last_peered":"2023-11-28T11:10:53.848659+0000","last_clean":"2023-11-28T11:10:53.848659+0000","last_became_active":"2023-11-28T11:10:52.398894+0000","last_became_peered":"2023-11-28T11:10:52.398894+0000","last_unstale":"2023-11-28T11:10:53.848659+0000","last_undegraded":"2023-11-28T11:10:53.848659+0000","last_fullsized":"2023-11-28T11:10:53.848659+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:09.926232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.381856+0000","last_change":"2023-11-28T11:10:51.990812+0000","last_active":"2023-11-28T11:10:52.381856+0000","last_peered":"2023-11-28T11:10:52.381856+0000","last_clean":"2023-11-28T11:10:52.381856+0000","last_became_active":"2023-11-28T11:10:51.990539+0000","last_became_peered":"2023-11-28T11:10:51.990539+0000","last_unstale":"2023-11-28T11:10:52.381856+0000","last_undegraded":"2023-11-28T11:10:52.381856+0000","last_fullsized":"2023-11-28T11:10:52.381856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:06.486410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847570+0000","last_change":"2023-11-28T11:10:52.381091+0000","last_active":"2023-11-28T11:10:53.847570+0000","last_peered":"2023-11-28T11:10:53.847570+0000","last_clean":"2023-11-28T11:10:53.847570+0000","last_became_active":"2023-11-28T11:10:52.380839+0000","last_became_peered":"2023-11-28T11:10:52.380839+0000","last_unstale":"2023-11-28T11:10:53.847570+0000","last_undegraded":"2023-11-28T11:10:53.847570+0000","last_fullsized":"2023-11-28T11:10:53.847570+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:29:30.173766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.375838+0000","last_change":"2023-11-28T11:10:51.985814+0000","last_active":"2023-11-28T11:10:52.375838+0000","last_peered":"2023-11-28T11:10:52.375838+0000","last_clean":"2023-11-28T11:10:52.375838+0000","last_became_active":"2023-11-28T11:10:51.985536+0000","last_became_peered":"2023-11-28T11:10:51.985536+0000","last_unstale":"2023-11-28T11:10:52.375838+0000","last_undegraded":"2023-11-28T11:10:52.375838+0000","last_fullsized":"2023-11-28T11:10:52.375838+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:11.092714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377014+0000","last_change":"2023-11-28T11:10:51.984690+0000","last_active":"2023-11-28T11:10:52.377014+0000","last_peered":"2023-11-28T11:10:52.377014+0000","last_clean":"2023-11-28T11:10:52.377014+0000","last_became_active":"2023-11-28T11:10:51.984385+0000","last_became_peered":"2023-11-28T11:10:51.984385+0000","last_unstale":"2023-11-28T11:10:52.377014+0000","last_undegraded":"2023-11-28T11:10:52.377014+0000","last_fullsized":"2023-11-28T11:10:52.377014+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:06.486410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376774+0000","last_change":"2023-11-28T11:10:52.390623+0000","last_active":"2023-11-28T11:10:53.376774+0000","last_peered":"2023-11-28T11:10:53.376774+0000","last_clean":"2023-11-28T11:10:53.376774+0000","last_became_active":"2023-11-28T11:10:52.390344+0000","last_became_peered":"2023-11-28T11:10:52.390344+0000","last_unstale":"2023-11-28T11:10:53.376774+0000","last_undegraded":"2023-11-28T11:10:53.376774+0000","last_fullsized":"2023-11-28T11:10:53.376774+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:45.616308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.370871+0000","last_change":"2023-11-28T11:10:51.991786+0000","last_active":"2023-11-28T11:10:52.370871+0000","last_peered":"2023-11-28T11:10:52.370871+0000","last_clean":"2023-11-28T11:10:52.370871+0000","last_became_active":"2023-11-28T11:10:51.991493+0000","last_became_peered":"2023-11-28T11:10:51.991493+0000","last_unstale":"2023-11-28T11:10:52.370871+0000","last_undegraded":"2023-11-28T11:10:52.370871+0000","last_fullsized":"2023-11-28T11:10:52.370871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:21.757386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375324+0000","last_change":"2023-11-28T11:10:52.398474+0000","last_active":"2023-11-28T11:10:53.375324+0000","last_peered":"2023-11-28T11:10:53.375324+0000","last_clean":"2023-11-28T11:10:53.375324+0000","last_became_active":"2023-11-28T11:10:52.398155+0000","last_became_peered":"2023-11-28T11:10:52.398155+0000","last_unstale":"2023-11-28T11:10:53.375324+0000","last_undegraded":"2023-11-28T11:10:53.375324+0000","last_fullsized":"2023-11-28T11:10:53.375324+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:23.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.383141+0000","last_change":"2023-11-28T11:10:51.995022+0000","last_active":"2023-11-28T11:10:52.383141+0000","last_peered":"2023-11-28T11:10:52.383141+0000","last_clean":"2023-11-28T11:10:52.383141+0000","last_became_active":"2023-11-28T11:10:51.994731+0000","last_became_peered":"2023-11-28T11:10:51.994731+0000","last_unstale":"2023-11-28T11:10:52.383141+0000","last_undegraded":"2023-11-28T11:10:52.383141+0000","last_fullsized":"2023-11-28T11:10:52.383141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:31:32.827101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.395098+0000","last_change":"2023-11-28T11:10:51.998518+0000","last_active":"2023-11-28T11:10:52.395098+0000","last_peered":"2023-11-28T11:10:52.395098+0000","last_clean":"2023-11-28T11:10:52.395098+0000","last_became_active":"2023-11-28T11:10:51.998086+0000","last_became_peered":"2023-11-28T11:10:51.998086+0000","last_unstale":"2023-11-28T11:10:52.395098+0000","last_undegraded":"2023-11-28T11:10:52.395098+0000","last_fullsized":"2023-11-28T11:10:52.395098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:21:56.126127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375245+0000","last_change":"2023-11-28T11:10:52.383474+0000","last_active":"2023-11-28T11:10:53.375245+0000","last_peered":"2023-11-28T11:10:53.375245+0000","last_clean":"2023-11-28T11:10:53.375245+0000","last_became_active":"2023-11-28T11:10:52.383211+0000","last_became_peered":"2023-11-28T11:10:52.383211+0000","last_unstale":"2023-11-28T11:10:53.375245+0000","last_undegraded":"2023-11-28T11:10:53.375245+0000","last_fullsized":"2023-11-28T11:10:53.375245+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:48.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.384146+0000","last_change":"2023-11-28T11:10:51.997551+0000","last_active":"2023-11-28T11:10:52.384146+0000","last_peered":"2023-11-28T11:10:52.384146+0000","last_clean":"2023-11-28T11:10:52.384146+0000","last_became_active":"2023-11-28T11:10:51.997278+0000","last_became_peered":"2023-11-28T11:10:51.997278+0000","last_unstale":"2023-11-28T11:10:52.384146+0000","last_undegraded":"2023-11-28T11:10:52.384146+0000","last_fullsized":"2023-11-28T11:10:52.384146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:21.858555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377081+0000","last_change":"2023-11-28T11:10:52.393694+0000","last_active":"2023-11-28T11:10:53.377081+0000","last_peered":"2023-11-28T11:10:53.377081+0000","last_clean":"2023-11-28T11:10:53.377081+0000","last_became_active":"2023-11-28T11:10:52.393435+0000","last_became_peered":"2023-11-28T11:10:52.393435+0000","last_unstale":"2023-11-28T11:10:53.377081+0000","last_undegraded":"2023-11-28T11:10:53.377081+0000","last_fullsized":"2023-11-28T11:10:53.377081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:05:25.141120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"62'3","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.399587+0000","last_change":"2023-11-28T11:10:51.997311+0000","last_active":"2023-11-28T11:10:52.399587+0000","last_peered":"2023-11-28T11:10:52.399587+0000","last_clean":"2023-11-28T11:10:52.399587+0000","last_became_active":"2023-11-28T11:10:51.997038+0000","last_became_peered":"2023-11-28T11:10:51.997038+0000","last_unstale":"2023-11-28T11:10:52.399587+0000","last_undegraded":"2023-11-28T11:10:52.399587+0000","last_fullsized":"2023-11-28T11:10:52.399587+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:47.034924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848569+0000","last_change":"2023-11-28T11:10:52.393378+0000","last_active":"2023-11-28T11:10:53.848569+0000","last_peered":"2023-11-28T11:10:53.848569+0000","last_clean":"2023-11-28T11:10:53.848569+0000","last_became_active":"2023-11-28T11:10:52.393089+0000","last_became_peered":"2023-11-28T11:10:52.393089+0000","last_unstale":"2023-11-28T11:10:53.848569+0000","last_undegraded":"2023-11-28T11:10:53.848569+0000","last_fullsized":"2023-11-28T11:10:53.848569+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:22.764970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978374+0000","last_change":"2023-11-28T11:10:50.978374+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978374+0000","last_undegraded":"2023-11-28T11:10:50.978374+0000","last_fullsized":"2023-11-28T11:10:50.978374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:41:07.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847717+0000","last_change":"2023-11-28T11:10:52.402741+0000","last_active":"2023-11-28T11:10:53.847717+0000","last_peered":"2023-11-28T11:10:53.847717+0000","last_clean":"2023-11-28T11:10:53.847717+0000","last_became_active":"2023-11-28T11:10:52.402537+0000","last_became_peered":"2023-11-28T11:10:52.402537+0000","last_unstale":"2023-11-28T11:10:53.847717+0000","last_undegraded":"2023-11-28T11:10:53.847717+0000","last_fullsized":"2023-11-28T11:10:53.847717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:48.042507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.403563+0000","last_change":"2023-11-28T11:10:51.999231+0000","last_active":"2023-11-28T11:10:52.403563+0000","last_peered":"2023-11-28T11:10:52.403563+0000","last_clean":"2023-11-28T11:10:52.403563+0000","last_became_active":"2023-11-28T11:10:51.998951+0000","last_became_peered":"2023-11-28T11:10:51.998951+0000","last_unstale":"2023-11-28T11:10:52.403563+0000","last_undegraded":"2023-11-28T11:10:52.403563+0000","last_fullsized":"2023-11-28T11:10:52.403563+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:27.337701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.981250+0000","last_change":"2023-11-28T11:10:50.981250+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.981250+0000","last_undegraded":"2023-11-28T11:10:50.981250+0000","last_fullsized":"2023-11-28T11:10:50.981250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:29.954884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377049+0000","last_change":"2023-11-28T11:10:52.383141+0000","last_active":"2023-11-28T11:10:53.377049+0000","last_peered":"2023-11-28T11:10:53.377049+0000","last_clean":"2023-11-28T11:10:53.377049+0000","last_became_active":"2023-11-28T11:10:52.382460+0000","last_became_peered":"2023-11-28T11:10:52.382460+0000","last_unstale":"2023-11-28T11:10:53.377049+0000","last_undegraded":"2023-11-28T11:10:53.377049+0000","last_fullsized":"2023-11-28T11:10:53.377049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:24.111335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.393434+0000","last_change":"2023-11-28T11:10:51.999127+0000","last_active":"2023-11-28T11:10:52.393434+0000","last_peered":"2023-11-28T11:10:52.393434+0000","last_clean":"2023-11-28T11:10:52.393434+0000","last_became_active":"2023-11-28T11:10:51.998846+0000","last_became_peered":"2023-11-28T11:10:51.998846+0000","last_unstale":"2023-11-28T11:10:52.393434+0000","last_undegraded":"2023-11-28T11:10:52.393434+0000","last_fullsized":"2023-11-28T11:10:52.393434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:41:07.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"62'8","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.402319+0000","last_change":"2023-11-28T11:10:51.996381+0000","last_active":"2023-11-28T11:10:52.402319+0000","last_peered":"2023-11-28T11:10:52.402319+0000","last_clean":"2023-11-28T11:10:52.402319+0000","last_became_active":"2023-11-28T11:10:51.996095+0000","last_became_peered":"2023-11-28T11:10:51.996095+0000","last_unstale":"2023-11-28T11:10:52.402319+0000","last_undegraded":"2023-11-28T11:10:52.402319+0000","last_fullsized":"2023-11-28T11:10:52.402319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:38.465853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847836+0000","last_change":"2023-11-28T11:10:52.394438+0000","last_active":"2023-11-28T11:10:53.847836+0000","last_peered":"2023-11-28T11:10:53.847836+0000","last_clean":"2023-11-28T11:10:53.847836+0000","last_became_active":"2023-11-28T11:10:52.394191+0000","last_became_peered":"2023-11-28T11:10:52.394191+0000","last_unstale":"2023-11-28T11:10:53.847836+0000","last_undegraded":"2023-11-28T11:10:53.847836+0000","last_fullsized":"2023-11-28T11:10:53.847836+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:35.306404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978466+0000","last_change":"2023-11-28T11:10:50.978466+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978466+0000","last_undegraded":"2023-11-28T11:10:50.978466+0000","last_fullsized":"2023-11-28T11:10:50.978466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:44.608724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848232+0000","last_change":"2023-11-28T11:10:52.384083+0000","last_active":"2023-11-28T11:10:53.848232+0000","last_peered":"2023-11-28T11:10:53.848232+0000","last_clean":"2023-11-28T11:10:53.848232+0000","last_became_active":"2023-11-28T11:10:52.383213+0000","last_became_peered":"2023-11-28T11:10:52.383213+0000","last_unstale":"2023-11-28T11:10:53.848232+0000","last_undegraded":"2023-11-28T11:10:53.848232+0000","last_fullsized":"2023-11-28T11:10:53.848232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:00.199905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978752+0000","last_change":"2023-11-28T11:10:50.978752+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978752+0000","last_undegraded":"2023-11-28T11:10:50.978752+0000","last_fullsized":"2023-11-28T11:10:50.978752+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:35.847896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.373540+0000","last_change":"2023-11-28T11:10:52.000024+0000","last_active":"2023-11-28T11:10:52.373540+0000","last_peered":"2023-11-28T11:10:52.373540+0000","last_clean":"2023-11-28T11:10:52.373540+0000","last_became_active":"2023-11-28T11:10:51.999765+0000","last_became_peered":"2023-11-28T11:10:51.999765+0000","last_unstale":"2023-11-28T11:10:52.373540+0000","last_undegraded":"2023-11-28T11:10:52.373540+0000","last_fullsized":"2023-11-28T11:10:52.373540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:56.382110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376866+0000","last_change":"2023-11-28T11:10:52.388835+0000","last_active":"2023-11-28T11:10:53.376866+0000","last_peered":"2023-11-28T11:10:53.376866+0000","last_clean":"2023-11-28T11:10:53.376866+0000","last_became_active":"2023-11-28T11:10:52.388560+0000","last_became_peered":"2023-11-28T11:10:52.388560+0000","last_unstale":"2023-11-28T11:10:53.376866+0000","last_undegraded":"2023-11-28T11:10:53.376866+0000","last_fullsized":"2023-11-28T11:10:53.376866+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:03:08.782678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.977949+0000","last_change":"2023-11-28T11:10:50.977949+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.977949+0000","last_undegraded":"2023-11-28T11:10:50.977949+0000","last_fullsized":"2023-11-28T11:10:50.977949+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:11.021567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376607+0000","last_change":"2023-11-28T11:10:52.387164+0000","last_active":"2023-11-28T11:10:53.376607+0000","last_peered":"2023-11-28T11:10:53.376607+0000","last_clean":"2023-11-28T11:10:53.376607+0000","last_became_active":"2023-11-28T11:10:52.386857+0000","last_became_peered":"2023-11-28T11:10:52.386857+0000","last_unstale":"2023-11-28T11:10:53.376607+0000","last_undegraded":"2023-11-28T11:10:53.376607+0000","last_fullsized":"2023-11-28T11:10:53.376607+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:15:03.899294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.389892+0000","last_change":"2023-11-28T11:10:51.999035+0000","last_active":"2023-11-28T11:10:52.389892+0000","last_peered":"2023-11-28T11:10:52.389892+0000","last_clean":"2023-11-28T11:10:52.389892+0000","last_became_active":"2023-11-28T11:10:51.998758+0000","last_became_peered":"2023-11-28T11:10:51.998758+0000","last_unstale":"2023-11-28T11:10:52.389892+0000","last_undegraded":"2023-11-28T11:10:52.389892+0000","last_fullsized":"2023-11-28T11:10:52.389892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:11.021567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848172+0000","last_change":"2023-11-28T11:10:52.401133+0000","last_active":"2023-11-28T11:10:53.848172+0000","last_peered":"2023-11-28T11:10:53.848172+0000","last_clean":"2023-11-28T11:10:53.848172+0000","last_became_active":"2023-11-28T11:10:52.400869+0000","last_became_peered":"2023-11-28T11:10:52.400869+0000","last_unstale":"2023-11-28T11:10:53.848172+0000","last_undegraded":"2023-11-28T11:10:53.848172+0000","last_fullsized":"2023-11-28T11:10:53.848172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:38.996786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.371070+0000","last_change":"2023-11-28T11:10:51.989318+0000","last_active":"2023-11-28T11:10:52.371070+0000","last_peered":"2023-11-28T11:10:52.371070+0000","last_clean":"2023-11-28T11:10:52.371070+0000","last_became_active":"2023-11-28T11:10:51.988981+0000","last_became_peered":"2023-11-28T11:10:51.988981+0000","last_unstale":"2023-11-28T11:10:52.371070+0000","last_undegraded":"2023-11-28T11:10:52.371070+0000","last_fullsized":"2023-11-28T11:10:52.371070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:05.195466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.370973+0000","last_change":"2023-11-28T11:10:51.997542+0000","last_active":"2023-11-28T11:10:52.370973+0000","last_peered":"2023-11-28T11:10:52.370973+0000","last_clean":"2023-11-28T11:10:52.370973+0000","last_became_active":"2023-11-28T11:10:51.997247+0000","last_became_peered":"2023-11-28T11:10:51.997247+0000","last_unstale":"2023-11-28T11:10:52.370973+0000","last_undegraded":"2023-11-28T11:10:52.370973+0000","last_fullsized":"2023-11-28T11:10:52.370973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:35.988439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847922+0000","last_change":"2023-11-28T11:10:52.388406+0000","last_active":"2023-11-28T11:10:53.847922+0000","last_peered":"2023-11-28T11:10:53.847922+0000","last_clean":"2023-11-28T11:10:53.847922+0000","last_became_active":"2023-11-28T11:10:52.388066+0000","last_became_peered":"2023-11-28T11:10:52.388066+0000","last_unstale":"2023-11-28T11:10:53.847922+0000","last_undegraded":"2023-11-28T11:10:53.847922+0000","last_fullsized":"2023-11-28T11:10:53.847922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:43.460660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.977903+0000","last_change":"2023-11-28T11:10:50.977903+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.977903+0000","last_undegraded":"2023-11-28T11:10:50.977903+0000","last_fullsized":"2023-11-28T11:10:50.977903+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:37.815667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848272+0000","last_change":"2023-11-28T11:10:52.398149+0000","last_active":"2023-11-28T11:10:53.848272+0000","last_peered":"2023-11-28T11:10:53.848272+0000","last_clean":"2023-11-28T11:10:53.848272+0000","last_became_active":"2023-11-28T11:10:52.397875+0000","last_became_peered":"2023-11-28T11:10:52.397875+0000","last_unstale":"2023-11-28T11:10:53.848272+0000","last_undegraded":"2023-11-28T11:10:53.848272+0000","last_fullsized":"2023-11-28T11:10:53.848272+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:00.423890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.979017+0000","last_change":"2023-11-28T11:10:50.979017+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.979017+0000","last_undegraded":"2023-11-28T11:10:50.979017+0000","last_fullsized":"2023-11-28T11:10:50.979017+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:00.060672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.027168+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:10:48.027168+0000","last_peered":"2023-11-28T11:10:48.027168+0000","last_clean":"2023-11-28T11:10:48.027168+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:10:48.027168+0000","last_undegraded":"2023-11-28T11:10:48.027168+0000","last_fullsized":"2023-11-28T11:10:48.027168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377047+0000","last_change":"2023-11-28T11:10:52.387511+0000","last_active":"2023-11-28T11:10:53.377047+0000","last_peered":"2023-11-28T11:10:53.377047+0000","last_clean":"2023-11-28T11:10:53.377047+0000","last_became_active":"2023-11-28T11:10:52.387143+0000","last_became_peered":"2023-11-28T11:10:52.387143+0000","last_unstale":"2023-11-28T11:10:53.377047+0000","last_undegraded":"2023-11-28T11:10:53.377047+0000","last_fullsized":"2023-11-28T11:10:53.377047+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:01.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.373304+0000","last_change":"2023-11-28T11:10:51.995232+0000","last_active":"2023-11-28T11:10:52.373304+0000","last_peered":"2023-11-28T11:10:52.373304+0000","last_clean":"2023-11-28T11:10:52.373304+0000","last_became_active":"2023-11-28T11:10:51.994941+0000","last_became_peered":"2023-11-28T11:10:51.994941+0000","last_unstale":"2023-11-28T11:10:52.373304+0000","last_undegraded":"2023-11-28T11:10:52.373304+0000","last_fullsized":"2023-11-28T11:10:52.373304+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:11:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:47.963710+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:10:47.963710+0000","last_peered":"2023-11-28T11:10:47.963710+0000","last_clean":"2023-11-28T11:10:47.963710+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:10:47.963710+0000","last_undegraded":"2023-11-28T11:10:47.963710+0000","last_fullsized":"2023-11-28T11:10:47.963710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:51.971628+0000","last_change":"2023-11-28T11:10:51.971628+0000","last_active":"2023-11-28T11:10:51.971628+0000","last_peered":"2023-11-28T11:10:51.971628+0000","last_clean":"2023-11-28T11:10:51.971628+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:51.971628+0000","last_undegraded":"2023-11-28T11:10:51.971628+0000","last_fullsized":"2023-11-28T11:10:51.971628+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T11:10:50.978394+0000","last_change":"2023-11-28T11:10:50.978394+0000","last_active":"2023-11-28T11:10:50.964045+0000","last_peered":"2023-11-28T11:10:50.964045+0000","last_clean":"2023-11-28T11:10:50.964045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:50.978394+0000","last_undegraded":"2023-11-28T11:10:50.978394+0000","last_fullsized":"2023-11-28T11:10:50.978394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:38.465853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377553+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:10:52.377553+0000","last_peered":"2023-11-28T11:10:52.377553+0000","last_clean":"2023-11-28T11:10:52.377553+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:10:52.377553+0000","last_undegraded":"2023-11-28T11:10:52.377553+0000","last_fullsized":"2023-11-28T11:10:52.377553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377169+0000","last_change":"2023-11-28T11:10:52.387614+0000","last_active":"2023-11-28T11:10:53.377169+0000","last_peered":"2023-11-28T11:10:53.377169+0000","last_clean":"2023-11-28T11:10:53.377169+0000","last_became_active":"2023-11-28T11:10:52.387338+0000","last_became_peered":"2023-11-28T11:10:52.387338+0000","last_unstale":"2023-11-28T11:10:53.377169+0000","last_undegraded":"2023-11-28T11:10:53.377169+0000","last_fullsized":"2023-11-28T11:10:53.377169+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:36.855479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.370866+0000","last_change":"2023-11-28T11:10:51.985858+0000","last_active":"2023-11-28T11:10:52.370866+0000","last_peered":"2023-11-28T11:10:52.370866+0000","last_clean":"2023-11-28T11:10:52.370866+0000","last_became_active":"2023-11-28T11:10:51.985623+0000","last_became_peered":"2023-11-28T11:10:51.985623+0000","last_unstale":"2023-11-28T11:10:52.370866+0000","last_undegraded":"2023-11-28T11:10:52.370866+0000","last_fullsized":"2023-11-28T11:10:52.370866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:05:42.312941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:47.963710+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:10:47.963710+0000","last_peered":"2023-11-28T11:10:47.963710+0000","last_clean":"2023-11-28T11:10:47.963710+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:10:47.963710+0000","last_undegraded":"2023-11-28T11:10:47.963710+0000","last_fullsized":"2023-11-28T11:10:47.963710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":204,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.389688+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:10:52.389688+0000","last_peered":"2023-11-28T11:10:52.389688+0000","last_clean":"2023-11-28T11:10:52.389688+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:10:52.389688+0000","last_undegraded":"2023-11-28T11:10:52.389688+0000","last_fullsized":"2023-11-28T11:10:52.389688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847400+0000","last_change":"2023-11-28T11:10:52.381996+0000","last_active":"2023-11-28T11:10:53.847400+0000","last_peered":"2023-11-28T11:10:53.847400+0000","last_clean":"2023-11-28T11:10:53.847400+0000","last_became_active":"2023-11-28T11:10:52.381358+0000","last_became_peered":"2023-11-28T11:10:52.381358+0000","last_unstale":"2023-11-28T11:10:53.847400+0000","last_undegraded":"2023-11-28T11:10:53.847400+0000","last_fullsized":"2023-11-28T11:10:53.847400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:48.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.386941+0000","last_change":"2023-11-28T11:10:51.996342+0000","last_active":"2023-11-28T11:10:52.386941+0000","last_peered":"2023-11-28T11:10:52.386941+0000","last_clean":"2023-11-28T11:10:52.386941+0000","last_became_active":"2023-11-28T11:10:51.995997+0000","last_became_peered":"2023-11-28T11:10:51.995997+0000","last_unstale":"2023-11-28T11:10:52.386941+0000","last_undegraded":"2023-11-28T11:10:52.386941+0000","last_fullsized":"2023-11-28T11:10:52.386941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:35.988439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.389449+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:10:52.389449+0000","last_peered":"2023-11-28T11:10:52.389449+0000","last_clean":"2023-11-28T11:10:52.389449+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:10:52.389449+0000","last_undegraded":"2023-11-28T11:10:52.389449+0000","last_fullsized":"2023-11-28T11:10:52.389449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376952+0000","last_change":"2023-11-28T11:10:52.397798+0000","last_active":"2023-11-28T11:10:53.376952+0000","last_peered":"2023-11-28T11:10:53.376952+0000","last_clean":"2023-11-28T11:10:53.376952+0000","last_became_active":"2023-11-28T11:10:52.397532+0000","last_became_peered":"2023-11-28T11:10:52.397532+0000","last_unstale":"2023-11-28T11:10:53.376952+0000","last_undegraded":"2023-11-28T11:10:53.376952+0000","last_fullsized":"2023-11-28T11:10:53.376952+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:38.823250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.398098+0000","last_change":"2023-11-28T11:10:52.000553+0000","last_active":"2023-11-28T11:10:52.398098+0000","last_peered":"2023-11-28T11:10:52.398098+0000","last_clean":"2023-11-28T11:10:52.398098+0000","last_became_active":"2023-11-28T11:10:52.000223+0000","last_became_peered":"2023-11-28T11:10:52.000223+0000","last_unstale":"2023-11-28T11:10:52.398098+0000","last_undegraded":"2023-11-28T11:10:52.398098+0000","last_fullsized":"2023-11-28T11:10:52.398098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:37.815667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.391001+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:10:52.391001+0000","last_peered":"2023-11-28T11:10:52.391001+0000","last_clean":"2023-11-28T11:10:52.391001+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:10:52.391001+0000","last_undegraded":"2023-11-28T11:10:52.391001+0000","last_fullsized":"2023-11-28T11:10:52.391001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848738+0000","last_change":"2023-11-28T11:10:52.388031+0000","last_active":"2023-11-28T11:10:53.848738+0000","last_peered":"2023-11-28T11:10:53.848738+0000","last_clean":"2023-11-28T11:10:53.848738+0000","last_became_active":"2023-11-28T11:10:52.387685+0000","last_became_peered":"2023-11-28T11:10:52.387685+0000","last_unstale":"2023-11-28T11:10:53.848738+0000","last_undegraded":"2023-11-28T11:10:53.848738+0000","last_fullsized":"2023-11-28T11:10:53.848738+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:19:57.624643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.399510+0000","last_change":"2023-11-28T11:10:51.998733+0000","last_active":"2023-11-28T11:10:52.399510+0000","last_peered":"2023-11-28T11:10:52.399510+0000","last_clean":"2023-11-28T11:10:52.399510+0000","last_became_active":"2023-11-28T11:10:51.998123+0000","last_became_peered":"2023-11-28T11:10:51.998123+0000","last_unstale":"2023-11-28T11:10:52.399510+0000","last_undegraded":"2023-11-28T11:10:52.399510+0000","last_fullsized":"2023-11-28T11:10:52.399510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:23.426647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:10:48.027097+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:10:48.027097+0000","last_peered":"2023-11-28T11:10:48.027097+0000","last_clean":"2023-11-28T11:10:48.027097+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:10:48.027097+0000","last_undegraded":"2023-11-28T11:10:48.027097+0000","last_fullsized":"2023-11-28T11:10:48.027097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376982+0000","last_change":"2023-11-28T11:10:52.389489+0000","last_active":"2023-11-28T11:10:53.376982+0000","last_peered":"2023-11-28T11:10:53.376982+0000","last_clean":"2023-11-28T11:10:53.376982+0000","last_became_active":"2023-11-28T11:10:52.387877+0000","last_became_peered":"2023-11-28T11:10:52.387877+0000","last_unstale":"2023-11-28T11:10:53.376982+0000","last_undegraded":"2023-11-28T11:10:53.376982+0000","last_fullsized":"2023-11-28T11:10:53.376982+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:08:26.596352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.371082+0000","last_change":"2023-11-28T11:10:51.992409+0000","last_active":"2023-11-28T11:10:52.371082+0000","last_peered":"2023-11-28T11:10:52.371082+0000","last_clean":"2023-11-28T11:10:52.371082+0000","last_became_active":"2023-11-28T11:10:51.992031+0000","last_became_peered":"2023-11-28T11:10:51.992031+0000","last_unstale":"2023-11-28T11:10:52.371082+0000","last_undegraded":"2023-11-28T11:10:52.371082+0000","last_fullsized":"2023-11-28T11:10:52.371082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:56.382110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.380028+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:10:52.380028+0000","last_peered":"2023-11-28T11:10:52.380028+0000","last_clean":"2023-11-28T11:10:52.380028+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:10:52.380028+0000","last_undegraded":"2023-11-28T11:10:52.380028+0000","last_fullsized":"2023-11-28T11:10:52.380028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":194,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10880,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360960,"statfs":{"total":96636764160,"available":96625623040,"internally_reserved":0,"allocated":1368064,"data_stored":925048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92200000000000004,"5min":1.006,"15min":1.006},"last":0.27800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.67500000000000004,"15min":0.69999999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71099999999999997,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.747,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.35199999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0349999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.432},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97199999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.39700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.33700000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.86699999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.38}]},{"osd":5,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92100000000000004,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.37},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.443}]},{"osd":6,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.47299999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.52000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9328,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362512,"statfs":{"total":96636764160,"available":96627212288,"internally_reserved":0,"allocated":761856,"data_stored":333954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69499999999999995,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.70299999999999996,"15min":0.73399999999999999},"min":{"1min":0.36299999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76800000000000002,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.69099999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.81599999999999995,"15min":0.85999999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76500000000000001,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.77000000000000002,"15min":0.80500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79800000000000004,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.78900000000000003,"15min":0.83599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67200000000000004,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.68899999999999995,"15min":0.71699999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68400000000000005,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.76800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.755,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.65400000000000003,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65700000000000003,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.755}]},{"osd":4,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.85099999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.379,"5min":0.373,"15min":0.373},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.88800000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76900000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.86099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9288,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362552,"statfs":{"total":96636764160,"available":96627253248,"internally_reserved":0,"allocated":720896,"data_stored":313519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.59999999999999998,"15min":0.64600000000000002},"min":{"1min":0.221,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.504,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.377,"15min":0.38200000000000001},"min":{"1min":0.248,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.46100000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.378}]},{"osd":2,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.64200000000000002,"15min":0.68999999999999995},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56999999999999995,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51900000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.44600000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42899999999999999,"15min":0.435},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58899999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.375,"15min":0.375},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54200000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.3120000000000001,"15min":1.431},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79500000000000004,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.76100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78800000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":1.3440000000000001,"15min":1.462},"min":{"1min":0.47799999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93600000000000005,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.67900000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":1.3500000000000001,"15min":1.4670000000000001},"min":{"1min":0.498,"5min":0.439,"15min":0.439},"max":{"1min":0.95999999999999996,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.48599999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94399999999999995,"5min":1.018,"15min":1.018},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.44700000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14768,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94357072,"statfs":{"total":96636764160,"available":96621641728,"internally_reserved":0,"allocated":1351680,"data_stored":924288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.48599999999999999,"5min":0.502,"15min":0.502},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.44400000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.26100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53600000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.378,"15min":0.378},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.50900000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.29699999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47799999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.56699999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.311}]},{"osd":4,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58999999999999997,"15min":0.59299999999999997},"min":{"1min":0.38800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66700000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.46700000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.35899999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.434,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.39600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.432},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.47399999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.62,"15min":0.62},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.81399999999999995,"5min":1.026,"15min":1.026},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.41699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9452,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362388,"statfs":{"total":96636764160,"available":96627085312,"internally_reserved":0,"allocated":757760,"data_stored":333920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.32500000000000001,"15min":0.316},"min":{"1min":0.22,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.33900000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.34899999999999998,"15min":0.33500000000000002},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33700000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.29299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.312},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.36899999999999999},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.35299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.437}]},{"osd":6,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.503},{"interface":"front","average":{"1min":0.63,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.44600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9772,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362068,"statfs":{"total":96636764160,"available":96626757632,"internally_reserved":0,"allocated":757760,"data_stored":333920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.42999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.31900000000000001,"15min":0.317},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.502,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.35399999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.56000000000000005,"5min":1.365,"15min":1.365},"last":0.33200000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48499999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.31}]},{"osd":4,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.495},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42899999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82999999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46000000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81200000000000006,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85999999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.48099999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9592,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362248,"statfs":{"total":96636764160,"available":96626941952,"internally_reserved":0,"allocated":770048,"data_stored":335468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69699999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.60099999999999998,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.9740000000000002,"5min":4.9740000000000002,"15min":4.9740000000000002},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":1.115,"15min":1.2070000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69699999999999995,"5min":12.263,"15min":12.263},"last":0.44700000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.505},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":1.236,"15min":1.236},"last":0.41399999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5600000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":4.3710000000000004,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.59499999999999997,"15min":0.55000000000000004},"min":{"1min":0.40200000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":5.0220000000000002,"5min":5.0220000000000002,"15min":5.0220000000000002},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.52000000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.65600000000000003,"15min":0.64100000000000001},"min":{"1min":0.55700000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.60199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9340,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362500,"statfs":{"total":96636764160,"available":96627200000,"internally_reserved":0,"allocated":708608,"data_stored":307349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.45700000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53100000000000003},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.504,"15min":0.505},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66300000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.438,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.432}]},{"osd":4,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.497,"15min":0.497},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72999999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.67500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72899999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.441,"5min":0.374,"15min":0.374},"max":{"1min":0.67700000000000005,"5min":0.753,"15min":0.753},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.747,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.59299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:11:00.987 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:11:01.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.121+0000 7f35e4cb5700 1 Processor -- start 2023-11-28T11:11:01.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.121+0000 7f35e4cb5700 1 -- start start 2023-11-28T11:11:01.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.122+0000 7f35e4cb5700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f35e015cef0 0x7f35e00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.122+0000 7f35e4cb5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35e006ba90 0x7f35e00ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.122+0000 7f35e4cb5700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e00cc5f0 con 0x7f35e00cbdf0 2023-11-28T11:11:01.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.122+0000 7f35e4cb5700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e015d470 con 0x7f35e015cef0 2023-11-28T11:11:01.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.122+0000 7f35e4cb5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e015d750 con 0x7f35e006ba90 2023-11-28T11:11:01.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35ded9d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35e006ba90 0x7f35e00ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35de59c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f35e015cef0 0x7f35e00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35ded9d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35e006ba90 0x7f35e00ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:39616/0 (socket says 172.21.15.156:39616) 2023-11-28T11:11:01.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35de59c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f35e015cef0 0x7f35e00af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:46764/0 (socket says 172.21.15.156:46764) 2023-11-28T11:11:01.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35ddd9b700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f35e00cbdf0 0x7f35e006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:60992/0 (socket says 172.21.15.156:60992) 2023-11-28T11:11:01.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35ded9d700 1 -- 172.21.15.156:0/4258632383 learned_addr learned my addr 172.21.15.156:0/4258632383 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:01.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35dd59a700 1 -- 172.21.15.156:0/4258632383 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36993553 0 0) 0x7f35e00cc5f0 con 0x7f35e00cbdf0 2023-11-28T11:11:01.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35dd59a700 1 -- 172.21.15.156:0/4258632383 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c80035b0 con 0x7f35e00cbdf0 2023-11-28T11:11:01.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35ded9d700 1 -- 172.21.15.156:0/4258632383 >> v2:172.21.15.186:3300/0 conn(0x7f35e015cef0 msgr2=0x7f35e00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:01.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35ded9d700 1 --2- 172.21.15.156:0/4258632383 >> v2:172.21.15.186:3300/0 conn(0x7f35e015cef0 0x7f35e00af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35ded9d700 1 -- 172.21.15.156:0/4258632383 >> v1:172.21.15.186:6789/0 conn(0x7f35e00cbdf0 legacy=0x7f35e006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35ded9d700 1 -- 172.21.15.156:0/4258632383 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e015da30 con 0x7f35e006ba90 2023-11-28T11:11:01.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35de59c700 1 --2- 172.21.15.156:0/4258632383 >> v2:172.21.15.186:3300/0 conn(0x7f35e015cef0 0x7f35e00af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:01.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.123+0000 7f35ded9d700 1 --2- 172.21.15.156:0/4258632383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35e006ba90 0x7f35e00ca650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f35d800d180 tx=0x7f35d800d490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e18582e682b1287 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:01.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.124+0000 7f35dd59a700 1 -- 172.21.15.156:0/4258632383 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8010070 con 0x7f35e006ba90 2023-11-28T11:11:01.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.124+0000 7f35dd59a700 1 -- 172.21.15.156:0/4258632383 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d800dd00 con 0x7f35e006ba90 2023-11-28T11:11:01.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.124+0000 7f35dd59a700 1 -- 172.21.15.156:0/4258632383 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d800b310 con 0x7f35e006ba90 2023-11-28T11:11:01.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.124+0000 7f35e4cb5700 1 -- 172.21.15.156:0/4258632383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35e006ba90 msgr2=0x7f35e00ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.124+0000 7f35e4cb5700 1 --2- 172.21.15.156:0/4258632383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35e006ba90 0x7f35e00ca650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f35d800d180 tx=0x7f35d800d490 comp rx=0 tx=0).stop 2023-11-28T11:11:01.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.124+0000 7f35e4cb5700 1 -- 172.21.15.156:0/4258632383 shutdown_connections 2023-11-28T11:11:01.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.124+0000 7f35e4cb5700 1 --2- 172.21.15.156:0/4258632383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35e006ba90 0x7f35e00ca650 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.124+0000 7f35e4cb5700 1 --2- 172.21.15.156:0/4258632383 >> v2:172.21.15.186:3300/0 conn(0x7f35e015cef0 0x7f35e00af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.124+0000 7f35e4cb5700 1 -- 172.21.15.156:0/4258632383 >> 172.21.15.156:0/4258632383 conn(0x7f35e00d0000 msgr2=0x7f35e00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:01.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.124+0000 7f35e4cb5700 1 -- 172.21.15.156:0/4258632383 shutdown_connections 2023-11-28T11:11:01.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.124+0000 7f35e4cb5700 1 -- 172.21.15.156:0/4258632383 wait complete. 2023-11-28T11:11:01.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.126+0000 7f35e4cb5700 1 Processor -- start 2023-11-28T11:11:01.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.126+0000 7f35e4cb5700 1 -- start start 2023-11-28T11:11:01.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.126+0000 7f35e4cb5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e006ba90 0x7f35e006dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.126+0000 7f35e4cb5700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35e00cbdf0 0x7f35e0155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.126+0000 7f35e4cb5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35e015cef0 0x7f35e0156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.126+0000 7f35e4cb5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e0157580 con 0x7f35e006ba90 2023-11-28T11:11:01.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.126+0000 7f35e4cb5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e0156970 con 0x7f35e015cef0 2023-11-28T11:11:01.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.126+0000 7f35e4cb5700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e0156c50 con 0x7f35e00cbdf0 2023-11-28T11:11:01.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35de59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e006ba90 0x7f35e006dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35de59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e006ba90 0x7f35e006dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35756/0 (socket says 172.21.15.156:35756) 2023-11-28T11:11:01.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35de59c700 1 -- 172.21.15.156:0/529593625 learned_addr learned my addr 172.21.15.156:0/529593625 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:01.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35ded9d700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35e015cef0 0x7f35e0156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35ddd9b700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35e00cbdf0 0x7f35e0155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35de59c700 1 -- 172.21.15.156:0/529593625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35e00cbdf0 msgr2=0x7f35e0155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35de59c700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35e00cbdf0 0x7f35e0155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35de59c700 1 -- 172.21.15.156:0/529593625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35e015cef0 msgr2=0x7f35e0156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35de59c700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35e015cef0 0x7f35e0156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35de59c700 1 -- 172.21.15.156:0/529593625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40094a0 con 0x7f35e006ba90 2023-11-28T11:11:01.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35de59c700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e006ba90 0x7f35e006dcd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f35d400ebe0 tx=0x7f35d400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:01.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35ceffd700 1 -- 172.21.15.156:0/529593625 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4009760 con 0x7f35e006ba90 2023-11-28T11:11:01.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35ceffd700 1 -- 172.21.15.156:0/529593625 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d400c850 con 0x7f35e006ba90 2023-11-28T11:11:01.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35e4cb5700 1 -- 172.21.15.156:0/529593625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d800bd40 con 0x7f35e006ba90 2023-11-28T11:11:01.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35ceffd700 1 -- 172.21.15.156:0/529593625 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d400cbe0 con 0x7f35e006ba90 2023-11-28T11:11:01.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.127+0000 7f35e4cb5700 1 -- 172.21.15.156:0/529593625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e015dff0 con 0x7f35e006ba90 2023-11-28T11:11:01.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.128+0000 7f35e4cb5700 1 -- 172.21.15.156:0/529593625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e015dd10 con 0x7f35e006ba90 2023-11-28T11:11:01.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.129+0000 7f35ceffd700 1 -- 172.21.15.156:0/529593625 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f35d4019030 con 0x7f35e006ba90 2023-11-28T11:11:01.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.129+0000 7f35ceffd700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35c00682f0 0x7f35c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.129+0000 7f35ddd9b700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35c00682f0 0x7f35c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.129+0000 7f35ceffd700 1 -- 172.21.15.156:0/529593625 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f35d4085b70 con 0x7f35e006ba90 2023-11-28T11:11:01.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.130+0000 7f35ddd9b700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35c00682f0 0x7f35c006a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f35e015c2a0 tx=0x7f35d0007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:01.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.133+0000 7f35ceffd700 1 -- 172.21.15.156:0/529593625 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f35d40523d0 con 0x7f35e006ba90 2023-11-28T11:11:01.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.310+0000 7f35e4cb5700 1 -- 172.21.15.156:0/529593625 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35e00cab90 con 0x7f35c00682f0 2023-11-28T11:11:01.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.319+0000 7f35ceffd700 1 -- 172.21.15.156:0/529593625 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311137 (secure 0 0 0) 0x7f35e00cab90 con 0x7f35c00682f0 2023-11-28T11:11:01.321 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:01.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.322+0000 7f35e4cb5700 1 -- 172.21.15.156:0/529593625 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35c00682f0 msgr2=0x7f35c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.322+0000 7f35e4cb5700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35c00682f0 0x7f35c006a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f35e015c2a0 tx=0x7f35d0007dc0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.322+0000 7f35e4cb5700 1 -- 172.21.15.156:0/529593625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e006ba90 msgr2=0x7f35e006dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.322+0000 7f35e4cb5700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e006ba90 0x7f35e006dcd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f35d400ebe0 tx=0x7f35d400eef0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.322+0000 7f35e4cb5700 1 -- 172.21.15.156:0/529593625 shutdown_connections 2023-11-28T11:11:01.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.322+0000 7f35e4cb5700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35e015cef0 0x7f35e0156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.322+0000 7f35e4cb5700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e006ba90 0x7f35e006dcd0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.322+0000 7f35e4cb5700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f35c00682f0 0x7f35c006a7a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.322+0000 7f35e4cb5700 1 --2- 172.21.15.156:0/529593625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35e00cbdf0 0x7f35e0155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.322+0000 7f35e4cb5700 1 -- 172.21.15.156:0/529593625 >> 172.21.15.156:0/529593625 conn(0x7f35e00d0000 msgr2=0x7f35e00caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:01.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.322+0000 7f35e4cb5700 1 -- 172.21.15.156:0/529593625 shutdown_connections 2023-11-28T11:11:01.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.322+0000 7f35e4cb5700 1 -- 172.21.15.156:0/529593625 wait complete. 2023-11-28T11:11:01.328 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:11:01.335 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2023-11-28T11:11:00.561144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":192,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82856,"kb_used_data":7144,"kb_used_omap":0,"kb_used_meta":75648,"kb_avail":754891864,"statfs":{"total":773094113280,"available":773009268736,"internally_reserved":0,"allocated":7315456,"data_stored":3856756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77463552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.191577"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848342+0000","last_change":"2023-11-28T11:10:52.388212+0000","last_active":"2023-11-28T11:10:53.848342+0000","last_peered":"2023-11-28T11:10:53.848342+0000","last_clean":"2023-11-28T11:10:53.848342+0000","last_became_active":"2023-11-28T11:10:52.388012+0000","last_became_peered":"2023-11-28T11:10:52.388012+0000","last_unstale":"2023-11-28T11:10:53.848342+0000","last_undegraded":"2023-11-28T11:10:53.848342+0000","last_fullsized":"2023-11-28T11:10:53.848342+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:07.493993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848849+0000","last_change":"2023-11-28T11:10:52.389599+0000","last_active":"2023-11-28T11:10:53.848849+0000","last_peered":"2023-11-28T11:10:53.848849+0000","last_clean":"2023-11-28T11:10:53.848849+0000","last_became_active":"2023-11-28T11:10:52.389325+0000","last_became_peered":"2023-11-28T11:10:52.389325+0000","last_unstale":"2023-11-28T11:10:53.848849+0000","last_undegraded":"2023-11-28T11:10:53.848849+0000","last_fullsized":"2023-11-28T11:10:53.848849+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:22.764970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848556+0000","last_change":"2023-11-28T11:10:52.399647+0000","last_active":"2023-11-28T11:10:53.848556+0000","last_peered":"2023-11-28T11:10:53.848556+0000","last_clean":"2023-11-28T11:10:53.848556+0000","last_became_active":"2023-11-28T11:10:52.399461+0000","last_became_peered":"2023-11-28T11:10:52.399461+0000","last_unstale":"2023-11-28T11:10:53.848556+0000","last_undegraded":"2023-11-28T11:10:53.848556+0000","last_fullsized":"2023-11-28T11:10:53.848556+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:36.282716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848853+0000","last_change":"2023-11-28T11:10:52.390854+0000","last_active":"2023-11-28T11:10:53.848853+0000","last_peered":"2023-11-28T11:10:53.848853+0000","last_clean":"2023-11-28T11:10:53.848853+0000","last_became_active":"2023-11-28T11:10:52.390541+0000","last_became_peered":"2023-11-28T11:10:52.390541+0000","last_unstale":"2023-11-28T11:10:53.848853+0000","last_undegraded":"2023-11-28T11:10:53.848853+0000","last_fullsized":"2023-11-28T11:10:53.848853+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:06.203049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377143+0000","last_change":"2023-11-28T11:10:52.394457+0000","last_active":"2023-11-28T11:10:53.377143+0000","last_peered":"2023-11-28T11:10:53.377143+0000","last_clean":"2023-11-28T11:10:53.377143+0000","last_became_active":"2023-11-28T11:10:52.394170+0000","last_became_peered":"2023-11-28T11:10:52.394170+0000","last_unstale":"2023-11-28T11:10:53.377143+0000","last_undegraded":"2023-11-28T11:10:53.377143+0000","last_fullsized":"2023-11-28T11:10:53.377143+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:12.029150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848451+0000","last_change":"2023-11-28T11:10:52.398045+0000","last_active":"2023-11-28T11:10:53.848451+0000","last_peered":"2023-11-28T11:10:53.848451+0000","last_clean":"2023-11-28T11:10:53.848451+0000","last_became_active":"2023-11-28T11:10:52.397753+0000","last_became_peered":"2023-11-28T11:10:52.397753+0000","last_unstale":"2023-11-28T11:10:53.848451+0000","last_undegraded":"2023-11-28T11:10:53.848451+0000","last_fullsized":"2023-11-28T11:10:53.848451+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:35.944815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847689+0000","last_change":"2023-11-28T11:10:52.397026+0000","last_active":"2023-11-28T11:10:53.847689+0000","last_peered":"2023-11-28T11:10:53.847689+0000","last_clean":"2023-11-28T11:10:53.847689+0000","last_became_active":"2023-11-28T11:10:52.396742+0000","last_became_peered":"2023-11-28T11:10:52.396742+0000","last_unstale":"2023-11-28T11:10:53.847689+0000","last_undegraded":"2023-11-28T11:10:53.847689+0000","last_fullsized":"2023-11-28T11:10:53.847689+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:00.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377880+0000","last_change":"2023-11-28T11:10:52.397362+0000","last_active":"2023-11-28T11:10:53.377880+0000","last_peered":"2023-11-28T11:10:53.377880+0000","last_clean":"2023-11-28T11:10:53.377880+0000","last_became_active":"2023-11-28T11:10:52.397003+0000","last_became_peered":"2023-11-28T11:10:52.397003+0000","last_unstale":"2023-11-28T11:10:53.377880+0000","last_undegraded":"2023-11-28T11:10:53.377880+0000","last_fullsized":"2023-11-28T11:10:53.377880+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:48.226111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848795+0000","last_change":"2023-11-28T11:10:52.401622+0000","last_active":"2023-11-28T11:10:53.848795+0000","last_peered":"2023-11-28T11:10:53.848795+0000","last_clean":"2023-11-28T11:10:53.848795+0000","last_became_active":"2023-11-28T11:10:52.401310+0000","last_became_peered":"2023-11-28T11:10:52.401310+0000","last_unstale":"2023-11-28T11:10:53.848795+0000","last_undegraded":"2023-11-28T11:10:53.848795+0000","last_fullsized":"2023-11-28T11:10:53.848795+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:23.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848431+0000","last_change":"2023-11-28T11:10:52.398550+0000","last_active":"2023-11-28T11:10:53.848431+0000","last_peered":"2023-11-28T11:10:53.848431+0000","last_clean":"2023-11-28T11:10:53.848431+0000","last_became_active":"2023-11-28T11:10:52.398277+0000","last_became_peered":"2023-11-28T11:10:52.398277+0000","last_unstale":"2023-11-28T11:10:53.848431+0000","last_undegraded":"2023-11-28T11:10:53.848431+0000","last_fullsized":"2023-11-28T11:10:53.848431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:53.807072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848691+0000","last_change":"2023-11-28T11:10:52.396844+0000","last_active":"2023-11-28T11:10:53.848691+0000","last_peered":"2023-11-28T11:10:53.848691+0000","last_clean":"2023-11-28T11:10:53.848691+0000","last_became_active":"2023-11-28T11:10:52.393214+0000","last_became_peered":"2023-11-28T11:10:52.393214+0000","last_unstale":"2023-11-28T11:10:53.848691+0000","last_undegraded":"2023-11-28T11:10:53.848691+0000","last_fullsized":"2023-11-28T11:10:53.848691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:12.100298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377957+0000","last_change":"2023-11-28T11:10:52.397906+0000","last_active":"2023-11-28T11:10:53.377957+0000","last_peered":"2023-11-28T11:10:53.377957+0000","last_clean":"2023-11-28T11:10:53.377957+0000","last_became_active":"2023-11-28T11:10:52.397624+0000","last_became_peered":"2023-11-28T11:10:52.397624+0000","last_unstale":"2023-11-28T11:10:53.377957+0000","last_undegraded":"2023-11-28T11:10:53.377957+0000","last_fullsized":"2023-11-28T11:10:53.377957+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:38.996786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.378179+0000","last_change":"2023-11-28T11:10:52.394384+0000","last_active":"2023-11-28T11:10:53.378179+0000","last_peered":"2023-11-28T11:10:53.378179+0000","last_clean":"2023-11-28T11:10:53.378179+0000","last_became_active":"2023-11-28T11:10:52.394149+0000","last_became_peered":"2023-11-28T11:10:52.394149+0000","last_unstale":"2023-11-28T11:10:53.378179+0000","last_undegraded":"2023-11-28T11:10:53.378179+0000","last_fullsized":"2023-11-28T11:10:53.378179+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:12.100298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848063+0000","last_change":"2023-11-28T11:10:52.397847+0000","last_active":"2023-11-28T11:10:53.848063+0000","last_peered":"2023-11-28T11:10:53.848063+0000","last_clean":"2023-11-28T11:10:53.848063+0000","last_became_active":"2023-11-28T11:10:52.397580+0000","last_became_peered":"2023-11-28T11:10:52.397580+0000","last_unstale":"2023-11-28T11:10:53.848063+0000","last_undegraded":"2023-11-28T11:10:53.848063+0000","last_fullsized":"2023-11-28T11:10:53.848063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:11:29.529381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848017+0000","last_change":"2023-11-28T11:10:52.397544+0000","last_active":"2023-11-28T11:10:53.848017+0000","last_peered":"2023-11-28T11:10:53.848017+0000","last_clean":"2023-11-28T11:10:53.848017+0000","last_became_active":"2023-11-28T11:10:52.397232+0000","last_became_peered":"2023-11-28T11:10:52.397232+0000","last_unstale":"2023-11-28T11:10:53.848017+0000","last_undegraded":"2023-11-28T11:10:53.848017+0000","last_fullsized":"2023-11-28T11:10:53.848017+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:33.270658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847532+0000","last_change":"2023-11-28T11:10:52.388564+0000","last_active":"2023-11-28T11:10:53.847532+0000","last_peered":"2023-11-28T11:10:53.847532+0000","last_clean":"2023-11-28T11:10:53.847532+0000","last_became_active":"2023-11-28T11:10:52.388307+0000","last_became_peered":"2023-11-28T11:10:52.388307+0000","last_unstale":"2023-11-28T11:10:53.847532+0000","last_undegraded":"2023-11-28T11:10:53.847532+0000","last_fullsized":"2023-11-28T11:10:53.847532+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:54.802666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848425+0000","last_change":"2023-11-28T11:10:52.394498+0000","last_active":"2023-11-28T11:10:53.848425+0000","last_peered":"2023-11-28T11:10:53.848425+0000","last_clean":"2023-11-28T11:10:53.848425+0000","last_became_active":"2023-11-28T11:10:52.394239+0000","last_became_peered":"2023-11-28T11:10:52.394239+0000","last_unstale":"2023-11-28T11:10:53.848425+0000","last_undegraded":"2023-11-28T11:10:53.848425+0000","last_fullsized":"2023-11-28T11:10:53.848425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:33.270658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848635+0000","last_change":"2023-11-28T11:10:52.391388+0000","last_active":"2023-11-28T11:10:53.848635+0000","last_peered":"2023-11-28T11:10:53.848635+0000","last_clean":"2023-11-28T11:10:53.848635+0000","last_became_active":"2023-11-28T11:10:52.391076+0000","last_became_peered":"2023-11-28T11:10:52.391076+0000","last_unstale":"2023-11-28T11:10:53.848635+0000","last_undegraded":"2023-11-28T11:10:53.848635+0000","last_fullsized":"2023-11-28T11:10:53.848635+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:05:43.320524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376913+0000","last_change":"2023-11-28T11:10:52.380958+0000","last_active":"2023-11-28T11:10:53.376913+0000","last_peered":"2023-11-28T11:10:53.376913+0000","last_clean":"2023-11-28T11:10:53.376913+0000","last_became_active":"2023-11-28T11:10:52.380688+0000","last_became_peered":"2023-11-28T11:10:52.380688+0000","last_unstale":"2023-11-28T11:10:53.376913+0000","last_undegraded":"2023-11-28T11:10:53.376913+0000","last_fullsized":"2023-11-28T11:10:53.376913+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:22.866138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375117+0000","last_change":"2023-11-28T11:10:52.387375+0000","last_active":"2023-11-28T11:10:53.375117+0000","last_peered":"2023-11-28T11:10:53.375117+0000","last_clean":"2023-11-28T11:10:53.375117+0000","last_became_active":"2023-11-28T11:10:52.387088+0000","last_became_peered":"2023-11-28T11:10:52.387088+0000","last_unstale":"2023-11-28T11:10:53.375117+0000","last_undegraded":"2023-11-28T11:10:53.375117+0000","last_fullsized":"2023-11-28T11:10:53.375117+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:54.810083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.378148+0000","last_change":"2023-11-28T11:10:52.386481+0000","last_active":"2023-11-28T11:10:53.378148+0000","last_peered":"2023-11-28T11:10:53.378148+0000","last_clean":"2023-11-28T11:10:53.378148+0000","last_became_active":"2023-11-28T11:10:52.386139+0000","last_became_peered":"2023-11-28T11:10:52.386139+0000","last_unstale":"2023-11-28T11:10:53.378148+0000","last_undegraded":"2023-11-28T11:10:53.378148+0000","last_fullsized":"2023-11-28T11:10:53.378148+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:22.764970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375344+0000","last_change":"2023-11-28T11:10:52.398710+0000","last_active":"2023-11-28T11:10:53.375344+0000","last_peered":"2023-11-28T11:10:53.375344+0000","last_clean":"2023-11-28T11:10:53.375344+0000","last_became_active":"2023-11-28T11:10:52.398388+0000","last_became_peered":"2023-11-28T11:10:52.398388+0000","last_unstale":"2023-11-28T11:10:53.375344+0000","last_undegraded":"2023-11-28T11:10:53.375344+0000","last_fullsized":"2023-11-28T11:10:53.375344+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:09.520497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.374950+0000","last_change":"2023-11-28T11:10:52.398644+0000","last_active":"2023-11-28T11:10:53.374950+0000","last_peered":"2023-11-28T11:10:53.374950+0000","last_clean":"2023-11-28T11:10:53.374950+0000","last_became_active":"2023-11-28T11:10:52.398342+0000","last_became_peered":"2023-11-28T11:10:52.398342+0000","last_unstale":"2023-11-28T11:10:53.374950+0000","last_undegraded":"2023-11-28T11:10:53.374950+0000","last_fullsized":"2023-11-28T11:10:53.374950+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:02.775343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848270+0000","last_change":"2023-11-28T11:10:52.400926+0000","last_active":"2023-11-28T11:10:53.848270+0000","last_peered":"2023-11-28T11:10:53.848270+0000","last_clean":"2023-11-28T11:10:53.848270+0000","last_became_active":"2023-11-28T11:10:52.400667+0000","last_became_peered":"2023-11-28T11:10:52.400667+0000","last_unstale":"2023-11-28T11:10:53.848270+0000","last_undegraded":"2023-11-28T11:10:53.848270+0000","last_fullsized":"2023-11-28T11:10:53.848270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:06.203049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.378310+0000","last_change":"2023-11-28T11:10:52.389581+0000","last_active":"2023-11-28T11:10:53.378310+0000","last_peered":"2023-11-28T11:10:53.378310+0000","last_clean":"2023-11-28T11:10:53.378310+0000","last_became_active":"2023-11-28T11:10:52.388111+0000","last_became_peered":"2023-11-28T11:10:52.388111+0000","last_unstale":"2023-11-28T11:10:53.378310+0000","last_undegraded":"2023-11-28T11:10:53.378310+0000","last_fullsized":"2023-11-28T11:10:53.378310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:05:43.320524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.378266+0000","last_change":"2023-11-28T11:10:52.387465+0000","last_active":"2023-11-28T11:10:53.378266+0000","last_peered":"2023-11-28T11:10:53.378266+0000","last_clean":"2023-11-28T11:10:53.378266+0000","last_became_active":"2023-11-28T11:10:52.387178+0000","last_became_peered":"2023-11-28T11:10:52.387178+0000","last_unstale":"2023-11-28T11:10:53.378266+0000","last_undegraded":"2023-11-28T11:10:53.378266+0000","last_fullsized":"2023-11-28T11:10:53.378266+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:53.807072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848749+0000","last_change":"2023-11-28T11:10:52.390919+0000","last_active":"2023-11-28T11:10:53.848749+0000","last_peered":"2023-11-28T11:10:53.848749+0000","last_clean":"2023-11-28T11:10:53.848749+0000","last_became_active":"2023-11-28T11:10:52.390435+0000","last_became_peered":"2023-11-28T11:10:52.390435+0000","last_unstale":"2023-11-28T11:10:53.848749+0000","last_undegraded":"2023-11-28T11:10:53.848749+0000","last_fullsized":"2023-11-28T11:10:53.848749+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:48.226111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848062+0000","last_change":"2023-11-28T11:10:52.387624+0000","last_active":"2023-11-28T11:10:53.848062+0000","last_peered":"2023-11-28T11:10:53.848062+0000","last_clean":"2023-11-28T11:10:53.848062+0000","last_became_active":"2023-11-28T11:10:52.387224+0000","last_became_peered":"2023-11-28T11:10:52.387224+0000","last_unstale":"2023-11-28T11:10:53.848062+0000","last_undegraded":"2023-11-28T11:10:53.848062+0000","last_fullsized":"2023-11-28T11:10:53.848062+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:57.389694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848092+0000","last_change":"2023-11-28T11:10:52.397318+0000","last_active":"2023-11-28T11:10:53.848092+0000","last_peered":"2023-11-28T11:10:53.848092+0000","last_clean":"2023-11-28T11:10:53.848092+0000","last_became_active":"2023-11-28T11:10:52.397039+0000","last_became_peered":"2023-11-28T11:10:52.397039+0000","last_unstale":"2023-11-28T11:10:53.848092+0000","last_undegraded":"2023-11-28T11:10:53.848092+0000","last_fullsized":"2023-11-28T11:10:53.848092+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:54.810083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377953+0000","last_change":"2023-11-28T11:10:52.400964+0000","last_active":"2023-11-28T11:10:53.377953+0000","last_peered":"2023-11-28T11:10:53.377953+0000","last_clean":"2023-11-28T11:10:53.377953+0000","last_became_active":"2023-11-28T11:10:52.400641+0000","last_became_peered":"2023-11-28T11:10:52.400641+0000","last_unstale":"2023-11-28T11:10:53.377953+0000","last_undegraded":"2023-11-28T11:10:53.377953+0000","last_fullsized":"2023-11-28T11:10:53.377953+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:57.389694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847782+0000","last_change":"2023-11-28T11:10:52.394969+0000","last_active":"2023-11-28T11:10:53.847782+0000","last_peered":"2023-11-28T11:10:53.847782+0000","last_clean":"2023-11-28T11:10:53.847782+0000","last_became_active":"2023-11-28T11:10:52.394620+0000","last_became_peered":"2023-11-28T11:10:52.394620+0000","last_unstale":"2023-11-28T11:10:53.847782+0000","last_undegraded":"2023-11-28T11:10:53.847782+0000","last_fullsized":"2023-11-28T11:10:53.847782+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:28.345284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847322+0000","last_change":"2023-11-28T11:10:52.382159+0000","last_active":"2023-11-28T11:10:53.847322+0000","last_peered":"2023-11-28T11:10:53.847322+0000","last_clean":"2023-11-28T11:10:53.847322+0000","last_became_active":"2023-11-28T11:10:52.381757+0000","last_became_peered":"2023-11-28T11:10:52.381757+0000","last_unstale":"2023-11-28T11:10:53.847322+0000","last_undegraded":"2023-11-28T11:10:53.847322+0000","last_fullsized":"2023-11-28T11:10:53.847322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:21.448143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377057+0000","last_change":"2023-11-28T11:10:52.389365+0000","last_active":"2023-11-28T11:10:53.377057+0000","last_peered":"2023-11-28T11:10:53.377057+0000","last_clean":"2023-11-28T11:10:53.377057+0000","last_became_active":"2023-11-28T11:10:52.388931+0000","last_became_peered":"2023-11-28T11:10:52.388931+0000","last_unstale":"2023-11-28T11:10:53.377057+0000","last_undegraded":"2023-11-28T11:10:53.377057+0000","last_fullsized":"2023-11-28T11:10:53.377057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:30.962468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"62'4","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.400177+0000","last_change":"2023-11-28T11:10:51.995946+0000","last_active":"2023-11-28T11:10:52.400177+0000","last_peered":"2023-11-28T11:10:52.400177+0000","last_clean":"2023-11-28T11:10:52.400177+0000","last_became_active":"2023-11-28T11:10:51.995701+0000","last_became_peered":"2023-11-28T11:10:51.995701+0000","last_unstale":"2023-11-28T11:10:52.400177+0000","last_undegraded":"2023-11-28T11:10:52.400177+0000","last_fullsized":"2023-11-28T11:10:52.400177+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:23.426647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848809+0000","last_change":"2023-11-28T11:10:52.390732+0000","last_active":"2023-11-28T11:10:53.848809+0000","last_peered":"2023-11-28T11:10:53.848809+0000","last_clean":"2023-11-28T11:10:53.848809+0000","last_became_active":"2023-11-28T11:10:52.390111+0000","last_became_peered":"2023-11-28T11:10:52.390111+0000","last_unstale":"2023-11-28T11:10:53.848809+0000","last_undegraded":"2023-11-28T11:10:53.848809+0000","last_fullsized":"2023-11-28T11:10:53.848809+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:18.651068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.378205+0000","last_change":"2023-11-28T11:10:51.995120+0000","last_active":"2023-11-28T11:10:52.378205+0000","last_peered":"2023-11-28T11:10:52.378205+0000","last_clean":"2023-11-28T11:10:52.378205+0000","last_became_active":"2023-11-28T11:10:51.994836+0000","last_became_peered":"2023-11-28T11:10:51.994836+0000","last_unstale":"2023-11-28T11:10:52.378205+0000","last_undegraded":"2023-11-28T11:10:52.378205+0000","last_fullsized":"2023-11-28T11:10:52.378205+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:32.263075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848457+0000","last_change":"2023-11-28T11:10:52.385151+0000","last_active":"2023-11-28T11:10:53.848457+0000","last_peered":"2023-11-28T11:10:53.848457+0000","last_clean":"2023-11-28T11:10:53.848457+0000","last_became_active":"2023-11-28T11:10:52.381849+0000","last_became_peered":"2023-11-28T11:10:52.381849+0000","last_unstale":"2023-11-28T11:10:53.848457+0000","last_undegraded":"2023-11-28T11:10:53.848457+0000","last_fullsized":"2023-11-28T11:10:53.848457+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:02.775343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.384672+0000","last_change":"2023-11-28T11:10:52.000037+0000","last_active":"2023-11-28T11:10:52.384672+0000","last_peered":"2023-11-28T11:10:52.384672+0000","last_clean":"2023-11-28T11:10:52.384672+0000","last_became_active":"2023-11-28T11:10:51.999788+0000","last_became_peered":"2023-11-28T11:10:51.999788+0000","last_unstale":"2023-11-28T11:10:52.384672+0000","last_undegraded":"2023-11-28T11:10:52.384672+0000","last_fullsized":"2023-11-28T11:10:52.384672+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:52.799489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848659+0000","last_change":"2023-11-28T11:10:52.399189+0000","last_active":"2023-11-28T11:10:53.848659+0000","last_peered":"2023-11-28T11:10:53.848659+0000","last_clean":"2023-11-28T11:10:53.848659+0000","last_became_active":"2023-11-28T11:10:52.398894+0000","last_became_peered":"2023-11-28T11:10:52.398894+0000","last_unstale":"2023-11-28T11:10:53.848659+0000","last_undegraded":"2023-11-28T11:10:53.848659+0000","last_fullsized":"2023-11-28T11:10:53.848659+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:59:09.926232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.381856+0000","last_change":"2023-11-28T11:10:51.990812+0000","last_active":"2023-11-28T11:10:52.381856+0000","last_peered":"2023-11-28T11:10:52.381856+0000","last_clean":"2023-11-28T11:10:52.381856+0000","last_became_active":"2023-11-28T11:10:51.990539+0000","last_became_peered":"2023-11-28T11:10:51.990539+0000","last_unstale":"2023-11-28T11:10:52.381856+0000","last_undegraded":"2023-11-28T11:10:52.381856+0000","last_fullsized":"2023-11-28T11:10:52.381856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:06.486410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847570+0000","last_change":"2023-11-28T11:10:52.381091+0000","last_active":"2023-11-28T11:10:53.847570+0000","last_peered":"2023-11-28T11:10:53.847570+0000","last_clean":"2023-11-28T11:10:53.847570+0000","last_became_active":"2023-11-28T11:10:52.380839+0000","last_became_peered":"2023-11-28T11:10:52.380839+0000","last_unstale":"2023-11-28T11:10:53.847570+0000","last_undegraded":"2023-11-28T11:10:53.847570+0000","last_fullsized":"2023-11-28T11:10:53.847570+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:29:30.173766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.375838+0000","last_change":"2023-11-28T11:10:51.985814+0000","last_active":"2023-11-28T11:10:52.375838+0000","last_peered":"2023-11-28T11:10:52.375838+0000","last_clean":"2023-11-28T11:10:52.375838+0000","last_became_active":"2023-11-28T11:10:51.985536+0000","last_became_peered":"2023-11-28T11:10:51.985536+0000","last_unstale":"2023-11-28T11:10:52.375838+0000","last_undegraded":"2023-11-28T11:10:52.375838+0000","last_fullsized":"2023-11-28T11:10:52.375838+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:11.092714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847625+0000","last_change":"2023-11-28T11:10:52.396018+0000","last_active":"2023-11-28T11:10:53.847625+0000","last_peered":"2023-11-28T11:10:53.847625+0000","last_clean":"2023-11-28T11:10:53.847625+0000","last_became_active":"2023-11-28T11:10:52.395720+0000","last_became_peered":"2023-11-28T11:10:52.395720+0000","last_unstale":"2023-11-28T11:10:53.847625+0000","last_undegraded":"2023-11-28T11:10:53.847625+0000","last_fullsized":"2023-11-28T11:10:53.847625+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:53.807072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377014+0000","last_change":"2023-11-28T11:10:51.984690+0000","last_active":"2023-11-28T11:10:52.377014+0000","last_peered":"2023-11-28T11:10:52.377014+0000","last_clean":"2023-11-28T11:10:52.377014+0000","last_became_active":"2023-11-28T11:10:51.984385+0000","last_became_peered":"2023-11-28T11:10:51.984385+0000","last_unstale":"2023-11-28T11:10:52.377014+0000","last_undegraded":"2023-11-28T11:10:52.377014+0000","last_fullsized":"2023-11-28T11:10:52.377014+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:06.486410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376774+0000","last_change":"2023-11-28T11:10:52.390623+0000","last_active":"2023-11-28T11:10:53.376774+0000","last_peered":"2023-11-28T11:10:53.376774+0000","last_clean":"2023-11-28T11:10:53.376774+0000","last_became_active":"2023-11-28T11:10:52.390344+0000","last_became_peered":"2023-11-28T11:10:52.390344+0000","last_unstale":"2023-11-28T11:10:53.376774+0000","last_undegraded":"2023-11-28T11:10:53.376774+0000","last_fullsized":"2023-11-28T11:10:53.376774+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:45.616308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.370871+0000","last_change":"2023-11-28T11:10:51.991786+0000","last_active":"2023-11-28T11:10:52.370871+0000","last_peered":"2023-11-28T11:10:52.370871+0000","last_clean":"2023-11-28T11:10:52.370871+0000","last_became_active":"2023-11-28T11:10:51.991493+0000","last_became_peered":"2023-11-28T11:10:51.991493+0000","last_unstale":"2023-11-28T11:10:52.370871+0000","last_undegraded":"2023-11-28T11:10:52.370871+0000","last_fullsized":"2023-11-28T11:10:52.370871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:21.757386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375324+0000","last_change":"2023-11-28T11:10:52.398474+0000","last_active":"2023-11-28T11:10:53.375324+0000","last_peered":"2023-11-28T11:10:53.375324+0000","last_clean":"2023-11-28T11:10:53.375324+0000","last_became_active":"2023-11-28T11:10:52.398155+0000","last_became_peered":"2023-11-28T11:10:52.398155+0000","last_unstale":"2023-11-28T11:10:53.375324+0000","last_undegraded":"2023-11-28T11:10:53.375324+0000","last_fullsized":"2023-11-28T11:10:53.375324+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:23.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.383141+0000","last_change":"2023-11-28T11:10:51.995022+0000","last_active":"2023-11-28T11:10:52.383141+0000","last_peered":"2023-11-28T11:10:52.383141+0000","last_clean":"2023-11-28T11:10:52.383141+0000","last_became_active":"2023-11-28T11:10:51.994731+0000","last_became_peered":"2023-11-28T11:10:51.994731+0000","last_unstale":"2023-11-28T11:10:52.383141+0000","last_undegraded":"2023-11-28T11:10:52.383141+0000","last_fullsized":"2023-11-28T11:10:52.383141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:31:32.827101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847866+0000","last_change":"2023-11-28T11:10:52.387459+0000","last_active":"2023-11-28T11:10:53.847866+0000","last_peered":"2023-11-28T11:10:53.847866+0000","last_clean":"2023-11-28T11:10:53.847866+0000","last_became_active":"2023-11-28T11:10:52.386874+0000","last_became_peered":"2023-11-28T11:10:52.386874+0000","last_unstale":"2023-11-28T11:10:53.847866+0000","last_undegraded":"2023-11-28T11:10:53.847866+0000","last_fullsized":"2023-11-28T11:10:53.847866+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:36.996022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.395098+0000","last_change":"2023-11-28T11:10:51.998518+0000","last_active":"2023-11-28T11:10:52.395098+0000","last_peered":"2023-11-28T11:10:52.395098+0000","last_clean":"2023-11-28T11:10:52.395098+0000","last_became_active":"2023-11-28T11:10:51.998086+0000","last_became_peered":"2023-11-28T11:10:51.998086+0000","last_unstale":"2023-11-28T11:10:52.395098+0000","last_undegraded":"2023-11-28T11:10:52.395098+0000","last_fullsized":"2023-11-28T11:10:52.395098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:21:56.126127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.375245+0000","last_change":"2023-11-28T11:10:52.383474+0000","last_active":"2023-11-28T11:10:53.375245+0000","last_peered":"2023-11-28T11:10:53.375245+0000","last_clean":"2023-11-28T11:10:53.375245+0000","last_became_active":"2023-11-28T11:10:52.383211+0000","last_became_peered":"2023-11-28T11:10:52.383211+0000","last_unstale":"2023-11-28T11:10:53.375245+0000","last_undegraded":"2023-11-28T11:10:53.375245+0000","last_fullsized":"2023-11-28T11:10:53.375245+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:48.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.384146+0000","last_change":"2023-11-28T11:10:51.997551+0000","last_active":"2023-11-28T11:10:52.384146+0000","last_peered":"2023-11-28T11:10:52.384146+0000","last_clean":"2023-11-28T11:10:52.384146+0000","last_became_active":"2023-11-28T11:10:51.997278+0000","last_became_peered":"2023-11-28T11:10:51.997278+0000","last_unstale":"2023-11-28T11:10:52.384146+0000","last_undegraded":"2023-11-28T11:10:52.384146+0000","last_fullsized":"2023-11-28T11:10:52.384146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:21.858555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377081+0000","last_change":"2023-11-28T11:10:52.393694+0000","last_active":"2023-11-28T11:10:53.377081+0000","last_peered":"2023-11-28T11:10:53.377081+0000","last_clean":"2023-11-28T11:10:53.377081+0000","last_became_active":"2023-11-28T11:10:52.393435+0000","last_became_peered":"2023-11-28T11:10:52.393435+0000","last_unstale":"2023-11-28T11:10:53.377081+0000","last_undegraded":"2023-11-28T11:10:53.377081+0000","last_fullsized":"2023-11-28T11:10:53.377081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:05:25.141120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"62'3","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.399587+0000","last_change":"2023-11-28T11:10:51.997311+0000","last_active":"2023-11-28T11:10:52.399587+0000","last_peered":"2023-11-28T11:10:52.399587+0000","last_clean":"2023-11-28T11:10:52.399587+0000","last_became_active":"2023-11-28T11:10:51.997038+0000","last_became_peered":"2023-11-28T11:10:51.997038+0000","last_unstale":"2023-11-28T11:10:52.399587+0000","last_undegraded":"2023-11-28T11:10:52.399587+0000","last_fullsized":"2023-11-28T11:10:52.399587+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:47.034924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848569+0000","last_change":"2023-11-28T11:10:52.393378+0000","last_active":"2023-11-28T11:10:53.848569+0000","last_peered":"2023-11-28T11:10:53.848569+0000","last_clean":"2023-11-28T11:10:53.848569+0000","last_became_active":"2023-11-28T11:10:52.393089+0000","last_became_peered":"2023-11-28T11:10:52.393089+0000","last_unstale":"2023-11-28T11:10:53.848569+0000","last_undegraded":"2023-11-28T11:10:53.848569+0000","last_fullsized":"2023-11-28T11:10:53.848569+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:22.764970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"62'6","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.400286+0000","last_change":"2023-11-28T11:10:51.999503+0000","last_active":"2023-11-28T11:10:52.400286+0000","last_peered":"2023-11-28T11:10:52.400286+0000","last_clean":"2023-11-28T11:10:52.400286+0000","last_became_active":"2023-11-28T11:10:51.999154+0000","last_became_peered":"2023-11-28T11:10:51.999154+0000","last_unstale":"2023-11-28T11:10:52.400286+0000","last_undegraded":"2023-11-28T11:10:52.400286+0000","last_fullsized":"2023-11-28T11:10:52.400286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:41:07.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847717+0000","last_change":"2023-11-28T11:10:52.402741+0000","last_active":"2023-11-28T11:10:53.847717+0000","last_peered":"2023-11-28T11:10:53.847717+0000","last_clean":"2023-11-28T11:10:53.847717+0000","last_became_active":"2023-11-28T11:10:52.402537+0000","last_became_peered":"2023-11-28T11:10:52.402537+0000","last_unstale":"2023-11-28T11:10:53.847717+0000","last_undegraded":"2023-11-28T11:10:53.847717+0000","last_fullsized":"2023-11-28T11:10:53.847717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:48.042507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.403563+0000","last_change":"2023-11-28T11:10:51.999231+0000","last_active":"2023-11-28T11:10:52.403563+0000","last_peered":"2023-11-28T11:10:52.403563+0000","last_clean":"2023-11-28T11:10:52.403563+0000","last_became_active":"2023-11-28T11:10:51.998951+0000","last_became_peered":"2023-11-28T11:10:51.998951+0000","last_unstale":"2023-11-28T11:10:52.403563+0000","last_undegraded":"2023-11-28T11:10:52.403563+0000","last_fullsized":"2023-11-28T11:10:52.403563+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:27.337701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377948+0000","last_change":"2023-11-28T11:10:52.382215+0000","last_active":"2023-11-28T11:10:53.377948+0000","last_peered":"2023-11-28T11:10:53.377948+0000","last_clean":"2023-11-28T11:10:53.377948+0000","last_became_active":"2023-11-28T11:10:52.381945+0000","last_became_peered":"2023-11-28T11:10:52.381945+0000","last_unstale":"2023-11-28T11:10:53.377948+0000","last_undegraded":"2023-11-28T11:10:53.377948+0000","last_fullsized":"2023-11-28T11:10:53.377948+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:48.042507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377576+0000","last_change":"2023-11-28T11:10:51.993373+0000","last_active":"2023-11-28T11:10:52.377576+0000","last_peered":"2023-11-28T11:10:52.377576+0000","last_clean":"2023-11-28T11:10:52.377576+0000","last_became_active":"2023-11-28T11:10:51.991960+0000","last_became_peered":"2023-11-28T11:10:51.991960+0000","last_unstale":"2023-11-28T11:10:52.377576+0000","last_undegraded":"2023-11-28T11:10:52.377576+0000","last_fullsized":"2023-11-28T11:10:52.377576+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:29.954884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377049+0000","last_change":"2023-11-28T11:10:52.383141+0000","last_active":"2023-11-28T11:10:53.377049+0000","last_peered":"2023-11-28T11:10:53.377049+0000","last_clean":"2023-11-28T11:10:53.377049+0000","last_became_active":"2023-11-28T11:10:52.382460+0000","last_became_peered":"2023-11-28T11:10:52.382460+0000","last_unstale":"2023-11-28T11:10:53.377049+0000","last_undegraded":"2023-11-28T11:10:53.377049+0000","last_fullsized":"2023-11-28T11:10:53.377049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:24.111335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.393434+0000","last_change":"2023-11-28T11:10:51.999127+0000","last_active":"2023-11-28T11:10:52.393434+0000","last_peered":"2023-11-28T11:10:52.393434+0000","last_clean":"2023-11-28T11:10:52.393434+0000","last_became_active":"2023-11-28T11:10:51.998846+0000","last_became_peered":"2023-11-28T11:10:51.998846+0000","last_unstale":"2023-11-28T11:10:52.393434+0000","last_undegraded":"2023-11-28T11:10:52.393434+0000","last_fullsized":"2023-11-28T11:10:52.393434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:41:07.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848638+0000","last_change":"2023-11-28T11:10:52.397124+0000","last_active":"2023-11-28T11:10:53.848638+0000","last_peered":"2023-11-28T11:10:53.848638+0000","last_clean":"2023-11-28T11:10:53.848638+0000","last_became_active":"2023-11-28T11:10:52.396862+0000","last_became_peered":"2023-11-28T11:10:52.396862+0000","last_unstale":"2023-11-28T11:10:53.848638+0000","last_undegraded":"2023-11-28T11:10:53.848638+0000","last_fullsized":"2023-11-28T11:10:53.848638+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:12.100298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"62'8","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.402319+0000","last_change":"2023-11-28T11:10:51.996381+0000","last_active":"2023-11-28T11:10:52.402319+0000","last_peered":"2023-11-28T11:10:52.402319+0000","last_clean":"2023-11-28T11:10:52.402319+0000","last_became_active":"2023-11-28T11:10:51.996095+0000","last_became_peered":"2023-11-28T11:10:51.996095+0000","last_unstale":"2023-11-28T11:10:52.402319+0000","last_undegraded":"2023-11-28T11:10:52.402319+0000","last_fullsized":"2023-11-28T11:10:52.402319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:38.465853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847836+0000","last_change":"2023-11-28T11:10:52.394438+0000","last_active":"2023-11-28T11:10:53.847836+0000","last_peered":"2023-11-28T11:10:53.847836+0000","last_clean":"2023-11-28T11:10:53.847836+0000","last_became_active":"2023-11-28T11:10:52.394191+0000","last_became_peered":"2023-11-28T11:10:52.394191+0000","last_unstale":"2023-11-28T11:10:53.847836+0000","last_undegraded":"2023-11-28T11:10:53.847836+0000","last_fullsized":"2023-11-28T11:10:53.847836+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:35.306404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.391213+0000","last_change":"2023-11-28T11:10:51.994649+0000","last_active":"2023-11-28T11:10:52.391213+0000","last_peered":"2023-11-28T11:10:52.391213+0000","last_clean":"2023-11-28T11:10:52.391213+0000","last_became_active":"2023-11-28T11:10:51.994387+0000","last_became_peered":"2023-11-28T11:10:51.994387+0000","last_unstale":"2023-11-28T11:10:52.391213+0000","last_undegraded":"2023-11-28T11:10:52.391213+0000","last_fullsized":"2023-11-28T11:10:52.391213+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:44.608724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848232+0000","last_change":"2023-11-28T11:10:52.384083+0000","last_active":"2023-11-28T11:10:53.848232+0000","last_peered":"2023-11-28T11:10:53.848232+0000","last_clean":"2023-11-28T11:10:53.848232+0000","last_became_active":"2023-11-28T11:10:52.383213+0000","last_became_peered":"2023-11-28T11:10:52.383213+0000","last_unstale":"2023-11-28T11:10:53.848232+0000","last_undegraded":"2023-11-28T11:10:53.848232+0000","last_fullsized":"2023-11-28T11:10:53.848232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:00.199905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377925+0000","last_change":"2023-11-28T11:10:51.993248+0000","last_active":"2023-11-28T11:10:52.377925+0000","last_peered":"2023-11-28T11:10:52.377925+0000","last_clean":"2023-11-28T11:10:52.377925+0000","last_became_active":"2023-11-28T11:10:51.991753+0000","last_became_peered":"2023-11-28T11:10:51.991753+0000","last_unstale":"2023-11-28T11:10:52.377925+0000","last_undegraded":"2023-11-28T11:10:52.377925+0000","last_fullsized":"2023-11-28T11:10:52.377925+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:35.847896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.378406+0000","last_change":"2023-11-28T11:10:52.383673+0000","last_active":"2023-11-28T11:10:53.378406+0000","last_peered":"2023-11-28T11:10:53.378406+0000","last_clean":"2023-11-28T11:10:53.378406+0000","last_became_active":"2023-11-28T11:10:52.381608+0000","last_became_peered":"2023-11-28T11:10:52.381608+0000","last_unstale":"2023-11-28T11:10:53.378406+0000","last_undegraded":"2023-11-28T11:10:53.378406+0000","last_fullsized":"2023-11-28T11:10:53.378406+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:07.493993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.373540+0000","last_change":"2023-11-28T11:10:52.000024+0000","last_active":"2023-11-28T11:10:52.373540+0000","last_peered":"2023-11-28T11:10:52.373540+0000","last_clean":"2023-11-28T11:10:52.373540+0000","last_became_active":"2023-11-28T11:10:51.999765+0000","last_became_peered":"2023-11-28T11:10:51.999765+0000","last_unstale":"2023-11-28T11:10:52.373540+0000","last_undegraded":"2023-11-28T11:10:52.373540+0000","last_fullsized":"2023-11-28T11:10:52.373540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:56.382110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376866+0000","last_change":"2023-11-28T11:10:52.388835+0000","last_active":"2023-11-28T11:10:53.376866+0000","last_peered":"2023-11-28T11:10:53.376866+0000","last_clean":"2023-11-28T11:10:53.376866+0000","last_became_active":"2023-11-28T11:10:52.388560+0000","last_became_peered":"2023-11-28T11:10:52.388560+0000","last_unstale":"2023-11-28T11:10:53.376866+0000","last_undegraded":"2023-11-28T11:10:53.376866+0000","last_fullsized":"2023-11-28T11:10:53.376866+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:03:08.782678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.383430+0000","last_change":"2023-11-28T11:10:51.998939+0000","last_active":"2023-11-28T11:10:52.383430+0000","last_peered":"2023-11-28T11:10:52.383430+0000","last_clean":"2023-11-28T11:10:52.383430+0000","last_became_active":"2023-11-28T11:10:51.998718+0000","last_became_peered":"2023-11-28T11:10:51.998718+0000","last_unstale":"2023-11-28T11:10:52.383430+0000","last_undegraded":"2023-11-28T11:10:52.383430+0000","last_fullsized":"2023-11-28T11:10:52.383430+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:11.021567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376607+0000","last_change":"2023-11-28T11:10:52.387164+0000","last_active":"2023-11-28T11:10:53.376607+0000","last_peered":"2023-11-28T11:10:53.376607+0000","last_clean":"2023-11-28T11:10:53.376607+0000","last_became_active":"2023-11-28T11:10:52.386857+0000","last_became_peered":"2023-11-28T11:10:52.386857+0000","last_unstale":"2023-11-28T11:10:53.376607+0000","last_undegraded":"2023-11-28T11:10:53.376607+0000","last_fullsized":"2023-11-28T11:10:53.376607+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:15:03.899294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.389892+0000","last_change":"2023-11-28T11:10:51.999035+0000","last_active":"2023-11-28T11:10:52.389892+0000","last_peered":"2023-11-28T11:10:52.389892+0000","last_clean":"2023-11-28T11:10:52.389892+0000","last_became_active":"2023-11-28T11:10:51.998758+0000","last_became_peered":"2023-11-28T11:10:51.998758+0000","last_unstale":"2023-11-28T11:10:52.389892+0000","last_undegraded":"2023-11-28T11:10:52.389892+0000","last_fullsized":"2023-11-28T11:10:52.389892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:11.021567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848172+0000","last_change":"2023-11-28T11:10:52.401133+0000","last_active":"2023-11-28T11:10:53.848172+0000","last_peered":"2023-11-28T11:10:53.848172+0000","last_clean":"2023-11-28T11:10:53.848172+0000","last_became_active":"2023-11-28T11:10:52.400869+0000","last_became_peered":"2023-11-28T11:10:52.400869+0000","last_unstale":"2023-11-28T11:10:53.848172+0000","last_undegraded":"2023-11-28T11:10:53.848172+0000","last_fullsized":"2023-11-28T11:10:53.848172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:38.996786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.371070+0000","last_change":"2023-11-28T11:10:51.989318+0000","last_active":"2023-11-28T11:10:52.371070+0000","last_peered":"2023-11-28T11:10:52.371070+0000","last_clean":"2023-11-28T11:10:52.371070+0000","last_became_active":"2023-11-28T11:10:51.988981+0000","last_became_peered":"2023-11-28T11:10:51.988981+0000","last_unstale":"2023-11-28T11:10:52.371070+0000","last_undegraded":"2023-11-28T11:10:52.371070+0000","last_fullsized":"2023-11-28T11:10:52.371070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:05.195466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848013+0000","last_change":"2023-11-28T11:10:52.387544+0000","last_active":"2023-11-28T11:10:53.848013+0000","last_peered":"2023-11-28T11:10:53.848013+0000","last_clean":"2023-11-28T11:10:53.848013+0000","last_became_active":"2023-11-28T11:10:52.387302+0000","last_became_peered":"2023-11-28T11:10:52.387302+0000","last_unstale":"2023-11-28T11:10:53.848013+0000","last_undegraded":"2023-11-28T11:10:53.848013+0000","last_fullsized":"2023-11-28T11:10:53.848013+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:11:29.529381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.370973+0000","last_change":"2023-11-28T11:10:51.997542+0000","last_active":"2023-11-28T11:10:52.370973+0000","last_peered":"2023-11-28T11:10:52.370973+0000","last_clean":"2023-11-28T11:10:52.370973+0000","last_became_active":"2023-11-28T11:10:51.997247+0000","last_became_peered":"2023-11-28T11:10:51.997247+0000","last_unstale":"2023-11-28T11:10:52.370973+0000","last_undegraded":"2023-11-28T11:10:52.370973+0000","last_fullsized":"2023-11-28T11:10:52.370973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:35.988439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847922+0000","last_change":"2023-11-28T11:10:52.388406+0000","last_active":"2023-11-28T11:10:53.847922+0000","last_peered":"2023-11-28T11:10:53.847922+0000","last_clean":"2023-11-28T11:10:53.847922+0000","last_became_active":"2023-11-28T11:10:52.388066+0000","last_became_peered":"2023-11-28T11:10:52.388066+0000","last_unstale":"2023-11-28T11:10:53.847922+0000","last_undegraded":"2023-11-28T11:10:53.847922+0000","last_fullsized":"2023-11-28T11:10:53.847922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:43.460660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.381424+0000","last_change":"2023-11-28T11:10:51.995664+0000","last_active":"2023-11-28T11:10:52.381424+0000","last_peered":"2023-11-28T11:10:52.381424+0000","last_clean":"2023-11-28T11:10:52.381424+0000","last_became_active":"2023-11-28T11:10:51.995401+0000","last_became_peered":"2023-11-28T11:10:51.995401+0000","last_unstale":"2023-11-28T11:10:52.381424+0000","last_undegraded":"2023-11-28T11:10:52.381424+0000","last_fullsized":"2023-11-28T11:10:52.381424+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:37.815667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848272+0000","last_change":"2023-11-28T11:10:52.398149+0000","last_active":"2023-11-28T11:10:53.848272+0000","last_peered":"2023-11-28T11:10:53.848272+0000","last_clean":"2023-11-28T11:10:53.848272+0000","last_became_active":"2023-11-28T11:10:52.397875+0000","last_became_peered":"2023-11-28T11:10:52.397875+0000","last_unstale":"2023-11-28T11:10:53.848272+0000","last_undegraded":"2023-11-28T11:10:53.848272+0000","last_fullsized":"2023-11-28T11:10:53.848272+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:00.423890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.386719+0000","last_change":"2023-11-28T11:10:51.991028+0000","last_active":"2023-11-28T11:10:52.386719+0000","last_peered":"2023-11-28T11:10:52.386719+0000","last_clean":"2023-11-28T11:10:52.386719+0000","last_became_active":"2023-11-28T11:10:51.990741+0000","last_became_peered":"2023-11-28T11:10:51.990741+0000","last_unstale":"2023-11-28T11:10:52.386719+0000","last_undegraded":"2023-11-28T11:10:52.386719+0000","last_fullsized":"2023-11-28T11:10:52.386719+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:00.060672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.383316+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:10:52.383316+0000","last_peered":"2023-11-28T11:10:52.383316+0000","last_clean":"2023-11-28T11:10:52.383316+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:10:52.383316+0000","last_undegraded":"2023-11-28T11:10:52.383316+0000","last_fullsized":"2023-11-28T11:10:52.383316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377047+0000","last_change":"2023-11-28T11:10:52.387511+0000","last_active":"2023-11-28T11:10:53.377047+0000","last_peered":"2023-11-28T11:10:53.377047+0000","last_clean":"2023-11-28T11:10:53.377047+0000","last_became_active":"2023-11-28T11:10:52.387143+0000","last_became_peered":"2023-11-28T11:10:52.387143+0000","last_unstale":"2023-11-28T11:10:53.377047+0000","last_undegraded":"2023-11-28T11:10:53.377047+0000","last_fullsized":"2023-11-28T11:10:53.377047+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:01.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.373304+0000","last_change":"2023-11-28T11:10:51.995232+0000","last_active":"2023-11-28T11:10:52.373304+0000","last_peered":"2023-11-28T11:10:52.373304+0000","last_clean":"2023-11-28T11:10:52.373304+0000","last_became_active":"2023-11-28T11:10:51.994941+0000","last_became_peered":"2023-11-28T11:10:51.994941+0000","last_unstale":"2023-11-28T11:10:52.373304+0000","last_undegraded":"2023-11-28T11:10:52.373304+0000","last_fullsized":"2023-11-28T11:10:52.373304+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:11:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377767+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:10:52.377767+0000","last_peered":"2023-11-28T11:10:52.377767+0000","last_clean":"2023-11-28T11:10:52.377767+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:10:52.377767+0000","last_undegraded":"2023-11-28T11:10:52.377767+0000","last_fullsized":"2023-11-28T11:10:52.377767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377867+0000","last_change":"2023-11-28T11:10:52.388289+0000","last_active":"2023-11-28T11:10:53.377867+0000","last_peered":"2023-11-28T11:10:53.377867+0000","last_clean":"2023-11-28T11:10:53.377867+0000","last_became_active":"2023-11-28T11:10:52.387979+0000","last_became_peered":"2023-11-28T11:10:52.387979+0000","last_unstale":"2023-11-28T11:10:53.377867+0000","last_undegraded":"2023-11-28T11:10:53.377867+0000","last_fullsized":"2023-11-28T11:10:53.377867+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:06.203049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377681+0000","last_change":"2023-11-28T11:10:51.993205+0000","last_active":"2023-11-28T11:10:52.377681+0000","last_peered":"2023-11-28T11:10:52.377681+0000","last_clean":"2023-11-28T11:10:52.377681+0000","last_became_active":"2023-11-28T11:10:51.991562+0000","last_became_peered":"2023-11-28T11:10:51.991562+0000","last_unstale":"2023-11-28T11:10:52.377681+0000","last_undegraded":"2023-11-28T11:10:52.377681+0000","last_fullsized":"2023-11-28T11:10:52.377681+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:38.465853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377553+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:10:52.377553+0000","last_peered":"2023-11-28T11:10:52.377553+0000","last_clean":"2023-11-28T11:10:52.377553+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:10:52.377553+0000","last_undegraded":"2023-11-28T11:10:52.377553+0000","last_fullsized":"2023-11-28T11:10:52.377553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.377169+0000","last_change":"2023-11-28T11:10:52.387614+0000","last_active":"2023-11-28T11:10:53.377169+0000","last_peered":"2023-11-28T11:10:53.377169+0000","last_clean":"2023-11-28T11:10:53.377169+0000","last_became_active":"2023-11-28T11:10:52.387338+0000","last_became_peered":"2023-11-28T11:10:52.387338+0000","last_unstale":"2023-11-28T11:10:53.377169+0000","last_undegraded":"2023-11-28T11:10:53.377169+0000","last_fullsized":"2023-11-28T11:10:53.377169+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:36.855479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.370866+0000","last_change":"2023-11-28T11:10:51.985858+0000","last_active":"2023-11-28T11:10:52.370866+0000","last_peered":"2023-11-28T11:10:52.370866+0000","last_clean":"2023-11-28T11:10:52.370866+0000","last_became_active":"2023-11-28T11:10:51.985623+0000","last_became_peered":"2023-11-28T11:10:51.985623+0000","last_unstale":"2023-11-28T11:10:52.370866+0000","last_undegraded":"2023-11-28T11:10:52.370866+0000","last_fullsized":"2023-11-28T11:10:52.370866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:05:42.312941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.377572+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:10:52.377572+0000","last_peered":"2023-11-28T11:10:52.377572+0000","last_clean":"2023-11-28T11:10:52.377572+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:10:52.377572+0000","last_undegraded":"2023-11-28T11:10:52.377572+0000","last_fullsized":"2023-11-28T11:10:52.377572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":204,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.389688+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:10:52.389688+0000","last_peered":"2023-11-28T11:10:52.389688+0000","last_clean":"2023-11-28T11:10:52.389688+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:10:52.389688+0000","last_undegraded":"2023-11-28T11:10:52.389688+0000","last_fullsized":"2023-11-28T11:10:52.389688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.847400+0000","last_change":"2023-11-28T11:10:52.381996+0000","last_active":"2023-11-28T11:10:53.847400+0000","last_peered":"2023-11-28T11:10:53.847400+0000","last_clean":"2023-11-28T11:10:53.847400+0000","last_became_active":"2023-11-28T11:10:52.381358+0000","last_became_peered":"2023-11-28T11:10:52.381358+0000","last_unstale":"2023-11-28T11:10:53.847400+0000","last_undegraded":"2023-11-28T11:10:53.847400+0000","last_fullsized":"2023-11-28T11:10:53.847400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:48.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.386941+0000","last_change":"2023-11-28T11:10:51.996342+0000","last_active":"2023-11-28T11:10:52.386941+0000","last_peered":"2023-11-28T11:10:52.386941+0000","last_clean":"2023-11-28T11:10:52.386941+0000","last_became_active":"2023-11-28T11:10:51.995997+0000","last_became_peered":"2023-11-28T11:10:51.995997+0000","last_unstale":"2023-11-28T11:10:52.386941+0000","last_undegraded":"2023-11-28T11:10:52.386941+0000","last_fullsized":"2023-11-28T11:10:52.386941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:35.988439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.389449+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:10:52.389449+0000","last_peered":"2023-11-28T11:10:52.389449+0000","last_clean":"2023-11-28T11:10:52.389449+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:10:52.389449+0000","last_undegraded":"2023-11-28T11:10:52.389449+0000","last_fullsized":"2023-11-28T11:10:52.389449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376952+0000","last_change":"2023-11-28T11:10:52.397798+0000","last_active":"2023-11-28T11:10:53.376952+0000","last_peered":"2023-11-28T11:10:53.376952+0000","last_clean":"2023-11-28T11:10:53.376952+0000","last_became_active":"2023-11-28T11:10:52.397532+0000","last_became_peered":"2023-11-28T11:10:52.397532+0000","last_unstale":"2023-11-28T11:10:53.376952+0000","last_undegraded":"2023-11-28T11:10:53.376952+0000","last_fullsized":"2023-11-28T11:10:53.376952+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:38.823250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.398098+0000","last_change":"2023-11-28T11:10:52.000553+0000","last_active":"2023-11-28T11:10:52.398098+0000","last_peered":"2023-11-28T11:10:52.398098+0000","last_clean":"2023-11-28T11:10:52.398098+0000","last_became_active":"2023-11-28T11:10:52.000223+0000","last_became_peered":"2023-11-28T11:10:52.000223+0000","last_unstale":"2023-11-28T11:10:52.398098+0000","last_undegraded":"2023-11-28T11:10:52.398098+0000","last_fullsized":"2023-11-28T11:10:52.398098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:37.815667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.391001+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:10:52.391001+0000","last_peered":"2023-11-28T11:10:52.391001+0000","last_clean":"2023-11-28T11:10:52.391001+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:10:52.391001+0000","last_undegraded":"2023-11-28T11:10:52.391001+0000","last_fullsized":"2023-11-28T11:10:52.391001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.848738+0000","last_change":"2023-11-28T11:10:52.388031+0000","last_active":"2023-11-28T11:10:53.848738+0000","last_peered":"2023-11-28T11:10:53.848738+0000","last_clean":"2023-11-28T11:10:53.848738+0000","last_became_active":"2023-11-28T11:10:52.387685+0000","last_became_peered":"2023-11-28T11:10:52.387685+0000","last_unstale":"2023-11-28T11:10:53.848738+0000","last_undegraded":"2023-11-28T11:10:53.848738+0000","last_fullsized":"2023-11-28T11:10:53.848738+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:19:57.624643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.399510+0000","last_change":"2023-11-28T11:10:51.998733+0000","last_active":"2023-11-28T11:10:52.399510+0000","last_peered":"2023-11-28T11:10:52.399510+0000","last_clean":"2023-11-28T11:10:52.399510+0000","last_became_active":"2023-11-28T11:10:51.998123+0000","last_became_peered":"2023-11-28T11:10:51.998123+0000","last_unstale":"2023-11-28T11:10:52.399510+0000","last_undegraded":"2023-11-28T11:10:52.399510+0000","last_fullsized":"2023-11-28T11:10:52.399510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:23.426647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.381755+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:10:52.381755+0000","last_peered":"2023-11-28T11:10:52.381755+0000","last_clean":"2023-11-28T11:10:52.381755+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:10:52.381755+0000","last_undegraded":"2023-11-28T11:10:52.381755+0000","last_fullsized":"2023-11-28T11:10:52.381755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T11:10:53.376982+0000","last_change":"2023-11-28T11:10:52.389489+0000","last_active":"2023-11-28T11:10:53.376982+0000","last_peered":"2023-11-28T11:10:53.376982+0000","last_clean":"2023-11-28T11:10:53.376982+0000","last_became_active":"2023-11-28T11:10:52.387877+0000","last_became_peered":"2023-11-28T11:10:52.387877+0000","last_unstale":"2023-11-28T11:10:53.376982+0000","last_undegraded":"2023-11-28T11:10:53.376982+0000","last_fullsized":"2023-11-28T11:10:53.376982+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:51.971628+0000","last_clean_scrub_stamp":"2023-11-28T11:10:51.971628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:08:26.596352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.371082+0000","last_change":"2023-11-28T11:10:51.992409+0000","last_active":"2023-11-28T11:10:52.371082+0000","last_peered":"2023-11-28T11:10:52.371082+0000","last_clean":"2023-11-28T11:10:52.371082+0000","last_became_active":"2023-11-28T11:10:51.992031+0000","last_became_peered":"2023-11-28T11:10:51.992031+0000","last_unstale":"2023-11-28T11:10:52.371082+0000","last_undegraded":"2023-11-28T11:10:52.371082+0000","last_fullsized":"2023-11-28T11:10:52.371082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:50.964045+0000","last_clean_scrub_stamp":"2023-11-28T11:10:50.964045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:11:56.382110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:10:52.380028+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:10:52.380028+0000","last_peered":"2023-11-28T11:10:52.380028+0000","last_clean":"2023-11-28T11:10:52.380028+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:10:52.380028+0000","last_undegraded":"2023-11-28T11:10:52.380028+0000","last_fullsized":"2023-11-28T11:10:52.380028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10880,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360960,"statfs":{"total":96636764160,"available":96625623040,"internally_reserved":0,"allocated":1368064,"data_stored":925048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92200000000000004,"5min":1.006,"15min":1.006},"last":0.27800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.67500000000000004,"15min":0.69999999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71099999999999997,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.747,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.35199999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0349999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.432},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97199999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.39700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.33700000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.86699999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.38}]},{"osd":5,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92100000000000004,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.37},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.443}]},{"osd":6,"last update":"Tue Nov 28 11:10:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.47299999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.52000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9328,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362512,"statfs":{"total":96636764160,"available":96627212288,"internally_reserved":0,"allocated":761856,"data_stored":333954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69499999999999995,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.70299999999999996,"15min":0.73399999999999999},"min":{"1min":0.36299999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76800000000000002,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.69099999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.81599999999999995,"15min":0.85999999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76500000000000001,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.77000000000000002,"15min":0.80500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79800000000000004,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.78900000000000003,"15min":0.83599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67200000000000004,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.68899999999999995,"15min":0.71699999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68400000000000005,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.76800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.755,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.65400000000000003,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65700000000000003,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.755}]},{"osd":4,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.85099999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.379,"5min":0.373,"15min":0.373},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.88800000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76900000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.86099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9544,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362296,"statfs":{"total":96636764160,"available":96626991104,"internally_reserved":0,"allocated":786432,"data_stored":336216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.59999999999999998,"15min":0.64600000000000002},"min":{"1min":0.221,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.504,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.31},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.377,"15min":0.38200000000000001},"min":{"1min":0.248,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.46100000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.29599999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.64200000000000002,"15min":0.68999999999999995},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56999999999999995,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51900000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.35499999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42899999999999999,"15min":0.435},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58899999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.375,"15min":0.375},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54200000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38500000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.3120000000000001,"15min":1.431},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79500000000000004,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78800000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":1.3440000000000001,"15min":1.462},"min":{"1min":0.47799999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93600000000000005,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":1.3500000000000001,"15min":1.4670000000000001},"min":{"1min":0.498,"5min":0.439,"15min":0.439},"max":{"1min":0.95999999999999996,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.60199999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.48599999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94399999999999995,"5min":1.018,"15min":1.018},"last":0.496},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.44700000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14768,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94357072,"statfs":{"total":96636764160,"available":96621641728,"internally_reserved":0,"allocated":1351680,"data_stored":924288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.48599999999999999,"5min":0.502,"15min":0.502},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.44400000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.26100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53600000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.378,"15min":0.378},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.50900000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.29699999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47799999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.56699999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.311}]},{"osd":4,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58999999999999997,"15min":0.59299999999999997},"min":{"1min":0.38800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66700000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.46700000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.35899999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.434,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.39600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.432},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.47399999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:10:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.62,"15min":0.62},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.81399999999999995,"5min":1.026,"15min":1.026},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.41699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9452,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362388,"statfs":{"total":96636764160,"available":96627085312,"internally_reserved":0,"allocated":757760,"data_stored":333920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.32500000000000001,"15min":0.316},"min":{"1min":0.22,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.372,"15min":0.36599999999999999},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.33900000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.34899999999999998,"15min":0.33500000000000002},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33700000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.29299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.312},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.36899999999999999},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.35299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.437}]},{"osd":6,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.503},{"interface":"front","average":{"1min":0.63,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:13 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.44600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9772,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362068,"statfs":{"total":96636764160,"available":96626757632,"internally_reserved":0,"allocated":757760,"data_stored":333920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.42999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.31900000000000001,"15min":0.317},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.502,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.35399999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.56000000000000005,"5min":1.365,"15min":1.365},"last":0.33200000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48499999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.31}]},{"osd":4,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.495},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42899999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82999999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46000000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81200000000000006,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85999999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.48099999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9592,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362248,"statfs":{"total":96636764160,"available":96626941952,"internally_reserved":0,"allocated":770048,"data_stored":335468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69699999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.60099999999999998,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.9740000000000002,"5min":4.9740000000000002,"15min":4.9740000000000002},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":1.115,"15min":1.2070000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69699999999999995,"5min":12.263,"15min":12.263},"last":0.44700000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.505},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":1.236,"15min":1.236},"last":0.41399999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5600000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":4.3710000000000004,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.59499999999999997,"15min":0.55000000000000004},"min":{"1min":0.40200000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":5.0220000000000002,"5min":5.0220000000000002,"15min":5.0220000000000002},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.52000000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.65600000000000003,"15min":0.64100000000000001},"min":{"1min":0.55700000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.60199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9520,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362320,"statfs":{"total":96636764160,"available":96627015680,"internally_reserved":0,"allocated":761856,"data_stored":333942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.434}]},{"osd":1,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53100000000000003},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.504,"15min":0.505},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.35199999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.495},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.36899999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.66300000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.438,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.497,"15min":0.497},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72999999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52800000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.67500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72899999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.56799999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:10:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.441,"5min":0.374,"15min":0.374},"max":{"1min":0.67700000000000005,"5min":0.753,"15min":0.753},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.747,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.51200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:11:01.340 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:11:01.341 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:11:01.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.502+0000 7fe480bf3700 1 Processor -- start 2023-11-28T11:11:01.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.502+0000 7fe480bf3700 1 -- start start 2023-11-28T11:11:01.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.503+0000 7fe480bf3700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe47c0cb0e0 0x7fe47c065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.503+0000 7fe480bf3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe47c0cec70 0x7fe47c06d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.503+0000 7fe480bf3700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47c066580 con 0x7fe47c06ec90 2023-11-28T11:11:01.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.503+0000 7fe480bf3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47c0cded0 con 0x7fe47c0cec70 2023-11-28T11:11:01.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.503+0000 7fe480bf3700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47c0ce1b0 con 0x7fe47c0cb0e0 2023-11-28T11:11:01.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.503+0000 7fe47ad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe47c0cec70 0x7fe47c06d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.503+0000 7fe47ad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe47c0cec70 0x7fe47c06d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35768/0 (socket says 172.21.15.156:35768) 2023-11-28T11:11:01.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.503+0000 7fe47ad9d700 1 -- 172.21.15.156:0/3508345405 learned_addr learned my addr 172.21.15.156:0/3508345405 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:01.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.504+0000 7fe479d9b700 1 --2- 172.21.15.156:0/3508345405 >> v2:172.21.15.186:3300/0 conn(0x7fe47c0cb0e0 0x7fe47c065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.504+0000 7fe47ad9d700 1 -- 172.21.15.156:0/3508345405 >> v2:172.21.15.186:3300/0 conn(0x7fe47c0cb0e0 msgr2=0x7fe47c065ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:01.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.504+0000 7fe47ad9d700 1 --2- 172.21.15.156:0/3508345405 >> v2:172.21.15.186:3300/0 conn(0x7fe47c0cb0e0 0x7fe47c065ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.504+0000 7fe47ad9d700 1 -- 172.21.15.156:0/3508345405 >> v1:172.21.15.186:6789/0 conn(0x7fe47c06ec90 legacy=0x7fe47c0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:01.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.504+0000 7fe47ad9d700 1 -- 172.21.15.156:0/3508345405 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c0ce490 con 0x7fe47c0cec70 2023-11-28T11:11:01.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.504+0000 7fe479d9b700 1 --2- 172.21.15.156:0/3508345405 >> v2:172.21.15.186:3300/0 conn(0x7fe47c0cb0e0 0x7fe47c065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:01.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.504+0000 7fe47ad9d700 1 --2- 172.21.15.156:0/3508345405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe47c0cec70 0x7fe47c06d230 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe47c070650 tx=0x7fe47400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac6b09809cf904d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:01.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.504+0000 7fe47959a700 1 -- 172.21.15.156:0/3508345405 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47400e070 con 0x7fe47c0cec70 2023-11-28T11:11:01.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.504+0000 7fe47959a700 1 -- 172.21.15.156:0/3508345405 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe474003a20 con 0x7fe47c0cec70 2023-11-28T11:11:01.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.504+0000 7fe47959a700 1 -- 172.21.15.156:0/3508345405 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe474003dd0 con 0x7fe47c0cec70 2023-11-28T11:11:01.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.504+0000 7fe480bf3700 1 -- 172.21.15.156:0/3508345405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe47c0cec70 msgr2=0x7fe47c06d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.504+0000 7fe480bf3700 1 --2- 172.21.15.156:0/3508345405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe47c0cec70 0x7fe47c06d230 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe47c070650 tx=0x7fe47400b330 comp rx=0 tx=0).stop 2023-11-28T11:11:01.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.505+0000 7fe480bf3700 1 -- 172.21.15.156:0/3508345405 shutdown_connections 2023-11-28T11:11:01.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.505+0000 7fe480bf3700 1 --2- 172.21.15.156:0/3508345405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe47c0cec70 0x7fe47c06d230 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.505+0000 7fe480bf3700 1 --2- 172.21.15.156:0/3508345405 >> v2:172.21.15.186:3300/0 conn(0x7fe47c0cb0e0 0x7fe47c065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.505+0000 7fe480bf3700 1 -- 172.21.15.156:0/3508345405 >> 172.21.15.156:0/3508345405 conn(0x7fe47c0b12b0 msgr2=0x7fe47c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:01.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.505+0000 7fe480bf3700 1 -- 172.21.15.156:0/3508345405 shutdown_connections 2023-11-28T11:11:01.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.505+0000 7fe480bf3700 1 -- 172.21.15.156:0/3508345405 wait complete. 2023-11-28T11:11:01.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.506+0000 7fe480bf3700 1 Processor -- start 2023-11-28T11:11:01.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.506+0000 7fe480bf3700 1 -- start start 2023-11-28T11:11:01.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe480bf3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe47c06ec90 0x7fe47c0c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe480bf3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe47c0cb0e0 0x7fe47c1e21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe480bf3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe47c0cec70 0x7fe47c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe480bf3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0c5650 con 0x7fe47c0cec70 2023-11-28T11:11:01.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe480bf3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0c37e0 con 0x7fe47c0cb0e0 2023-11-28T11:11:01.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe480bf3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0c3500 con 0x7fe47c06ec90 2023-11-28T11:11:01.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe47ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe47c0cec70 0x7fe47c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe47ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe47c0cec70 0x7fe47c06a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35774/0 (socket says 172.21.15.156:35774) 2023-11-28T11:11:01.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe47ad9d700 1 -- 172.21.15.156:0/3923993975 learned_addr learned my addr 172.21.15.156:0/3923993975 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:01.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe479d9b700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe47c0cb0e0 0x7fe47c1e21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe47a59c700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe47c06ec90 0x7fe47c0c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe47ad9d700 1 -- 172.21.15.156:0/3923993975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe47c06ec90 msgr2=0x7fe47c0c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe47ad9d700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe47c06ec90 0x7fe47c0c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe47ad9d700 1 -- 172.21.15.156:0/3923993975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe47c0cb0e0 msgr2=0x7fe47c1e21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe47ad9d700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe47c0cb0e0 0x7fe47c1e21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe47ad9d700 1 -- 172.21.15.156:0/3923993975 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe474009b30 con 0x7fe47c0cec70 2023-11-28T11:11:01.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe479d9b700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe47c0cb0e0 0x7fe47c1e21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:01.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe47a59c700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe47c06ec90 0x7fe47c0c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:01.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe47ad9d700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe47c0cec70 0x7fe47c06a270 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe47400b370 tx=0x7fe474004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:01.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe472ffd700 1 -- 172.21.15.156:0/3923993975 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe474004500 con 0x7fe47c0cec70 2023-11-28T11:11:01.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe472ffd700 1 -- 172.21.15.156:0/3923993975 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe474004680 con 0x7fe47c0cec70 2023-11-28T11:11:01.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe480bf3700 1 -- 172.21.15.156:0/3923993975 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47c0cb6c0 con 0x7fe47c0cec70 2023-11-28T11:11:01.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe472ffd700 1 -- 172.21.15.156:0/3923993975 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe474012430 con 0x7fe47c0cec70 2023-11-28T11:11:01.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.507+0000 7fe480bf3700 1 -- 172.21.15.156:0/3923993975 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47c0ce770 con 0x7fe47c0cec70 2023-11-28T11:11:01.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.508+0000 7fe480bf3700 1 -- 172.21.15.156:0/3923993975 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47c06f060 con 0x7fe47c0cec70 2023-11-28T11:11:01.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.509+0000 7fe472ffd700 1 -- 172.21.15.156:0/3923993975 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe474022030 con 0x7fe47c0cec70 2023-11-28T11:11:01.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.509+0000 7fe472ffd700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe45c0682f0 0x7fe45c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.509+0000 7fe47a59c700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe45c0682f0 0x7fe45c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.509+0000 7fe472ffd700 1 -- 172.21.15.156:0/3923993975 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe474087150 con 0x7fe47c0cec70 2023-11-28T11:11:01.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.510+0000 7fe47a59c700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe45c0682f0 0x7fe45c06a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe464006fd0 tx=0x7fe464008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:01.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.514+0000 7fe472ffd700 1 -- 172.21.15.156:0/3923993975 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe4740539b0 con 0x7fe47c0cec70 2023-11-28T11:11:01.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.769+0000 7fe480bf3700 1 -- 172.21.15.156:0/3923993975 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fe47c0c3950 con 0x7fe47c0cec70 2023-11-28T11:11:01.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.772+0000 7fe472ffd700 1 -- 172.21.15.156:0/3923993975 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe47c0c3950 con 0x7fe47c0cec70 2023-11-28T11:11:01.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.775+0000 7fe480bf3700 1 -- 172.21.15.156:0/3923993975 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe45c0682f0 msgr2=0x7fe45c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.775+0000 7fe480bf3700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe45c0682f0 0x7fe45c06a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe464006fd0 tx=0x7fe464008040 comp rx=0 tx=0).stop 2023-11-28T11:11:01.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.775+0000 7fe480bf3700 1 -- 172.21.15.156:0/3923993975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe47c0cec70 msgr2=0x7fe47c06a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.775+0000 7fe480bf3700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe47c0cec70 0x7fe47c06a270 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe47400b370 tx=0x7fe474004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.775+0000 7fe480bf3700 1 -- 172.21.15.156:0/3923993975 shutdown_connections 2023-11-28T11:11:01.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.775+0000 7fe480bf3700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe47c0cec70 0x7fe47c06a270 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.775+0000 7fe480bf3700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe45c0682f0 0x7fe45c06a7a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.775+0000 7fe480bf3700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe47c06ec90 0x7fe47c0c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.775+0000 7fe480bf3700 1 --2- 172.21.15.156:0/3923993975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe47c0cb0e0 0x7fe47c1e21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.775+0000 7fe480bf3700 1 -- 172.21.15.156:0/3923993975 >> 172.21.15.156:0/3923993975 conn(0x7fe47c0b12b0 msgr2=0x7fe47c0b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:01.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.775+0000 7fe480bf3700 1 -- 172.21.15.156:0/3923993975 shutdown_connections 2023-11-28T11:11:01.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.775+0000 7fe480bf3700 1 -- 172.21.15.156:0/3923993975 wait complete. 2023-11-28T11:11:01.777 INFO:teuthology.orchestra.run.smithi156.stderr:updated caps for client.0 2023-11-28T11:11:01.790 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:01.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.911+0000 7fe2b7198700 1 Processor -- start 2023-11-28T11:11:01.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.911+0000 7fe2b7198700 1 -- start start 2023-11-28T11:11:01.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.911+0000 7fe2b7198700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe2b00cbdf0 0x7fe2b00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.911+0000 7fe2b7198700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b00cc5f0 con 0x7fe2b006ba90 2023-11-28T11:11:01.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.911+0000 7fe2b7198700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b015d470 con 0x7fe2b015cef0 2023-11-28T11:11:01.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.911+0000 7fe2b7198700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b015d750 con 0x7fe2b00cbdf0 2023-11-28T11:11:01.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2affff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe2b006ba90 0x7fe2b00ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50658/0 (socket says 172.21.15.156:50658) 2023-11-28T11:11:01.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2affff700 1 -- 172.21.15.156:0/3822396422 learned_addr learned my addr 172.21.15.156:0/3822396422 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:01.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2b5735700 1 --2- 172.21.15.156:0/3822396422 >> v2:172.21.15.186:3300/0 conn(0x7fe2b00cbdf0 0x7fe2b00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622069638 0 0) 0x7fe2b00cc5f0 con 0x7fe2b006ba90 2023-11-28T11:11:01.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a00057c0 con 0x7fe2b006ba90 2023-11-28T11:11:01.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4060805955 0 0) 0x7fe2a00057c0 con 0x7fe2b006ba90 2023-11-28T11:11:01.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 >> v2:172.21.15.186:3300/0 conn(0x7fe2b00cbdf0 msgr2=0x7fe2b00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:01.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 --2- 172.21.15.156:0/3822396422 >> v2:172.21.15.186:3300/0 conn(0x7fe2b00cbdf0 0x7fe2b00af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 >> v1:172.21.15.186:6789/0 conn(0x7fe2b015cef0 legacy=0x7fe2b006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b015da30 con 0x7fe2b006ba90 2023-11-28T11:11:01.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fe29c001670 con 0x7fe2b006ba90 2023-11-28T11:11:01.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 >> v1:172.21.15.156:6789/0 conn(0x7fe2b006ba90 legacy=0x7fe2b00ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 --2- 172.21.15.156:0/3822396422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2a0005c10 0x7fe2a0006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 --2- 172.21.15.156:0/3822396422 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2a0006a60 0x7fe2a0006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 --2- 172.21.15.156:0/3822396422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2a0007410 0x7fe2a000dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b015d750 con 0x7fe2a0007410 2023-11-28T11:11:01.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a000e290 con 0x7fe2a0005c10 2023-11-28T11:11:01.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a000e450 con 0x7fe2a0006a60 2023-11-28T11:11:01.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2b5735700 1 --2- 172.21.15.156:0/3822396422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2a0007410 0x7fe2a000dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.912+0000 7fe2b4f34700 1 --2- 172.21.15.156:0/3822396422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2a0005c10 0x7fe2a0006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2affff700 1 --2- 172.21.15.156:0/3822396422 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2a0006a60 0x7fe2a0006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2b5735700 1 -- 172.21.15.156:0/3822396422 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2a0006a60 msgr2=0x7fe2a0006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:01.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2b5735700 1 --2- 172.21.15.156:0/3822396422 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2a0006a60 0x7fe2a0006ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2b5735700 1 -- 172.21.15.156:0/3822396422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2a0005c10 msgr2=0x7fe2a0006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:01.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2b5735700 1 --2- 172.21.15.156:0/3822396422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2a0005c10 0x7fe2a0006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2b5735700 1 -- 172.21.15.156:0/3822396422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a00057c0 con 0x7fe2a0007410 2023-11-28T11:11:01.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2b5735700 1 -- 172.21.15.156:0/3822396422 reap_dead start 2023-11-28T11:11:01.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2b5735700 1 --2- 172.21.15.156:0/3822396422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2a0007410 0x7fe2a000dc20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009fc0 tx=0x7fe2a0007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687874f9d6af3006 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:01.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8003a20 con 0x7fe2a0007410 2023-11-28T11:11:01.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a800f040 con 0x7fe2a0007410 2023-11-28T11:11:01.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2af7fe700 1 -- 172.21.15.156:0/3822396422 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8003de0 con 0x7fe2a0007410 2023-11-28T11:11:01.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2b7198700 1 -- 172.21.15.156:0/3822396422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2a0007410 msgr2=0x7fe2a000dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.913+0000 7fe2b7198700 1 --2- 172.21.15.156:0/3822396422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2a0007410 0x7fe2a000dc20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009fc0 tx=0x7fe2a0007a90 comp rx=0 tx=0).stop 2023-11-28T11:11:01.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.914+0000 7fe2b7198700 1 -- 172.21.15.156:0/3822396422 shutdown_connections 2023-11-28T11:11:01.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.914+0000 7fe2b7198700 1 --2- 172.21.15.156:0/3822396422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2a0007410 0x7fe2a000dc20 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.914+0000 7fe2b7198700 1 -- 172.21.15.156:0/3822396422 >> 172.21.15.156:0/3822396422 conn(0x7fe2b00d0000 msgr2=0x7fe2b00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:01.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.914+0000 7fe2b7198700 1 -- 172.21.15.156:0/3822396422 shutdown_connections 2023-11-28T11:11:01.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.914+0000 7fe2b7198700 1 -- 172.21.15.156:0/3822396422 wait complete. 2023-11-28T11:11:01.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.915+0000 7fe2b7198700 1 Processor -- start 2023-11-28T11:11:01.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.915+0000 7fe2b7198700 1 -- start start 2023-11-28T11:11:01.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.916+0000 7fe2b7198700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2a0007410 0x7fe2b0154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.916+0000 7fe2b7198700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2b0153d40 0x7fe2b00b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.916+0000 7fe2b7198700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b00c98d0 0x7fe2b00b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.916+0000 7fe2b7198700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00b4430 con 0x7fe2b00c98d0 2023-11-28T11:11:01.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.916+0000 7fe2b7198700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b01541b0 con 0x7fe2b0153d40 2023-11-28T11:11:01.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.916+0000 7fe2b7198700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00b3840 con 0x7fe2a0007410 2023-11-28T11:11:01.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.916+0000 7fe2b5735700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b00c98d0 0x7fe2b00b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.916+0000 7fe2b5735700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b00c98d0 0x7fe2b00b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35794/0 (socket says 172.21.15.156:35794) 2023-11-28T11:11:01.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.916+0000 7fe2b5735700 1 -- 172.21.15.156:0/1435318718 learned_addr learned my addr 172.21.15.156:0/1435318718 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:01.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.916+0000 7fe2affff700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2b0153d40 0x7fe2b00b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.917+0000 7fe2b4f34700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2a0007410 0x7fe2b0154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.917+0000 7fe2b5735700 1 -- 172.21.15.156:0/1435318718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2a0007410 msgr2=0x7fe2b0154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.917+0000 7fe2b5735700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2a0007410 0x7fe2b0154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.917+0000 7fe2b5735700 1 -- 172.21.15.156:0/1435318718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2b0153d40 msgr2=0x7fe2b00b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.917+0000 7fe2b5735700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2b0153d40 0x7fe2b00b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.917+0000 7fe2b5735700 1 -- 172.21.15.156:0/1435318718 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b015dff0 con 0x7fe2b00c98d0 2023-11-28T11:11:01.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.917+0000 7fe2b5735700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b00c98d0 0x7fe2b00b5000 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800eeb0 tx=0x7fe2a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:01.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.917+0000 7fe2ad7fa700 1 -- 172.21.15.156:0/1435318718 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8003e10 con 0x7fe2b00c98d0 2023-11-28T11:11:01.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.917+0000 7fe2ad7fa700 1 -- 172.21.15.156:0/1435318718 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a8004230 con 0x7fe2b00c98d0 2023-11-28T11:11:01.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.917+0000 7fe2b7198700 1 -- 172.21.15.156:0/1435318718 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b015dd10 con 0x7fe2b00c98d0 2023-11-28T11:11:01.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.917+0000 7fe2ad7fa700 1 -- 172.21.15.156:0/1435318718 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80045e0 con 0x7fe2b00c98d0 2023-11-28T11:11:01.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.917+0000 7fe2b7198700 1 -- 172.21.15.156:0/1435318718 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b00af400 con 0x7fe2b00c98d0 2023-11-28T11:11:01.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.918+0000 7fe2b7198700 1 -- 172.21.15.156:0/1435318718 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b00b4570 con 0x7fe2b00c98d0 2023-11-28T11:11:01.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.918+0000 7fe2ad7fa700 1 -- 172.21.15.156:0/1435318718 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe2a8004740 con 0x7fe2b00c98d0 2023-11-28T11:11:01.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.919+0000 7fe2ad7fa700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe2940682f0 0x7fe29406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.919+0000 7fe2ad7fa700 1 -- 172.21.15.156:0/1435318718 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fe2a8087070 con 0x7fe2b00c98d0 2023-11-28T11:11:01.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.919+0000 7fe2b4f34700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe2940682f0 0x7fe29406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.919+0000 7fe2b4f34700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe2940682f0 0x7fe29406a7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00c6bc0 tx=0x7fe2a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:01.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:01.923+0000 7fe2ad7fa700 1 -- 172.21.15.156:0/1435318718 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe2b00b4570 con 0x7fe2b00c98d0 2023-11-28T11:11:02.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.149+0000 7fe2b7198700 1 -- 172.21.15.156:0/1435318718 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2b00b4570 con 0x7fe2b00c98d0 2023-11-28T11:11:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.150+0000 7fe2ad7fa700 1 -- 172.21.15.156:0/1435318718 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4655 (secure 0 0 0) 0x7fe2b00b4570 con 0x7fe2b00c98d0 2023-11-28T11:11:02.151 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:02.151 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6006,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1991065152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1991065152},{"type":"v1","addr":"172.21.15.156:6836","nonce":1991065152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/43543030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":43543030},{"type":"v1","addr":"172.21.15.156:6837","nonce":43543030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/2480189069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2480189069},{"type":"v1","addr":"172.21.15.186:6834","nonce":2480189069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:52.367543+0000","modified":"2023-11-28T11:10:55.380411+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6021":{"gid":6021,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3811791267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3811791267},{"type":"v1","addr":"172.21.15.186:6835","nonce":3811791267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:11:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.152+0000 7fe2b7198700 1 -- 172.21.15.156:0/1435318718 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe2940682f0 msgr2=0x7fe29406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.152+0000 7fe2b7198700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe2940682f0 0x7fe29406a7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00c6bc0 tx=0x7fe2a4009040 comp rx=0 tx=0).stop 2023-11-28T11:11:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.152+0000 7fe2b7198700 1 -- 172.21.15.156:0/1435318718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b00c98d0 msgr2=0x7fe2b00b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.152+0000 7fe2b7198700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b00c98d0 0x7fe2b00b5000 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800eeb0 tx=0x7fe2a8009750 comp rx=0 tx=0).stop 2023-11-28T11:11:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.152+0000 7fe2b7198700 1 -- 172.21.15.156:0/1435318718 shutdown_connections 2023-11-28T11:11:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.152+0000 7fe2b7198700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2b00c98d0 0x7fe2b00b5000 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.152+0000 7fe2b7198700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe2940682f0 0x7fe29406a7a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.152+0000 7fe2b7198700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2a0007410 0x7fe2b0154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.152+0000 7fe2b7198700 1 --2- 172.21.15.156:0/1435318718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2b0153d40 0x7fe2b00b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.152+0000 7fe2b7198700 1 -- 172.21.15.156:0/1435318718 >> 172.21.15.156:0/1435318718 conn(0x7fe2b00d0000 msgr2=0x7fe2b00cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.152+0000 7fe2b7198700 1 -- 172.21.15.156:0/1435318718 shutdown_connections 2023-11-28T11:11:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.152+0000 7fe2b7198700 1 -- 172.21.15.156:0/1435318718 wait complete. 2023-11-28T11:11:02.155 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 31 2023-11-28T11:11:02.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:10:52.367543+0000', 'modified': '2023-11-28T11:10:55.380411+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6021}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6021': {'gid': 6021, 'name': 'd', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6835/3811791267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6833', 'nonce': 3811791267}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 3811791267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:11:02.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:11:02.569 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T11:11:02.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.692+0000 7fde6faaf700 1 Processor -- start 2023-11-28T11:11:02.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.692+0000 7fde6faaf700 1 -- start start 2023-11-28T11:11:02.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6faaf700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fde68071040 0x7fde68071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6faaf700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fde6806dee0 0x7fde6806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6faaf700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde6806cdc0 0x7fde6806d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6faaf700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde68071950 con 0x7fde68071040 2023-11-28T11:11:02.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6faaf700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6815d060 con 0x7fde6806dee0 2023-11-28T11:11:02.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6faaf700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6815d340 con 0x7fde6806cdc0 2023-11-28T11:11:02.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6d84b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fde68071040 0x7fde68071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6d84b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fde68071040 0x7fde68071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35798/0 (socket says 172.21.15.156:35798) 2023-11-28T11:11:02.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6d84b700 1 -- 172.21.15.156:0/4038695586 learned_addr learned my addr 172.21.15.156:0/4038695586 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6d04a700 1 --2- 172.21.15.156:0/4038695586 >> v2:172.21.15.186:3300/0 conn(0x7fde6806dee0 0x7fde6806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6e04c700 1 --2- 172.21.15.156:0/4038695586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde6806cdc0 0x7fde6806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6d84b700 1 -- 172.21.15.156:0/4038695586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde6806cdc0 msgr2=0x7fde6806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6d84b700 1 --2- 172.21.15.156:0/4038695586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde6806cdc0 0x7fde6806d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6d84b700 1 -- 172.21.15.156:0/4038695586 >> v2:172.21.15.186:3300/0 conn(0x7fde6806dee0 msgr2=0x7fde6806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6d84b700 1 --2- 172.21.15.156:0/4038695586 >> v2:172.21.15.186:3300/0 conn(0x7fde6806dee0 0x7fde6806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6d84b700 1 -- 172.21.15.156:0/4038695586 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6815d620 con 0x7fde68071040 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6d84b700 1 --2- 172.21.15.156:0/4038695586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde68071040 0x7fde68071410 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fde6806fb00 tx=0x7fde64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282cc71a870856 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.693+0000 7fde6c849700 1 -- 172.21.15.156:0/4038695586 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6400b870 con 0x7fde68071040 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.694+0000 7fde6c849700 1 -- 172.21.15.156:0/4038695586 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6400b9d0 con 0x7fde68071040 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.694+0000 7fde6c849700 1 -- 172.21.15.156:0/4038695586 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6400bd60 con 0x7fde68071040 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.694+0000 7fde6faaf700 1 -- 172.21.15.156:0/4038695586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde68071040 msgr2=0x7fde68071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.694+0000 7fde6faaf700 1 --2- 172.21.15.156:0/4038695586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde68071040 0x7fde68071410 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fde6806fb00 tx=0x7fde64009bd0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.694+0000 7fde6faaf700 1 -- 172.21.15.156:0/4038695586 shutdown_connections 2023-11-28T11:11:02.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.694+0000 7fde6faaf700 1 --2- 172.21.15.156:0/4038695586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde6806cdc0 0x7fde6806d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.694+0000 7fde6faaf700 1 --2- 172.21.15.156:0/4038695586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde68071040 0x7fde68071410 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.694+0000 7fde6faaf700 1 --2- 172.21.15.156:0/4038695586 >> v2:172.21.15.186:3300/0 conn(0x7fde6806dee0 0x7fde6806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.694+0000 7fde6faaf700 1 -- 172.21.15.156:0/4038695586 >> 172.21.15.156:0/4038695586 conn(0x7fde680d0000 msgr2=0x7fde680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:02.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.694+0000 7fde6faaf700 1 -- 172.21.15.156:0/4038695586 shutdown_connections 2023-11-28T11:11:02.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.694+0000 7fde6faaf700 1 -- 172.21.15.156:0/4038695586 wait complete. 2023-11-28T11:11:02.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.695+0000 7fde6faaf700 1 Processor -- start 2023-11-28T11:11:02.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.695+0000 7fde6faaf700 1 -- start start 2023-11-28T11:11:02.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.696+0000 7fde6faaf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde6806cdc0 0x7fde68154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.696+0000 7fde6faaf700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde6806dee0 0x7fde68152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.696+0000 7fde6faaf700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde68071040 0x7fde680b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.696+0000 7fde6faaf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde680c3aa0 con 0x7fde6806cdc0 2023-11-28T11:11:02.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.696+0000 7fde6faaf700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde680c3d80 con 0x7fde6806dee0 2023-11-28T11:11:02.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.696+0000 7fde6faaf700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde680c4060 con 0x7fde68071040 2023-11-28T11:11:02.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.696+0000 7fde6d84b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde6806cdc0 0x7fde68154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.696+0000 7fde6d84b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde6806cdc0 0x7fde68154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35802/0 (socket says 172.21.15.156:35802) 2023-11-28T11:11:02.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.696+0000 7fde6d84b700 1 -- 172.21.15.156:0/896581676 learned_addr learned my addr 172.21.15.156:0/896581676 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:02.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.696+0000 7fde6d04a700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde6806dee0 0x7fde68152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.696+0000 7fde6e04c700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde68071040 0x7fde680b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.697+0000 7fde6d84b700 1 -- 172.21.15.156:0/896581676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde68071040 msgr2=0x7fde680b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.697+0000 7fde6d84b700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde68071040 0x7fde680b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.697+0000 7fde6d84b700 1 -- 172.21.15.156:0/896581676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde6806dee0 msgr2=0x7fde68152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.697+0000 7fde6d84b700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde6806dee0 0x7fde68152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.697+0000 7fde6d84b700 1 -- 172.21.15.156:0/896581676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde64009550 con 0x7fde6806cdc0 2023-11-28T11:11:02.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.697+0000 7fde6d04a700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde6806dee0 0x7fde68152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:02.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.697+0000 7fde6d84b700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde6806cdc0 0x7fde68154420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fde64014040 tx=0x7fde64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:02.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.697+0000 7fde5e7fc700 1 -- 172.21.15.156:0/896581676 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64004520 con 0x7fde6806cdc0 2023-11-28T11:11:02.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.697+0000 7fde5e7fc700 1 -- 172.21.15.156:0/896581676 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6401e070 con 0x7fde6806cdc0 2023-11-28T11:11:02.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.697+0000 7fde5e7fc700 1 -- 172.21.15.156:0/896581676 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde640038c0 con 0x7fde6806cdc0 2023-11-28T11:11:02.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.697+0000 7fde6faaf700 1 -- 172.21.15.156:0/896581676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6806e870 con 0x7fde6806cdc0 2023-11-28T11:11:02.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.697+0000 7fde6faaf700 1 -- 172.21.15.156:0/896581676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6815d900 con 0x7fde6806cdc0 2023-11-28T11:11:02.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.698+0000 7fde5e7fc700 1 -- 172.21.15.156:0/896581676 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fde6400f050 con 0x7fde6806cdc0 2023-11-28T11:11:02.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.698+0000 7fde5e7fc700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fde4c0682f0 0x7fde4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.699+0000 7fde5e7fc700 1 -- 172.21.15.156:0/896581676 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fde64020070 con 0x7fde6806cdc0 2023-11-28T11:11:02.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.699+0000 7fde6d04a700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fde4c0682f0 0x7fde4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.699+0000 7fde6d04a700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fde4c0682f0 0x7fde4c06a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fde6806d9b0 tx=0x7fde58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:02.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.701+0000 7fde7bc38b80 1 Processor -- start 2023-11-28T11:11:02.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.701+0000 7fde7bc38b80 1 -- start start 2023-11-28T11:11:02.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.701+0000 7fde7bc38b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b3810 0x55739f089a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.701+0000 7fde7bc38b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b1130 0x55739f0b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.701+0000 7fde7bc38b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 0x55739f0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.701+0000 7fde7bc38b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ef73bc0 con 0x55739f0b4120 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.701+0000 7fde7bc38b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739efe2350 con 0x55739f0b3810 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.701+0000 7fde7bc38b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55739efdb880 con 0x55739f0b1130 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde6e04c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 0x55739f0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde6e04c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 0x55739f0cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35814/0 (socket says 172.21.15.156:35814) 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde6e04c700 1 -- 172.21.15.156:0/1330019679 learned_addr learned my addr 172.21.15.156:0/1330019679 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde6d84b700 1 --2- 172.21.15.156:0/1330019679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b3810 0x55739f089a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde6d04a700 1 --2- 172.21.15.156:0/1330019679 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b1130 0x55739f0b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde6e04c700 1 -- 172.21.15.156:0/1330019679 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b1130 msgr2=0x55739f0b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde6e04c700 1 --2- 172.21.15.156:0/1330019679 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b1130 0x55739f0b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde6e04c700 1 -- 172.21.15.156:0/1330019679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b3810 msgr2=0x55739f089a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde6e04c700 1 --2- 172.21.15.156:0/1330019679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b3810 0x55739f089a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde6e04c700 1 -- 172.21.15.156:0/1330019679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55739ee5d5e0 con 0x55739f0b4120 2023-11-28T11:11:02.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde6d84b700 1 --2- 172.21.15.156:0/1330019679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b3810 0x55739f089a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde6e04c700 1 --2- 172.21.15.156:0/1330019679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 0x55739f0cf770 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fde60009fd0 tx=0x7fde60007610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d463b6206b08b13e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde4b7fe700 1 -- 172.21.15.156:0/1330019679 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde60015670 con 0x55739f0b4120 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde4b7fe700 1 -- 172.21.15.156:0/1330019679 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6000f050 con 0x55739f0b4120 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.702+0000 7fde4b7fe700 1 -- 172.21.15.156:0/1330019679 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde600159f0 con 0x55739f0b4120 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.703+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1330019679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 msgr2=0x55739f0cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.703+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/1330019679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 0x55739f0cf770 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fde60009fd0 tx=0x7fde60007610 comp rx=0 tx=0).stop 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.703+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1330019679 shutdown_connections 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.703+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/1330019679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 0x55739f0cf770 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.703+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/1330019679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b3810 0x55739f089a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.703+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/1330019679 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b1130 0x55739f0b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.703+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1330019679 >> 172.21.15.156:0/1330019679 conn(0x55739f086d60 msgr2=0x55739f0876f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.703+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1330019679 shutdown_connections 2023-11-28T11:11:02.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.703+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1330019679 wait complete. 2023-11-28T11:11:02.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.703+0000 7fde7bc38b80 1 -- start start 2023-11-28T11:11:02.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.704+0000 7fde7bc38b80 1 Processor -- start 2023-11-28T11:11:02.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.704+0000 7fde7bc38b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 0x55739f0ab890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.704+0000 7fde7bc38b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b3810 0x55739f0d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.704+0000 7fde7bc38b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 0x55739f0d6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.704+0000 7fde7bc38b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ee5f8d0 con 0x55739f0b4120 2023-11-28T11:11:02.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.704+0000 7fde7bc38b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739f0838d0 con 0x55739f0b1130 2023-11-28T11:11:02.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.704+0000 7fde7bc38b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55739f0ac650 con 0x55739f0b3810 2023-11-28T11:11:02.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde7bc38b80 10 client.0 _ll_register_callbacks cb 0x55739ee81240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:02.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde6e04c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 0x55739f0d6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde6e04c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 0x55739f0d6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35820/0 (socket says 172.21.15.156:35820) 2023-11-28T11:11:02.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde6e04c700 1 -- 172.21.15.156:0/1690697588 learned_addr learned my addr 172.21.15.156:0/1690697588 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde6d84b700 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 0x55739f0ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde6d04a700 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b3810 0x55739f0d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde6e04c700 1 -- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b3810 msgr2=0x55739f0d62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde6e04c700 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b3810 0x55739f0d62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde6e04c700 1 -- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 msgr2=0x55739f0ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde6e04c700 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 0x55739f0ab890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde6e04c700 1 -- 172.21.15.156:0/1690697588 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde60009bd0 con 0x55739f0b4120 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde6d04a700 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b3810 0x55739f0d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde6e04c700 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 0x55739f0d6830 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fde60017040 tx=0x7fde60004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4646bd1e83614df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde48ff9700 1 -- 172.21.15.156:0/1690697588 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde60009a50 con 0x55739f0b4120 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde48ff9700 1 -- 172.21.15.156:0/1690697588 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fde60024030 con 0x55739f0b4120 2023-11-28T11:11:02.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1690697588 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55739efa9140 con 0x55739f0b4120 2023-11-28T11:11:02.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde48ff9700 1 -- 172.21.15.156:0/1690697588 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde60007000 con 0x55739f0b4120 2023-11-28T11:11:02.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde7bc38b80 10 client.6171 fetch_fsmap learned FSMap version 31 2023-11-28T11:11:02.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.705+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1690697588 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55739ee09d90 con 0x55739f0b4120 2023-11-28T11:11:02.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.706+0000 7fde48ff9700 1 -- 172.21.15.156:0/1690697588 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55739efa9140 con 0x55739f0b4120 2023-11-28T11:11:02.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.706+0000 7fde48ff9700 1 -- 172.21.15.156:0/1690697588 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fde60029640 con 0x55739f0b4120 2023-11-28T11:11:02.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.706+0000 7fde7bc38b80 10 client.6171 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T11:11:02.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.706+0000 7fde7bc38b80 10 client.6171 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:11:02.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.706+0000 7fde7bc38b80 20 client.6171 populate_metadata read hostname 'smithi156' 2023-11-28T11:11:02.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.706+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55739f0b0bc0 0x55739f0b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.706+0000 7fde7bc38b80 4 client.6171 mds_command: new command op to 6021 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:02.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.706+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1690697588 --> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55739ee5ff10 con 0x55739f0b0bc0 2023-11-28T11:11:02.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.706+0000 7fde6d84b700 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55739f0b0bc0 0x55739f0b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.706+0000 7fde48ff9700 10 client.6171 ms_handle_connect on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:02.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.706+0000 7fde6d84b700 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55739f0b0bc0 0x55739f0b64c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b136231aed9fb9c5 server_cookie=d7d6df265850a284 in_seq=0 out_seq=0 2023-11-28T11:11:02.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.707+0000 7fde48ff9700 1 -- 172.21.15.156:0/1690697588 <== mds.0 v2:172.21.15.186:6833/3811791267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55739ee5ff10 con 0x55739f0b0bc0 2023-11-28T11:11:02.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.707+0000 7fde48ff9700 10 client.6171 handle_command_reply: tid=0 2023-11-28T11:11:02.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.707+0000 7fde7bc38b80 1 client.6171 shutdown 2023-11-28T11:11:02.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.707+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1690697588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 msgr2=0x55739f0d6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.707+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 0x55739f0d6830 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fde60017040 tx=0x7fde60004b40 comp rx=0 tx=0).stop 2023-11-28T11:11:02.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.708+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1690697588 shutdown_connections 2023-11-28T11:11:02.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.708+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55739f0b0bc0 0x55739f0b64c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.708+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b4120 0x55739f0d6830 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.708+0000 7fde48ff9700 0 client.6171 ms_handle_reset on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:02.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.708+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 0x55739f0ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.708+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/1690697588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b3810 0x55739f0d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.708+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1690697588 >> 172.21.15.156:0/1690697588 conn(0x55739f086d60 msgr2=0x55739f089990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:02.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.708+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1690697588 shutdown_connections 2023-11-28T11:11:02.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.708+0000 7fde7bc38b80 1 -- 172.21.15.156:0/1690697588 wait complete. 2023-11-28T11:11:02.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.708+0000 7fde7bc38b80 20 client.6171 trim_cache size 0 max 16384 2023-11-28T11:11:02.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.725+0000 7fde7bc38b80 1 Processor -- start 2023-11-28T11:11:02.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.725+0000 7fde7bc38b80 1 -- start start 2023-11-28T11:11:02.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde7bc38b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b0bc0 0x55739f0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde7bc38b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 0x55739f09f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde7bc38b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 0x55739f087790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde7bc38b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739f085490 con 0x55739f0b3810 2023-11-28T11:11:02.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde7bc38b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739f0d4f70 con 0x55739f0b1130 2023-11-28T11:11:02.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde7bc38b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55739f0d5590 con 0x55739f0b0bc0 2023-11-28T11:11:02.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde6e04c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 0x55739f087790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde6e04c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 0x55739f087790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35828/0 (socket says 172.21.15.156:35828) 2023-11-28T11:11:02.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde6e04c700 1 -- 172.21.15.156:0/389711611 learned_addr learned my addr 172.21.15.156:0/389711611 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:02.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde6e04c700 1 -- 172.21.15.156:0/389711611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b0bc0 msgr2=0x55739f0b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde6e04c700 1 --2- 172.21.15.156:0/389711611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b0bc0 0x55739f0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde6e04c700 1 -- 172.21.15.156:0/389711611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 msgr2=0x55739f09f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde6e04c700 1 --2- 172.21.15.156:0/389711611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 0x55739f09f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde6e04c700 1 -- 172.21.15.156:0/389711611 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55739efa2140 con 0x55739f0b3810 2023-11-28T11:11:02.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde6e04c700 1 --2- 172.21.15.156:0/389711611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 0x55739f087790 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fde60002230 tx=0x7fde60020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202a0476cc6ed3c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:02.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde4bfff700 1 -- 172.21.15.156:0/389711611 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde60029b50 con 0x55739f0b3810 2023-11-28T11:11:02.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde4bfff700 1 -- 172.21.15.156:0/389711611 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6000f030 con 0x55739f0b3810 2023-11-28T11:11:02.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.726+0000 7fde4bfff700 1 -- 172.21.15.156:0/389711611 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6001e760 con 0x55739f0b3810 2023-11-28T11:11:02.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.727+0000 7fde7bc38b80 1 -- 172.21.15.156:0/389711611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 msgr2=0x55739f087790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.727+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/389711611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 0x55739f087790 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fde60002230 tx=0x7fde60020070 comp rx=0 tx=0).stop 2023-11-28T11:11:02.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.727+0000 7fde7bc38b80 1 -- 172.21.15.156:0/389711611 shutdown_connections 2023-11-28T11:11:02.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.727+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/389711611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 0x55739f087790 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.727+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/389711611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b0bc0 0x55739f0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.727+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/389711611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 0x55739f09f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.727+0000 7fde7bc38b80 1 -- 172.21.15.156:0/389711611 >> 172.21.15.156:0/389711611 conn(0x55739f086d60 msgr2=0x55739f08a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:02.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.727+0000 7fde7bc38b80 1 -- 172.21.15.156:0/389711611 shutdown_connections 2023-11-28T11:11:02.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.727+0000 7fde7bc38b80 1 -- 172.21.15.156:0/389711611 wait complete. 2023-11-28T11:11:02.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.728+0000 7fde7bc38b80 1 -- start start 2023-11-28T11:11:02.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.728+0000 7fde7bc38b80 1 Processor -- start 2023-11-28T11:11:02.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde7bc38b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b0bc0 0x55739f132aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde7bc38b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 0x55739f0c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde7bc38b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 0x55739f0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde7bc38b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739f0a2e20 con 0x55739f0b3810 2023-11-28T11:11:02.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde7bc38b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739f0c4aa0 con 0x55739f0b1130 2023-11-28T11:11:02.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde7bc38b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55739f0a2330 con 0x55739f0b0bc0 2023-11-28T11:11:02.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde7bc38b80 10 client.0 _ll_register_callbacks cb 0x55739ee81240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:02.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde6e04c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 0x55739f0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde6e04c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 0x55739f0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35834/0 (socket says 172.21.15.156:35834) 2023-11-28T11:11:02.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde6e04c700 1 -- 172.21.15.156:0/4040683364 learned_addr learned my addr 172.21.15.156:0/4040683364 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:02.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde6e04c700 1 -- 172.21.15.156:0/4040683364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b0bc0 msgr2=0x55739f132aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde6d84b700 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b0bc0 0x55739f132aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde6d04a700 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 0x55739f0c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde6e04c700 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b0bc0 0x55739f132aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde6e04c700 1 -- 172.21.15.156:0/4040683364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 msgr2=0x55739f0c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde6e04c700 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 0x55739f0c3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde6e04c700 1 -- 172.21.15.156:0/4040683364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55739ee09d90 con 0x55739f0b3810 2023-11-28T11:11:02.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde6e04c700 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 0x55739f0b7400 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fde60036040 tx=0x7fde6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a7c977e09a107b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:02.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:11:02.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde48ff9700 1 -- 172.21.15.156:0/4040683364 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6001e3f0 con 0x55739f0b3810 2023-11-28T11:11:02.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde48ff9700 1 -- 172.21.15.156:0/4040683364 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fde60024070 con 0x55739f0b3810 2023-11-28T11:11:02.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde7bc38b80 1 -- 172.21.15.156:0/4040683364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55739ef99160 con 0x55739f0b3810 2023-11-28T11:11:02.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde48ff9700 1 -- 172.21.15.156:0/4040683364 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde6001e550 con 0x55739f0b3810 2023-11-28T11:11:02.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde7bc38b80 10 client.6177 fetch_fsmap learned FSMap version 31 2023-11-28T11:11:02.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.729+0000 7fde7bc38b80 1 -- 172.21.15.156:0/4040683364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55739ee30200 con 0x55739f0b3810 2023-11-28T11:11:02.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.730+0000 7fde48ff9700 1 -- 172.21.15.156:0/4040683364 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55739ef99160 con 0x55739f0b3810 2023-11-28T11:11:02.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.730+0000 7fde48ff9700 1 -- 172.21.15.156:0/4040683364 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fde6001e980 con 0x55739f0b3810 2023-11-28T11:11:02.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.730+0000 7fde7bc38b80 10 client.6177 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T11:11:02.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.730+0000 7fde7bc38b80 10 client.6177 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:11:02.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.730+0000 7fde7bc38b80 20 client.6177 populate_metadata read hostname 'smithi156' 2023-11-28T11:11:02.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.730+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55739f0b4120 0x55739f0c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.730+0000 7fde7bc38b80 4 client.6177 mds_command: new command op to 6021 tid=0[{"prefix": "status"}] 2023-11-28T11:11:02.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.730+0000 7fde7bc38b80 1 -- 172.21.15.156:0/4040683364 --> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55739ef83170 con 0x55739f0b4120 2023-11-28T11:11:02.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.730+0000 7fde6d84b700 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55739f0b4120 0x55739f0c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.730+0000 7fde6d84b700 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55739f0b4120 0x55739f0c80f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1962fc3affded server_cookie=e43c5ea4dde8592 in_seq=0 out_seq=0 2023-11-28T11:11:02.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.730+0000 7fde48ff9700 10 client.6177 ms_handle_connect on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:02.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde48ff9700 1 -- 172.21.15.156:0/4040683364 <== mds.0 v2:172.21.15.186:6833/3811791267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55739ef83170 con 0x55739f0b4120 2023-11-28T11:11:02.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde48ff9700 10 client.6177 handle_command_reply: tid=0 2023-11-28T11:11:02.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde7bc38b80 1 client.6177 shutdown 2023-11-28T11:11:02.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde7bc38b80 1 -- 172.21.15.156:0/4040683364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 msgr2=0x55739f0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 0x55739f0b7400 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fde60036040 tx=0x7fde6000b040 comp rx=0 tx=0).stop 2023-11-28T11:11:02.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde7bc38b80 1 -- 172.21.15.156:0/4040683364 shutdown_connections 2023-11-28T11:11:02.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55739f0b4120 0x55739f0c80f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55739f0b3810 0x55739f0b7400 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55739f0b0bc0 0x55739f132aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde48ff9700 0 client.6177 ms_handle_reset on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:02.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde7bc38b80 1 --2- 172.21.15.156:0/4040683364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55739f0b1130 0x55739f0c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde7bc38b80 1 -- 172.21.15.156:0/4040683364 >> 172.21.15.156:0/4040683364 conn(0x55739f086d60 msgr2=0x55739f0aa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:02.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde7bc38b80 1 -- 172.21.15.156:0/4040683364 shutdown_connections 2023-11-28T11:11:02.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde7bc38b80 1 -- 172.21.15.156:0/4040683364 wait complete. 2023-11-28T11:11:02.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde7bc38b80 20 client.6177 trim_cache size 0 max 16384 2023-11-28T11:11:02.742 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:11:02.743 INFO:teuthology.orchestra.run.smithi156.stdout: "cluster_fsid": "351da896-8dbf-473b-9472-c3454d9368bf", 2023-11-28T11:11:02.743 INFO:teuthology.orchestra.run.smithi156.stdout: "whoami": 0, 2023-11-28T11:11:02.743 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 6021, 2023-11-28T11:11:02.743 INFO:teuthology.orchestra.run.smithi156.stdout: "want_state": "up:active", 2023-11-28T11:11:02.743 INFO:teuthology.orchestra.run.smithi156.stdout: "state": "up:active", 2023-11-28T11:11:02.743 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs", 2023-11-28T11:11:02.743 INFO:teuthology.orchestra.run.smithi156.stdout: "rank_uptime": 10.36217145, 2023-11-28T11:11:02.744 INFO:teuthology.orchestra.run.smithi156.stdout: "mdsmap_epoch": 31, 2023-11-28T11:11:02.744 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch": 64, 2023-11-28T11:11:02.744 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch_barrier": 63, 2023-11-28T11:11:02.744 INFO:teuthology.orchestra.run.smithi156.stdout: "uptime": 15.791010480000001 2023-11-28T11:11:02.744 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:11:02.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde6faaf700 1 -- 172.21.15.156:0/896581676 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fde4c0682f0 msgr2=0x7fde4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde6faaf700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fde4c0682f0 0x7fde4c06a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fde6806d9b0 tx=0x7fde58008040 comp rx=0 tx=0).stop 2023-11-28T11:11:02.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde6faaf700 1 -- 172.21.15.156:0/896581676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde6806cdc0 msgr2=0x7fde68154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde6faaf700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde6806cdc0 0x7fde68154420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fde64014040 tx=0x7fde64004750 comp rx=0 tx=0).stop 2023-11-28T11:11:02.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde6faaf700 1 -- 172.21.15.156:0/896581676 shutdown_connections 2023-11-28T11:11:02.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde6faaf700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fde4c0682f0 0x7fde4c06a7a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde6faaf700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde68071040 0x7fde680b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde6faaf700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde6806cdc0 0x7fde68154420 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde6faaf700 1 --2- 172.21.15.156:0/896581676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde6806dee0 0x7fde68152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.731+0000 7fde6faaf700 1 -- 172.21.15.156:0/896581676 >> 172.21.15.156:0/896581676 conn(0x7fde680d0000 msgr2=0x7fde6815e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:02.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.732+0000 7fde6faaf700 1 -- 172.21.15.156:0/896581676 shutdown_connections 2023-11-28T11:11:02.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.732+0000 7fde6faaf700 1 -- 172.21.15.156:0/896581676 wait complete. 2023-11-28T11:11:02.748 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:11:02.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.872+0000 7f296bcdf700 1 Processor -- start 2023-11-28T11:11:02.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.872+0000 7f296bcdf700 1 -- start start 2023-11-28T11:11:02.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f296bcdf700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29640cbdf0 0x7f29640af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f296bcdf700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29640cc5f0 con 0x7f296406ba90 2023-11-28T11:11:02.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f296bcdf700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296415d470 con 0x7f296415cef0 2023-11-28T11:11:02.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f296bcdf700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296415d750 con 0x7f29640cbdf0 2023-11-28T11:11:02.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f296927a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f296406ba90 0x7f29640ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50716/0 (socket says 172.21.15.156:50716) 2023-11-28T11:11:02.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f296927a700 1 -- 172.21.15.156:0/2016148811 learned_addr learned my addr 172.21.15.156:0/2016148811 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:02.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f296a27c700 1 --2- 172.21.15.156:0/2016148811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29640cbdf0 0x7f29640af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841180073 0 0) 0x7f29640cc5f0 con 0x7f296406ba90 2023-11-28T11:11:02.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29500035b0 con 0x7f296406ba90 2023-11-28T11:11:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1115548270 0 0) 0x7f29500035b0 con 0x7f296406ba90 2023-11-28T11:11:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29640cbdf0 msgr2=0x7f29640af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f2968a79700 1 --2- 172.21.15.156:0/2016148811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29640cbdf0 0x7f29640af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 >> v1:172.21.15.186:6789/0 conn(0x7f296415cef0 legacy=0x7f296406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296415da30 con 0x7f296406ba90 2023-11-28T11:11:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f2954001670 con 0x7f296406ba90 2023-11-28T11:11:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.873+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 >> v1:172.21.15.156:6789/0 conn(0x7f296406ba90 legacy=0x7f29640ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2968a79700 1 --2- 172.21.15.156:0/2016148811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2950003a30 0x7f2950003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2968a79700 1 --2- 172.21.15.156:0/2016148811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29500043d0 0x7f2950004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2968a79700 1 --2- 172.21.15.156:0/2016148811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29640cbdf0 0x7f29500050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296415d750 con 0x7f2950003a30 2023-11-28T11:11:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295000dc20 con 0x7f29500043d0 2023-11-28T11:11:02.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295000ddb0 con 0x7f29640cbdf0 2023-11-28T11:11:02.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2969a7b700 1 --2- 172.21.15.156:0/2016148811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2950003a30 0x7f2950003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f296927a700 1 --2- 172.21.15.156:0/2016148811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29500043d0 0x7f2950004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f296a27c700 1 --2- 172.21.15.156:0/2016148811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29640cbdf0 0x7f29500050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2969a7b700 1 -- 172.21.15.156:0/2016148811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29640cbdf0 msgr2=0x7f29500050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2969a7b700 1 --2- 172.21.15.156:0/2016148811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29640cbdf0 0x7f29500050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2969a7b700 1 -- 172.21.15.156:0/2016148811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29500043d0 msgr2=0x7f2950004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:02.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2969a7b700 1 --2- 172.21.15.156:0/2016148811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29500043d0 0x7f2950004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2969a7b700 1 -- 172.21.15.156:0/2016148811 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29500035b0 con 0x7f2950003a30 2023-11-28T11:11:02.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2969a7b700 1 --2- 172.21.15.156:0/2016148811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2950003a30 0x7f2950003e00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2960001980 tx=0x7f2950005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5737bc273b65f9fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:02.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960016070 con 0x7f2950003a30 2023-11-28T11:11:02.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2960006860 con 0x7f2950003a30 2023-11-28T11:11:02.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.874+0000 7f2968a79700 1 -- 172.21.15.156:0/2016148811 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960001420 con 0x7f2950003a30 2023-11-28T11:11:02.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.875+0000 7f296bcdf700 1 -- 172.21.15.156:0/2016148811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2950003a30 msgr2=0x7f2950003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.875+0000 7f296bcdf700 1 --2- 172.21.15.156:0/2016148811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2950003a30 0x7f2950003e00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2960001980 tx=0x7f2950005880 comp rx=0 tx=0).stop 2023-11-28T11:11:02.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.875+0000 7f296a27c700 1 -- 172.21.15.156:0/2016148811 reap_dead start 2023-11-28T11:11:02.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.875+0000 7f296bcdf700 1 -- 172.21.15.156:0/2016148811 shutdown_connections 2023-11-28T11:11:02.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.875+0000 7f296bcdf700 1 -- 172.21.15.156:0/2016148811 >> 172.21.15.156:0/2016148811 conn(0x7f29640d0000 msgr2=0x7f29640d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:02.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.875+0000 7f296bcdf700 1 -- 172.21.15.156:0/2016148811 shutdown_connections 2023-11-28T11:11:02.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.875+0000 7f296bcdf700 1 -- 172.21.15.156:0/2016148811 wait complete. 2023-11-28T11:11:02.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.876+0000 7f296bcdf700 1 Processor -- start 2023-11-28T11:11:02.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.876+0000 7f296bcdf700 1 -- start start 2023-11-28T11:11:02.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296bcdf700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2950003a30 0x7f29641543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296bcdf700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29640b0b70 0x7f29640b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296bcdf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29640c58d0 0x7f29640c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296bcdf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29640b1520 con 0x7f29640c58d0 2023-11-28T11:11:02.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296bcdf700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29640b2e30 con 0x7f29640b0b70 2023-11-28T11:11:02.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296bcdf700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29640b3080 con 0x7f2950003a30 2023-11-28T11:11:02.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296a27c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29640c58d0 0x7f29640c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296a27c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29640c58d0 0x7f29640c5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35854/0 (socket says 172.21.15.156:35854) 2023-11-28T11:11:02.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296a27c700 1 -- 172.21.15.156:0/1421574442 learned_addr learned my addr 172.21.15.156:0/1421574442 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:02.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f2969a7b700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2950003a30 0x7f29641543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296927a700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29640b0b70 0x7f29640b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296a27c700 1 -- 172.21.15.156:0/1421574442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2950003a30 msgr2=0x7f29641543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296a27c700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2950003a30 0x7f29641543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296a27c700 1 -- 172.21.15.156:0/1421574442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29640b0b70 msgr2=0x7f29640b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:02.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296a27c700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29640b0b70 0x7f29640b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:02.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296a27c700 1 -- 172.21.15.156:0/1421574442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296415dff0 con 0x7f29640c58d0 2023-11-28T11:11:02.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296927a700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29640b0b70 0x7f29640b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:02.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296a27c700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29640c58d0 0x7f29640c5d70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f296406c940 tx=0x7f295c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:02.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f295a7fc700 1 -- 172.21.15.156:0/1421574442 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c00e040 con 0x7f29640c58d0 2023-11-28T11:11:02.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f295a7fc700 1 -- 172.21.15.156:0/1421574442 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c00f040 con 0x7f29640c58d0 2023-11-28T11:11:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f296bcdf700 1 -- 172.21.15.156:0/1421574442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296415dd10 con 0x7f29640c58d0 2023-11-28T11:11:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.877+0000 7f295a7fc700 1 -- 172.21.15.156:0/1421574442 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c015430 con 0x7f29640c58d0 2023-11-28T11:11:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.878+0000 7f296bcdf700 1 -- 172.21.15.156:0/1421574442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29640af400 con 0x7f29640c58d0 2023-11-28T11:11:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.879+0000 7f2947fff700 1 -- 172.21.15.156:0/1421574442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29400052f0 con 0x7f29640c58d0 2023-11-28T11:11:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.879+0000 7f295a7fc700 1 -- 172.21.15.156:0/1421574442 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f295c015590 con 0x7f29640c58d0 2023-11-28T11:11:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.879+0000 7f295a7fc700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f29480682f0 0x7f294806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.880+0000 7f295a7fc700 1 -- 172.21.15.156:0/1421574442 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f295c086340 con 0x7f29640c58d0 2023-11-28T11:11:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.880+0000 7f2969a7b700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f29480682f0 0x7f294806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:02.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.880+0000 7f2969a7b700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f29480682f0 0x7f294806a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2960000fb0 tx=0x7f2960009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:02.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:02.884+0000 7f295a7fc700 1 -- 172.21.15.156:0/1421574442 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f295c052a30 con 0x7f29640c58d0 2023-11-28T11:11:03.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.114+0000 7f2947fff700 1 -- 172.21.15.156:0/1421574442 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2940005db0 con 0x7f29640c58d0 2023-11-28T11:11:03.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.114+0000 7f295a7fc700 1 -- 172.21.15.156:0/1421574442 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7f295c056050 con 0x7f29640c58d0 2023-11-28T11:11:03.115 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:11:03.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.116+0000 7f2947fff700 1 -- 172.21.15.156:0/1421574442 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f29480682f0 msgr2=0x7f294806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.116+0000 7f2947fff700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f29480682f0 0x7f294806a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2960000fb0 tx=0x7f2960009030 comp rx=0 tx=0).stop 2023-11-28T11:11:03.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.116+0000 7f2947fff700 1 -- 172.21.15.156:0/1421574442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29640c58d0 msgr2=0x7f29640c5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.116+0000 7f2947fff700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29640c58d0 0x7f29640c5d70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f296406c940 tx=0x7f295c011f60 comp rx=0 tx=0).stop 2023-11-28T11:11:03.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.117+0000 7f2947fff700 1 -- 172.21.15.156:0/1421574442 shutdown_connections 2023-11-28T11:11:03.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.117+0000 7f2947fff700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29640c58d0 0x7f29640c5d70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.117+0000 7f2947fff700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f29480682f0 0x7f294806a7a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.117+0000 7f2947fff700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2950003a30 0x7f29641543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.117+0000 7f2947fff700 1 --2- 172.21.15.156:0/1421574442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29640b0b70 0x7f29640b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.117+0000 7f2947fff700 1 -- 172.21.15.156:0/1421574442 >> 172.21.15.156:0/1421574442 conn(0x7f29640d0000 msgr2=0x7f296415c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:03.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.117+0000 7f2947fff700 1 -- 172.21.15.156:0/1421574442 shutdown_connections 2023-11-28T11:11:03.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:03.117+0000 7f2947fff700 1 -- 172.21.15.156:0/1421574442 wait complete. 2023-11-28T11:11:03.130 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:11:03.130 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:11:03.130 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-28T11:11:03.130 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:11:03.130 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:03.130 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:11:03.130 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:11:03.130 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:11:03.130 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:11:03.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:03.131 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-28T11:11:03.148 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:11:03.148 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:11:03.148 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:11:03.148 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:03.148 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T11:11:03.148 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:03.148 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:11:03.148 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:03.148 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:11:03.148 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:03.149 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:11:03.149 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:03.149 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:11:03.149 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:03.149 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:11:03.149 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42219sec preferred_lft 42219sec 2023-11-28T11:11:03.149 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::c260:d776:a692:b0cc/64 scope link noprefixroute 2023-11-28T11:11:03.149 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:03.149 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:11:03.149 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 8e:07:21:2f:e8:e5 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:03.149 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:11:03.149 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:03.150 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::a871:90ff:feb9:397d/64 scope link 2023-11-28T11:11:03.150 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:03.150 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:11:03.150 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 8e:07:21:2f:e8:e5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:11:03.150 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::8c07:21ff:fe2f:e8e5/64 scope link 2023-11-28T11:11:03.150 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:03.150 INFO:teuthology.orchestra.run.smithi156.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:11:03.150 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether d2:39:d6:e9:98:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:11:03.150 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::d039:d6ff:fee9:985b/64 scope link 2023-11-28T11:11:03.150 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:03.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:03.151 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:11:03.213 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-28T11:11:03.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:03.215 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-28T11:11:03.273 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-28T11:11:03.274 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:11:03.274 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:11:03.274 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:11:03.274 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:11:03.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:03.274 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T11:11:03.275 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T11:11:03.275 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:11:03.275 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:11:03.275 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:11:03.275 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T11:11:03.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:03.379 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T11:11:03.395 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:11:03.396 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:11:03.396 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:11:03.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:03.396 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-28T11:11:03.462 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:11:03.462 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:03.519 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:11:03.520 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:11:03.520 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:03.576 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:11:03.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:03.577 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:11:03.663 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:11:03.663 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:11:03.663 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:11:03.672 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:11:03.672 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:11:03.672 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:11:03.672 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:11:03.672 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:11:03.697 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:11:03.697 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:11:03.697 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T11:11:03.698 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T11:11:03.698 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:11:03.698 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T11:11:03.698 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T11:11:03.698 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=351da896-8dbf-473b-9472-c3454d9368bf" will pass to kernel 2023-11-28T11:11:03.701 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:11:03.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:03.702 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:03.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:03.732 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:11:03.819 INFO:teuthology.orchestra.run.smithi156.stdout:client6186 2023-11-28T11:11:03.819 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:11:03.819 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:03.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:03.836 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2023-11-28T11:11:03.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:03.899 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2023-11-28T11:11:03.957 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:11:03.958 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T11:11:03.958 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T11:11:03.958 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T11:11:03.958 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2023-11-28T11:11:03.958 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T11:11:04.064 INFO:teuthology.orchestra.run.smithi156.stdout:1099511627777 2023-11-28T11:11:04.253 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:04.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.413+0000 7f89853f7700 1 Processor -- start 2023-11-28T11:11:04.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.413+0000 7f89853f7700 1 -- start start 2023-11-28T11:11:04.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.414+0000 7f89853f7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8980071750 0x7f898015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.414+0000 7f89853f7700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8980155240 con 0x7f8980154e70 2023-11-28T11:11:04.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.414+0000 7f89853f7700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898015b100 con 0x7f89800cdbd0 2023-11-28T11:11:04.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.414+0000 7f89853f7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89800ce050 con 0x7f8980071750 2023-11-28T11:11:04.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.414+0000 7f897effd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f8980154e70 0x7f89800cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50740/0 (socket says 172.21.15.156:50740) 2023-11-28T11:11:04.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.414+0000 7f897effd700 1 -- 172.21.15.156:0/581709859 learned_addr learned my addr 172.21.15.156:0/581709859 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:04.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.414+0000 7f897f7fe700 1 --2- 172.21.15.156:0/581709859 >> v2:172.21.15.156:3300/0 conn(0x7f8980071750 0x7f898015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.414+0000 7f897dffb700 1 -- 172.21.15.156:0/581709859 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3872232424 0 0) 0x7f8980155240 con 0x7f8980154e70 2023-11-28T11:11:04.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.414+0000 7f897dffb700 1 -- 172.21.15.156:0/581709859 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89680035b0 con 0x7f8980154e70 2023-11-28T11:11:04.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.414+0000 7f897f7fe700 1 -- 172.21.15.156:0/581709859 >> v1:172.21.15.186:6789/0 conn(0x7f89800cdbd0 legacy=0x7f8980071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.414+0000 7f897f7fe700 1 -- 172.21.15.156:0/581709859 >> v1:172.21.15.156:6789/0 conn(0x7f8980154e70 legacy=0x7f89800cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.414+0000 7f897f7fe700 1 -- 172.21.15.156:0/581709859 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898015b3e0 con 0x7f8980071750 2023-11-28T11:11:04.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.415+0000 7f897f7fe700 1 --2- 172.21.15.156:0/581709859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980071750 0x7f898015e680 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8978009d90 tx=0x7f897800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00b8bea0bf29588 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.415+0000 7f897dffb700 1 -- 172.21.15.156:0/581709859 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897800e070 con 0x7f8980071750 2023-11-28T11:11:04.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.415+0000 7f897dffb700 1 -- 172.21.15.156:0/581709859 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897800bb20 con 0x7f8980071750 2023-11-28T11:11:04.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.415+0000 7f897dffb700 1 -- 172.21.15.156:0/581709859 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978012430 con 0x7f8980071750 2023-11-28T11:11:04.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.415+0000 7f89853f7700 1 -- 172.21.15.156:0/581709859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980071750 msgr2=0x7f898015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.415+0000 7f89853f7700 1 --2- 172.21.15.156:0/581709859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980071750 0x7f898015e680 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8978009d90 tx=0x7f897800b330 comp rx=0 tx=0).stop 2023-11-28T11:11:04.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.415+0000 7f89853f7700 1 -- 172.21.15.156:0/581709859 shutdown_connections 2023-11-28T11:11:04.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.415+0000 7f89853f7700 1 --2- 172.21.15.156:0/581709859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980071750 0x7f898015e680 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.415+0000 7f89853f7700 1 -- 172.21.15.156:0/581709859 >> 172.21.15.156:0/581709859 conn(0x7f89800d0000 msgr2=0x7f89800d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:04.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.415+0000 7f89853f7700 1 -- 172.21.15.156:0/581709859 shutdown_connections 2023-11-28T11:11:04.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.415+0000 7f89853f7700 1 -- 172.21.15.156:0/581709859 wait complete. 2023-11-28T11:11:04.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.417+0000 7f89853f7700 1 Processor -- start 2023-11-28T11:11:04.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.417+0000 7f89853f7700 1 -- start start 2023-11-28T11:11:04.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.417+0000 7f89853f7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980071750 0x7f8980153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.417+0000 7f89853f7700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89800cdbd0 0x7f89801537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.417+0000 7f89853f7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8980154e70 0x7f89800c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.417+0000 7f89853f7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800c8380 con 0x7f8980071750 2023-11-28T11:11:04.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.417+0000 7f89853f7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800c7aa0 con 0x7f8980154e70 2023-11-28T11:11:04.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.417+0000 7f89853f7700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898015a050 con 0x7f89800cdbd0 2023-11-28T11:11:04.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f897effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980071750 0x7f8980153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f897effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980071750 0x7f8980153e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35868/0 (socket says 172.21.15.156:35868) 2023-11-28T11:11:04.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f897effd700 1 -- 172.21.15.156:0/1311797188 learned_addr learned my addr 172.21.15.156:0/1311797188 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:04.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f897effd700 1 -- 172.21.15.156:0/1311797188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89800cdbd0 msgr2=0x7f89801537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f897effd700 1 --2- 172.21.15.156:0/1311797188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89800cdbd0 0x7f89801537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f897effd700 1 -- 172.21.15.156:0/1311797188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8980154e70 msgr2=0x7f89800c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f897effd700 1 --2- 172.21.15.156:0/1311797188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8980154e70 0x7f89800c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f897effd700 1 -- 172.21.15.156:0/1311797188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8978009a70 con 0x7f8980071750 2023-11-28T11:11:04.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f897effd700 1 --2- 172.21.15.156:0/1311797188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980071750 0x7f8980153e80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8970001eb0 tx=0x7f897000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f89677fe700 1 -- 172.21.15.156:0/1311797188 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8970005d70 con 0x7f8980071750 2023-11-28T11:11:04.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f89677fe700 1 -- 172.21.15.156:0/1311797188 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89700070d0 con 0x7f8980071750 2023-11-28T11:11:04.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f89853f7700 1 -- 172.21.15.156:0/1311797188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898015b9a0 con 0x7f8980071750 2023-11-28T11:11:04.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f89677fe700 1 -- 172.21.15.156:0/1311797188 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8970007460 con 0x7f8980071750 2023-11-28T11:11:04.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.418+0000 7f89853f7700 1 -- 172.21.15.156:0/1311797188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898015b6c0 con 0x7f8980071750 2023-11-28T11:11:04.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.419+0000 7f89853f7700 1 -- 172.21.15.156:0/1311797188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89800af520 con 0x7f8980071750 2023-11-28T11:11:04.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.420+0000 7f89677fe700 1 -- 172.21.15.156:0/1311797188 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8970019030 con 0x7f8980071750 2023-11-28T11:11:04.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.420+0000 7f89677fe700 1 --2- 172.21.15.156:0/1311797188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f896c0682f0 0x7f896c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.420+0000 7f89677fe700 1 -- 172.21.15.156:0/1311797188 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f8970085110 con 0x7f8980071750 2023-11-28T11:11:04.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.420+0000 7f897e7fc700 1 --2- 172.21.15.156:0/1311797188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f896c0682f0 0x7f896c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.420+0000 7f897e7fc700 1 --2- 172.21.15.156:0/1311797188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f896c0682f0 0x7f896c06a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8974000f60 tx=0x7f8974007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.424+0000 7f89677fe700 1 -- 172.21.15.156:0/1311797188 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f89700518f0 con 0x7f8980071750 2023-11-28T11:11:04.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.648+0000 7f89853f7700 1 -- 172.21.15.156:0/1311797188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f898006c010 con 0x7f8980071750 2023-11-28T11:11:04.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.649+0000 7f89677fe700 1 -- 172.21.15.156:0/1311797188 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f8970054f10 con 0x7f8980071750 2023-11-28T11:11:04.650 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:04.650 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6006,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1991065152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1991065152},{"type":"v1","addr":"172.21.15.156:6836","nonce":1991065152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/43543030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":43543030},{"type":"v1","addr":"172.21.15.156:6837","nonce":43543030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/2480189069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2480189069},{"type":"v1","addr":"172.21.15.186:6834","nonce":2480189069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:52.367543+0000","modified":"2023-11-28T11:10:55.380411+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6021":{"gid":6021,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6835/3811791267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3811791267},{"type":"v1","addr":"172.21.15.186:6835","nonce":3811791267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:11:04.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.651+0000 7f89853f7700 1 -- 172.21.15.156:0/1311797188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f896c0682f0 msgr2=0x7f896c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.651+0000 7f89853f7700 1 --2- 172.21.15.156:0/1311797188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f896c0682f0 0x7f896c06a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8974000f60 tx=0x7f8974007a40 comp rx=0 tx=0).stop 2023-11-28T11:11:04.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.651+0000 7f89853f7700 1 -- 172.21.15.156:0/1311797188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980071750 msgr2=0x7f8980153e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.651+0000 7f89853f7700 1 --2- 172.21.15.156:0/1311797188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980071750 0x7f8980153e80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8970001eb0 tx=0x7f897000c860 comp rx=0 tx=0).stop 2023-11-28T11:11:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.651+0000 7f89853f7700 1 -- 172.21.15.156:0/1311797188 shutdown_connections 2023-11-28T11:11:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.651+0000 7f89853f7700 1 --2- 172.21.15.156:0/1311797188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8980154e70 0x7f89800c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.651+0000 7f89853f7700 1 --2- 172.21.15.156:0/1311797188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8980071750 0x7f8980153e80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.651+0000 7f89853f7700 1 --2- 172.21.15.156:0/1311797188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f896c0682f0 0x7f896c06a7a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.651+0000 7f89853f7700 1 --2- 172.21.15.156:0/1311797188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89800cdbd0 0x7f89801537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.651+0000 7f89853f7700 1 -- 172.21.15.156:0/1311797188 >> 172.21.15.156:0/1311797188 conn(0x7f89800d0000 msgr2=0x7f89800aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.651+0000 7f89853f7700 1 -- 172.21.15.156:0/1311797188 shutdown_connections 2023-11-28T11:11:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.651+0000 7f89853f7700 1 -- 172.21.15.156:0/1311797188 wait complete. 2023-11-28T11:11:04.654 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 31 2023-11-28T11:11:04.664 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:11:04.822 INFO:teuthology.orchestra.run.smithi186.stdout:{"message":"","return_code":0} 2023-11-28T11:11:04.823 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:11:04.823 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:11:04.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.866+0000 7fdd9cbe9080 1 Processor -- start 2023-11-28T11:11:04.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.866+0000 7fdd9cbe9080 1 -- start start 2023-11-28T11:11:04.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd9cbe9080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d7725f0 0x556f1d7b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd9cbe9080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556f1d7b7f40 0x556f1d7b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd9cbe9080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f1d846300 0x556f1d7a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd9cbe9080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1d719000 con 0x556f1d7725f0 2023-11-28T11:11:04.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd9cbe9080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1d6abfe0 con 0x556f1d846300 2023-11-28T11:11:04.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd9cbe9080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f1d7b0a60 con 0x556f1d7b7f40 2023-11-28T11:11:04.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd92cec700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d7725f0 0x556f1d7b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd92cec700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d7725f0 0x556f1d7b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35882/0 (socket says 172.21.15.156:35882) 2023-11-28T11:11:04.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd92cec700 1 -- 172.21.15.156:0/3862381914 learned_addr learned my addr 172.21.15.156:0/3862381914 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:04.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd934ed700 1 --2- 172.21.15.156:0/3862381914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f1d846300 0x556f1d7a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd8bfff700 1 --2- 172.21.15.156:0/3862381914 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556f1d7b7f40 0x556f1d7b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd92cec700 1 -- 172.21.15.156:0/3862381914 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556f1d7b7f40 msgr2=0x556f1d7b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:04.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd92cec700 1 --2- 172.21.15.156:0/3862381914 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556f1d7b7f40 0x556f1d7b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd92cec700 1 -- 172.21.15.156:0/3862381914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f1d846300 msgr2=0x556f1d7a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:04.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd92cec700 1 --2- 172.21.15.156:0/3862381914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f1d846300 0x556f1d7a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd92cec700 1 -- 172.21.15.156:0/3862381914 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c00a100 con 0x556f1d7725f0 2023-11-28T11:11:04.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd92cec700 1 --2- 172.21.15.156:0/3862381914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d7725f0 0x556f1d7b1f80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdd84001ed0 tx=0x7fdd8400b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18858d4fedeb23ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd91cea700 1 -- 172.21.15.156:0/3862381914 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84014070 con 0x556f1d7725f0 2023-11-28T11:11:04.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd91cea700 1 -- 172.21.15.156:0/3862381914 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8400be80 con 0x556f1d7725f0 2023-11-28T11:11:04.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.867+0000 7fdd91cea700 1 -- 172.21.15.156:0/3862381914 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400f610 con 0x556f1d7725f0 2023-11-28T11:11:04.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.868+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3862381914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d7725f0 msgr2=0x556f1d7b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.868+0000 7fdd9cbe9080 1 --2- 172.21.15.156:0/3862381914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d7725f0 0x556f1d7b1f80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdd84001ed0 tx=0x7fdd8400b570 comp rx=0 tx=0).stop 2023-11-28T11:11:04.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.868+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3862381914 shutdown_connections 2023-11-28T11:11:04.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.868+0000 7fdd9cbe9080 1 --2- 172.21.15.156:0/3862381914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f1d846300 0x556f1d7a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.868+0000 7fdd9cbe9080 1 --2- 172.21.15.156:0/3862381914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d7725f0 0x556f1d7b1f80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.868+0000 7fdd9cbe9080 1 --2- 172.21.15.156:0/3862381914 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556f1d7b7f40 0x556f1d7b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.868+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3862381914 >> 172.21.15.156:0/3862381914 conn(0x556f1d79db60 msgr2=0x556f1d7b66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:04.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.868+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3862381914 shutdown_connections 2023-11-28T11:11:04.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.868+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3862381914 wait complete. 2023-11-28T11:11:04.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.868+0000 7fdd9cbe9080 1 Processor -- start 2023-11-28T11:11:04.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.869+0000 7fdd9cbe9080 1 -- start start 2023-11-28T11:11:04.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.869+0000 7fdd9cbe9080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556f1d7725f0 0x556f1d841f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.869+0000 7fdd9cbe9080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f1d7b7f40 0x556f1d7b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.869+0000 7fdd9cbe9080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d846300 0x556f1d7b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.869+0000 7fdd9cbe9080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1d79f180 con 0x556f1d846300 2023-11-28T11:11:04.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.869+0000 7fdd9cbe9080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1d842440 con 0x556f1d7b7f40 2023-11-28T11:11:04.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.869+0000 7fdd9cbe9080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f1d842600 con 0x556f1d7725f0 2023-11-28T11:11:04.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.869+0000 7fdd934ed700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d846300 0x556f1d7b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.869+0000 7fdd8bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f1d7b7f40 0x556f1d7b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.869+0000 7fdd934ed700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d846300 0x556f1d7b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35892/0 (socket says 172.21.15.156:35892) 2023-11-28T11:11:04.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.869+0000 7fdd934ed700 1 -- 172.21.15.156:0/3056161074 learned_addr learned my addr 172.21.15.156:0/3056161074 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:04.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.869+0000 7fdd92cec700 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556f1d7725f0 0x556f1d841f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.870+0000 7fdd934ed700 1 -- 172.21.15.156:0/3056161074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556f1d7725f0 msgr2=0x556f1d841f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.870+0000 7fdd934ed700 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556f1d7725f0 0x556f1d841f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.870+0000 7fdd934ed700 1 -- 172.21.15.156:0/3056161074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f1d7b7f40 msgr2=0x556f1d7b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.870+0000 7fdd934ed700 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556f1d7b7f40 0x556f1d7b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.870+0000 7fdd934ed700 1 -- 172.21.15.156:0/3056161074 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84001bb0 con 0x556f1d846300 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.870+0000 7fdd934ed700 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d846300 0x556f1d7b2410 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c013040 tx=0x7fdd8c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.870+0000 7fdd90ce8700 1 -- 172.21.15.156:0/3056161074 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c005870 con 0x556f1d846300 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.870+0000 7fdd90ce8700 1 -- 172.21.15.156:0/3056161074 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8c009030 con 0x556f1d846300 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.870+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3056161074 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556f1d6b7830 con 0x556f1d846300 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.870+0000 7fdd90ce8700 1 -- 172.21.15.156:0/3056161074 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c0256b0 con 0x556f1d846300 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.870+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3056161074 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f1d6c4360 con 0x556f1d846300 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.871+0000 7fdd90ce8700 1 -- 172.21.15.156:0/3056161074 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdd8c00e050 con 0x556f1d846300 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.872+0000 7fdd90ce8700 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd74066c10 0x7fdd740690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.872+0000 7fdd90ce8700 1 -- 172.21.15.156:0/3056161074 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fdd8c086160 con 0x556f1d846300 2023-11-28T11:11:04.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.872+0000 7fdd92cec700 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd74066c10 0x7fdd740690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.872+0000 7fdd9cbe9080 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x556f1d7b5eb0 0x556f1d8537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.872+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3056161074 --> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65) v8 -- 0x556f1d772ce0 con 0x556f1d7b5eb0 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.872+0000 7fdd8bfff700 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x556f1d7b5eb0 0x556f1d8537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.872+0000 7fdd92cec700 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd74066c10 0x7fdd740690c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdd84001ea0 tx=0x7fdd84011040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.872+0000 7fdd8bfff700 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x556f1d7b5eb0 0x556f1d8537a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.873+0000 7fdd8bfff700 1 -- 172.21.15.156:0/3056161074 <== osd.6 v2:172.21.15.186:6800/103779 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7fdd7c011770 con 0x556f1d7b5eb0 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.873+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3056161074 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x556f1d7b5eb0 msgr2=0x556f1d8537a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.873+0000 7fdd9cbe9080 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x556f1d7b5eb0 0x556f1d8537a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.873+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3056161074 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd74066c10 msgr2=0x7fdd740690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.873+0000 7fdd9cbe9080 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fdd74066c10 0x7fdd740690c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdd84001ea0 tx=0x7fdd84011040 comp rx=0 tx=0).stop 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.873+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3056161074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d846300 msgr2=0x556f1d7b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.873+0000 7fdd9cbe9080 1 --2- 172.21.15.156:0/3056161074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f1d846300 0x556f1d7b2410 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c013040 tx=0x7fdd8c003330 comp rx=0 tx=0).stop 2023-11-28T11:11:04.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.873+0000 7fdd934ed700 1 -- 172.21.15.156:0/3056161074 reap_dead start 2023-11-28T11:11:04.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.873+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3056161074 shutdown_connections 2023-11-28T11:11:04.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.873+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3056161074 >> 172.21.15.156:0/3056161074 conn(0x556f1d79db60 msgr2=0x556f1d7ba9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:04.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.873+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3056161074 shutdown_connections 2023-11-28T11:11:04.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:04.873+0000 7fdd9cbe9080 1 -- 172.21.15.156:0/3056161074 wait complete. 2023-11-28T11:11:04.882 INFO:teuthology.orchestra.run.smithi156.stdout:Xfiledir_test 2023-11-28T11:11:04.882 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:04.882 DEBUG:teuthology.orchestra.run.smithi156:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:11:04.938 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:11:04.938 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 1099511627777, 2023-11-28T11:11:04.938 INFO:teuthology.orchestra.run.smithi156.stdout: "ancestors": [ 2023-11-28T11:11:04.938 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:11:04.938 INFO:teuthology.orchestra.run.smithi156.stdout: "dirino": 1099511627776, 2023-11-28T11:11:04.938 INFO:teuthology.orchestra.run.smithi156.stdout: "dname": "file", 2023-11-28T11:11:04.938 INFO:teuthology.orchestra.run.smithi156.stdout: "version": 2 2023-11-28T11:11:04.938 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T11:11:04.938 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:11:04.938 INFO:teuthology.orchestra.run.smithi156.stdout: "dirino": 1, 2023-11-28T11:11:04.938 INFO:teuthology.orchestra.run.smithi156.stdout: "dname": "dir_test", 2023-11-28T11:11:04.939 INFO:teuthology.orchestra.run.smithi156.stdout: "version": 4 2023-11-28T11:11:04.939 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:11:04.939 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2023-11-28T11:11:04.939 INFO:teuthology.orchestra.run.smithi156.stdout: "pool": 10, 2023-11-28T11:11:04.939 INFO:teuthology.orchestra.run.smithi156.stdout: "old_pools": [] 2023-11-28T11:11:04.939 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:11:04.939 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:04.940 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2023-11-28T11:11:05.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.024+0000 7f5bb47c4080 1 Processor -- start 2023-11-28T11:11:05.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.024+0000 7f5bb47c4080 1 -- start start 2023-11-28T11:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.025+0000 7f5bb47c4080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b940030b0 0x559f905a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.025+0000 7f5bb47c4080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f90672310 0x559f90672780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.025+0000 7f5bb47c4080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559f905d2f50 0x559f905d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.025+0000 7f5bb47c4080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9054c000 con 0x7f5b940030b0 2023-11-28T11:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.025+0000 7f5bb47c4080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f904defe0 con 0x559f90672310 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.025+0000 7f5bb47c4080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f9067b130 con 0x559f905d2f50 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.025+0000 7f5baa8c7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b940030b0 0x559f905a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.025+0000 7f5baa8c7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b940030b0 0x559f905a6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35910/0 (socket says 172.21.15.156:35910) 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.025+0000 7f5baa8c7700 1 -- 172.21.15.156:0/1741727960 learned_addr learned my addr 172.21.15.156:0/1741727960 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.025+0000 7f5baa0c6700 1 --2- 172.21.15.156:0/1741727960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f90672310 0x559f90672780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.025+0000 7f5bab0c8700 1 --2- 172.21.15.156:0/1741727960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559f905d2f50 0x559f905d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5baa8c7700 1 -- 172.21.15.156:0/1741727960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559f905d2f50 msgr2=0x559f905d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5baa8c7700 1 --2- 172.21.15.156:0/1741727960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559f905d2f50 0x559f905d33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5baa8c7700 1 -- 172.21.15.156:0/1741727960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f90672310 msgr2=0x559f90672780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5baa8c7700 1 --2- 172.21.15.156:0/1741727960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f90672310 0x559f90672780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5baa8c7700 1 -- 172.21.15.156:0/1741727960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f904f7360 con 0x7f5b940030b0 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5bab0c8700 1 --2- 172.21.15.156:0/1741727960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559f905d2f50 0x559f905d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5baa0c6700 1 --2- 172.21.15.156:0/1741727960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f90672310 0x559f90672780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5baa8c7700 1 --2- 172.21.15.156:0/1741727960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b940030b0 0x559f905a6a80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009980 tx=0x7f5ba0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96ab76b94b8c923 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5ba90c4700 1 -- 172.21.15.156:0/1741727960 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba000f040 con 0x7f5b940030b0 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5ba90c4700 1 -- 172.21.15.156:0/1741727960 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba0008830 con 0x7f5b940030b0 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5ba90c4700 1 -- 172.21.15.156:0/1741727960 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0008be0 con 0x7f5b940030b0 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5bb47c4080 1 -- 172.21.15.156:0/1741727960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b940030b0 msgr2=0x559f905a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5bb47c4080 1 --2- 172.21.15.156:0/1741727960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b940030b0 0x559f905a6a80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009980 tx=0x7f5ba0005a00 comp rx=0 tx=0).stop 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5bb47c4080 1 -- 172.21.15.156:0/1741727960 shutdown_connections 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5bb47c4080 1 --2- 172.21.15.156:0/1741727960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559f905d2f50 0x559f905d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5bb47c4080 1 --2- 172.21.15.156:0/1741727960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b940030b0 0x559f905a6a80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5bb47c4080 1 --2- 172.21.15.156:0/1741727960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f90672310 0x559f90672780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5bb47c4080 1 -- 172.21.15.156:0/1741727960 >> 172.21.15.156:0/1741727960 conn(0x559f905d0b70 msgr2=0x559f9067b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5bb47c4080 1 -- 172.21.15.156:0/1741727960 shutdown_connections 2023-11-28T11:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.026+0000 7f5bb47c4080 1 -- 172.21.15.156:0/1741727960 wait complete. 2023-11-28T11:11:05.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.027+0000 7f5bb47c4080 1 Processor -- start 2023-11-28T11:11:05.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.027+0000 7f5bb47c4080 1 -- start start 2023-11-28T11:11:05.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5bb47c4080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f905d2f50 0x559f906697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5bb47c4080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f90672310 0x559f9066fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5bb47c4080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b940030b0 0x559f9066ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5bb47c4080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f90669d30 con 0x559f905d2f50 2023-11-28T11:11:05.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5bb47c4080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f905e68c0 con 0x559f90672310 2023-11-28T11:11:05.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5bb47c4080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f905e78c0 con 0x7f5b940030b0 2023-11-28T11:11:05.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5baa8c7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f905d2f50 0x559f906697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5baa8c7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f905d2f50 0x559f906697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35916/0 (socket says 172.21.15.156:35916) 2023-11-28T11:11:05.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5baa8c7700 1 -- 172.21.15.156:0/268611513 learned_addr learned my addr 172.21.15.156:0/268611513 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5bab0c8700 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b940030b0 0x559f9066ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5baa0c6700 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f90672310 0x559f9066fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5baa8c7700 1 -- 172.21.15.156:0/268611513 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b940030b0 msgr2=0x559f9066ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5baa8c7700 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b940030b0 0x559f9066ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5baa8c7700 1 -- 172.21.15.156:0/268611513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f90672310 msgr2=0x559f9066fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5baa8c7700 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559f90672310 0x559f9066fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5baa8c7700 1 -- 172.21.15.156:0/268611513 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba00095b0 con 0x559f905d2f50 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5baa8c7700 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f905d2f50 0x559f906697f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0000c00 tx=0x7f5ba0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5b93fff700 1 -- 172.21.15.156:0/268611513 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba000f030 con 0x559f905d2f50 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5b93fff700 1 -- 172.21.15.156:0/268611513 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba0015780 con 0x559f905d2f50 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5bb47c4080 1 -- 172.21.15.156:0/268611513 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559f905edeb0 con 0x559f905d2f50 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5b93fff700 1 -- 172.21.15.156:0/268611513 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0015b70 con 0x559f905d2f50 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.028+0000 7f5bb47c4080 1 -- 172.21.15.156:0/268611513 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f904ea830 con 0x559f905d2f50 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.030+0000 7f5b93fff700 1 -- 172.21.15.156:0/268611513 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5ba0015cd0 con 0x559f905d2f50 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.030+0000 7f5b93fff700 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5b98066c10 0x7f5b980690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.030+0000 7f5b93fff700 1 -- 172.21.15.156:0/268611513 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f5ba0086cc0 con 0x559f905d2f50 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.030+0000 7f5baa0c6700 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5b98066c10 0x7f5b980690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.030+0000 7f5bb47c4080 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x559f905d3700 0x559f907200f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.030+0000 7f5bb47c4080 1 -- 172.21.15.156:0/268611513 --> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65) v8 -- 0x559f905ea250 con 0x559f905d3700 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.030+0000 7f5baa0c6700 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5b98066c10 0x7f5b980690c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009fd0 tx=0x7f5b9c010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.031+0000 7f5bab0c8700 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x559f905d3700 0x559f907200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.031+0000 7f5bab0c8700 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x559f905d3700 0x559f907200f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.032+0000 7f5bab0c8700 1 -- 172.21.15.156:0/268611513 <== osd.6 v2:172.21.15.186:6800/103779 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v65'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5ba4006120 con 0x559f905d3700 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.032+0000 7f5bb47c4080 1 -- 172.21.15.156:0/268611513 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x559f905d3700 msgr2=0x559f907200f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.032+0000 7f5bb47c4080 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x559f905d3700 0x559f907200f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.032+0000 7f5bb47c4080 1 -- 172.21.15.156:0/268611513 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5b98066c10 msgr2=0x7f5b980690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.032+0000 7f5bb47c4080 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f5b98066c10 0x7f5b980690c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009fd0 tx=0x7f5b9c010000 comp rx=0 tx=0).stop 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.032+0000 7f5bb47c4080 1 -- 172.21.15.156:0/268611513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f905d2f50 msgr2=0x559f906697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.032+0000 7f5bb47c4080 1 --2- 172.21.15.156:0/268611513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f905d2f50 0x559f906697f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0000c00 tx=0x7f5ba0002fb0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.032+0000 7f5bab0c8700 1 -- 172.21.15.156:0/268611513 reap_dead start 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.032+0000 7f5bb47c4080 1 -- 172.21.15.156:0/268611513 shutdown_connections 2023-11-28T11:11:05.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.032+0000 7f5bb47c4080 1 -- 172.21.15.156:0/268611513 >> 172.21.15.156:0/268611513 conn(0x559f905d0b70 msgr2=0x559f9067b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.032+0000 7f5bb47c4080 1 -- 172.21.15.156:0/268611513 shutdown_connections 2023-11-28T11:11:05.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.032+0000 7f5bb47c4080 1 -- 172.21.15.156:0/268611513 wait complete. 2023-11-28T11:11:05.040 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2023-11-28T11:11:05.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.163+0000 7fcd3945f700 1 Processor -- start 2023-11-28T11:11:05.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.163+0000 7fcd3945f700 1 -- start start 2023-11-28T11:11:05.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd3945f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcd3415a760 0x7fcd34161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd3945f700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcd340cded0 0x7fcd340b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd3945f700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3415e0a0 con 0x7fcd3415dc30 2023-11-28T11:11:05.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd3945f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3415ab30 con 0x7fcd3415a760 2023-11-28T11:11:05.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd3945f700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd34160d70 con 0x7fcd340cded0 2023-11-28T11:11:05.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd32ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcd3415a760 0x7fcd34161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd327fc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fcd3415dc30 0x7fcd340cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50806/0 (socket says 172.21.15.156:50806) 2023-11-28T11:11:05.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd327fc700 1 -- 172.21.15.156:0/1757227145 learned_addr learned my addr 172.21.15.156:0/1757227145 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:05.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd337fe700 1 --2- 172.21.15.156:0/1757227145 >> v2:172.21.15.186:3300/0 conn(0x7fcd340cded0 0x7fcd340b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd32ffd700 1 -- 172.21.15.156:0/1757227145 >> v2:172.21.15.186:3300/0 conn(0x7fcd340cded0 msgr2=0x7fcd340b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd32ffd700 1 --2- 172.21.15.156:0/1757227145 >> v2:172.21.15.186:3300/0 conn(0x7fcd340cded0 0x7fcd340b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd32ffd700 1 -- 172.21.15.156:0/1757227145 >> v1:172.21.15.156:6789/0 conn(0x7fcd3415dc30 legacy=0x7fcd340cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd31ffb700 1 -- 172.21.15.156:0/1757227145 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796851175 0 0) 0x7fcd3415e0a0 con 0x7fcd3415dc30 2023-11-28T11:11:05.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd32ffd700 1 -- 172.21.15.156:0/1757227145 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd340b5130 con 0x7fcd3415a760 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd337fe700 1 --2- 172.21.15.156:0/1757227145 >> v2:172.21.15.186:3300/0 conn(0x7fcd340cded0 0x7fcd340b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.164+0000 7fcd32ffd700 1 --2- 172.21.15.156:0/1757227145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3415a760 0x7fcd34161200 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcd34067500 tx=0x7fcd28009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad1f1584f307aa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.165+0000 7fcd31ffb700 1 -- 172.21.15.156:0/1757227145 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28011830 con 0x7fcd3415a760 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.165+0000 7fcd31ffb700 1 -- 172.21.15.156:0/1757227145 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd28011990 con 0x7fcd3415a760 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.165+0000 7fcd31ffb700 1 -- 172.21.15.156:0/1757227145 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28011d20 con 0x7fcd3415a760 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.165+0000 7fcd3945f700 1 -- 172.21.15.156:0/1757227145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3415a760 msgr2=0x7fcd34161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.165+0000 7fcd3945f700 1 --2- 172.21.15.156:0/1757227145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3415a760 0x7fcd34161200 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcd34067500 tx=0x7fcd28009b20 comp rx=0 tx=0).stop 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.165+0000 7fcd3945f700 1 -- 172.21.15.156:0/1757227145 shutdown_connections 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.165+0000 7fcd3945f700 1 --2- 172.21.15.156:0/1757227145 >> v2:172.21.15.186:3300/0 conn(0x7fcd340cded0 0x7fcd340b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.165+0000 7fcd3945f700 1 --2- 172.21.15.156:0/1757227145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3415a760 0x7fcd34161200 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.165+0000 7fcd3945f700 1 -- 172.21.15.156:0/1757227145 >> 172.21.15.156:0/1757227145 conn(0x7fcd34072620 msgr2=0x7fcd340cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.165+0000 7fcd3945f700 1 -- 172.21.15.156:0/1757227145 shutdown_connections 2023-11-28T11:11:05.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.165+0000 7fcd3945f700 1 -- 172.21.15.156:0/1757227145 wait complete. 2023-11-28T11:11:05.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.166+0000 7fcd3945f700 1 Processor -- start 2023-11-28T11:11:05.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.167+0000 7fcd3945f700 1 -- start start 2023-11-28T11:11:05.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.167+0000 7fcd3945f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd340cded0 0x7fcd3414b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.167+0000 7fcd3945f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3415a760 0x7fcd340ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.167+0000 7fcd3945f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3415dc30 0x7fcd340b7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.167+0000 7fcd3945f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd340ca320 con 0x7fcd3415a760 2023-11-28T11:11:05.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.167+0000 7fcd3945f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd34159b30 con 0x7fcd3415dc30 2023-11-28T11:11:05.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.167+0000 7fcd3945f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3414bb40 con 0x7fcd340cded0 2023-11-28T11:11:05.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.167+0000 7fcd327fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3415a760 0x7fcd340ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.167+0000 7fcd327fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3415a760 0x7fcd340ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35934/0 (socket says 172.21.15.156:35934) 2023-11-28T11:11:05.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.167+0000 7fcd327fc700 1 -- 172.21.15.156:0/3341086736 learned_addr learned my addr 172.21.15.156:0/3341086736 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.167+0000 7fcd337fe700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3415dc30 0x7fcd340b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.167+0000 7fcd32ffd700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd340cded0 0x7fcd3414b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.168+0000 7fcd327fc700 1 -- 172.21.15.156:0/3341086736 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd340cded0 msgr2=0x7fcd3414b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.168+0000 7fcd327fc700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd340cded0 0x7fcd3414b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.168+0000 7fcd327fc700 1 -- 172.21.15.156:0/3341086736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3415dc30 msgr2=0x7fcd340b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.168+0000 7fcd327fc700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3415dc30 0x7fcd340b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.168+0000 7fcd327fc700 1 -- 172.21.15.156:0/3341086736 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd280094a0 con 0x7fcd3415a760 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.168+0000 7fcd327fc700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3415a760 0x7fcd340ccb00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400c550 tx=0x7fcd2400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.168+0000 7fcd1b7fe700 1 -- 172.21.15.156:0/3341086736 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24004c80 con 0x7fcd3415a760 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.168+0000 7fcd1b7fe700 1 -- 172.21.15.156:0/3341086736 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd24004e00 con 0x7fcd3415a760 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.168+0000 7fcd3945f700 1 -- 172.21.15.156:0/3341086736 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd340b5670 con 0x7fcd3415a760 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.168+0000 7fcd1b7fe700 1 -- 172.21.15.156:0/3341086736 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24007310 con 0x7fcd3415a760 2023-11-28T11:11:05.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.168+0000 7fcd3945f700 1 -- 172.21.15.156:0/3341086736 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd340b5390 con 0x7fcd3415a760 2023-11-28T11:11:05.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.169+0000 7fcd1b7fe700 1 -- 172.21.15.156:0/3341086736 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcd2401a020 con 0x7fcd3415a760 2023-11-28T11:11:05.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.170+0000 7fcd1b7fe700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcd20068320 0x7fcd2006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.170+0000 7fcd1b7fe700 1 -- 172.21.15.156:0/3341086736 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fcd24085810 con 0x7fcd3415a760 2023-11-28T11:11:05.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.170+0000 7fcd32ffd700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcd20068320 0x7fcd2006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.170+0000 7fcd32ffd700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcd20068320 0x7fcd2006a7d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcd280090b0 tx=0x7fcd28008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.172+0000 7fcd455e8b80 1 Processor -- start 2023-11-28T11:11:05.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.172+0000 7fcd455e8b80 1 -- start start 2023-11-28T11:11:05.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd455e8b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e718c0 0x558f71e47ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd455e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e6f1e0 0x558f71e71c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd455e8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e721d0 0x558f71e8d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd455e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f71d31c70 con 0x558f71e6f1e0 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd455e8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f71da0400 con 0x558f71e721d0 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd455e8b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f71d99930 con 0x558f71e718c0 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd327fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e6f1e0 0x558f71e71c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd327fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e6f1e0 0x558f71e71c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35942/0 (socket says 172.21.15.156:35942) 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd327fc700 1 -- 172.21.15.156:0/3899718659 learned_addr learned my addr 172.21.15.156:0/3899718659 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd337fe700 1 --2- 172.21.15.156:0/3899718659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e721d0 0x558f71e8d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd32ffd700 1 --2- 172.21.15.156:0/3899718659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e718c0 0x558f71e47ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd327fc700 1 -- 172.21.15.156:0/3899718659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e718c0 msgr2=0x558f71e47ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd327fc700 1 --2- 172.21.15.156:0/3899718659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e718c0 0x558f71e47ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd327fc700 1 -- 172.21.15.156:0/3899718659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e721d0 msgr2=0x558f71e8d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd327fc700 1 --2- 172.21.15.156:0/3899718659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e721d0 0x558f71e8d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd327fc700 1 -- 172.21.15.156:0/3899718659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f71c1b690 con 0x558f71e6f1e0 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd327fc700 1 --2- 172.21.15.156:0/3899718659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e6f1e0 0x558f71e71c90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcd240558c0 tx=0x7fcd24055520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4824a3639a8df6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd18ff9700 1 -- 172.21.15.156:0/3899718659 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24007470 con 0x558f71e6f1e0 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd18ff9700 1 -- 172.21.15.156:0/3899718659 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd240075d0 con 0x558f71e6f1e0 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.173+0000 7fcd18ff9700 1 -- 172.21.15.156:0/3899718659 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24007740 con 0x558f71e6f1e0 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.174+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3899718659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e6f1e0 msgr2=0x558f71e71c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.174+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/3899718659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e6f1e0 0x558f71e71c90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcd240558c0 tx=0x7fcd24055520 comp rx=0 tx=0).stop 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.174+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3899718659 shutdown_connections 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.174+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/3899718659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e721d0 0x558f71e8d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.174+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/3899718659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e718c0 0x558f71e47ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.174+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/3899718659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e6f1e0 0x558f71e71c90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.174+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3899718659 >> 172.21.15.156:0/3899718659 conn(0x558f71e44e10 msgr2=0x558f71e457a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.174+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3899718659 shutdown_connections 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.174+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3899718659 wait complete. 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.175+0000 7fcd455e8b80 1 -- start start 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.175+0000 7fcd455e8b80 1 Processor -- start 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd455e8b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e6f1e0 0x558f71e84c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd455e8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e718c0 0x558f71e90da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd455e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e721d0 0x558f71e702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd455e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f71c1d980 con 0x558f71e721d0 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd455e8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f71e66860 con 0x558f71e718c0 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd455e8b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f71e912e0 con 0x558f71e6f1e0 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd455e8b80 10 client.0 _ll_register_callbacks cb 0x558f71c3f2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd337fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e721d0 0x558f71e702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd337fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e721d0 0x558f71e702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35950/0 (socket says 172.21.15.156:35950) 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd337fe700 1 -- 172.21.15.156:0/3300385906 learned_addr learned my addr 172.21.15.156:0/3300385906 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd327fc700 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e718c0 0x558f71e90da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd32ffd700 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e6f1e0 0x558f71e84c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd337fe700 1 -- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e6f1e0 msgr2=0x558f71e84c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd337fe700 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e6f1e0 0x558f71e84c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd337fe700 1 -- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e718c0 msgr2=0x558f71e90da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd337fe700 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e718c0 0x558f71e90da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd337fe700 1 -- 172.21.15.156:0/3300385906 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd340b5390 con 0x558f71e721d0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd327fc700 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e718c0 0x558f71e90da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd337fe700 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e721d0 0x558f71e702e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c004340 tx=0x7fcd2c004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30118284db1809b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd11ffb700 1 -- 172.21.15.156:0/3300385906 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c016840 con 0x558f71e721d0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3300385906 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f71d671f0 con 0x558f71e721d0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd11ffb700 1 -- 172.21.15.156:0/3300385906 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fcd2c0169a0 con 0x558f71e721d0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd11ffb700 1 -- 172.21.15.156:0/3300385906 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd2c016b60 con 0x558f71e721d0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.176+0000 7fcd455e8b80 10 client.6228 fetch_fsmap learned FSMap version 31 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.177+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3300385906 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x558f71bc7e40 con 0x558f71e721d0 2023-11-28T11:11:05.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.177+0000 7fcd11ffb700 1 -- 172.21.15.156:0/3300385906 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f71d671f0 con 0x558f71e721d0 2023-11-28T11:11:05.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.177+0000 7fcd11ffb700 1 -- 172.21.15.156:0/3300385906 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fcd2c016cc0 con 0x558f71e721d0 2023-11-28T11:11:05.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.177+0000 7fcd455e8b80 10 client.6228 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T11:11:05.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.177+0000 7fcd455e8b80 10 client.6228 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-11-28T11:11:05.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.177+0000 7fcd455e8b80 20 client.6228 populate_metadata read hostname 'smithi156' 2023-11-28T11:11:05.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.177+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x558f71e63ca0 0x558f71e67830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.177+0000 7fcd455e8b80 4 client.6228 mds_command: new command op to 6021 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:05.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.177+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3300385906 --> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f71c1dfc0 con 0x558f71e63ca0 2023-11-28T11:11:05.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.177+0000 7fcd32ffd700 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x558f71e63ca0 0x558f71e67830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.177+0000 7fcd11ffb700 10 client.6228 ms_handle_connect on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:05.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.177+0000 7fcd32ffd700 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x558f71e63ca0 0x558f71e67830 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=610799bb6fdeccd3 server_cookie=30572554114b7b2a in_seq=0 out_seq=0 2023-11-28T11:11:05.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd11ffb700 1 -- 172.21.15.156:0/3300385906 <== mds.0 v2:172.21.15.186:6833/3811791267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x558f71c1dfc0 con 0x558f71e63ca0 2023-11-28T11:11:05.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd11ffb700 10 client.6228 handle_command_reply: tid=0 2023-11-28T11:11:05.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd455e8b80 1 client.6228 shutdown 2023-11-28T11:11:05.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3300385906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e721d0 msgr2=0x558f71e702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e721d0 0x558f71e702e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c004340 tx=0x7fcd2c004370 comp rx=0 tx=0).stop 2023-11-28T11:11:05.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3300385906 shutdown_connections 2023-11-28T11:11:05.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x558f71e63ca0 0x558f71e67830 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e721d0 0x558f71e702e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e6f1e0 0x558f71e84c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd11ffb700 0 client.6228 ms_handle_reset on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:05.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/3300385906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e718c0 0x558f71e90da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3300385906 >> 172.21.15.156:0/3300385906 conn(0x558f71e44e10 msgr2=0x558f71e462e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3300385906 shutdown_connections 2023-11-28T11:11:05.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd455e8b80 1 -- 172.21.15.156:0/3300385906 wait complete. 2023-11-28T11:11:05.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.178+0000 7fcd455e8b80 20 client.6228 trim_cache size 0 max 16384 2023-11-28T11:11:05.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.195+0000 7fcd455e8b80 1 Processor -- start 2023-11-28T11:11:05.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.195+0000 7fcd455e8b80 1 -- start start 2023-11-28T11:11:05.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.195+0000 7fcd455e8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e6f1e0 0x558f71e84850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.195+0000 7fcd455e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 0x558f71eae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.195+0000 7fcd455e8b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e721d0 0x558f71e700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.195+0000 7fcd455e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f71e711e0 con 0x558f71e718c0 2023-11-28T11:11:05.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.195+0000 7fcd455e8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f71e8ffc0 con 0x558f71e6f1e0 2023-11-28T11:11:05.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.195+0000 7fcd455e8b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f71e84130 con 0x558f71e721d0 2023-11-28T11:11:05.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd327fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 0x558f71eae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd327fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 0x558f71eae910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35956/0 (socket says 172.21.15.156:35956) 2023-11-28T11:11:05.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd327fc700 1 -- 172.21.15.156:0/2730614184 learned_addr learned my addr 172.21.15.156:0/2730614184 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd32ffd700 1 --2- 172.21.15.156:0/2730614184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e6f1e0 0x558f71e84850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd337fe700 1 --2- 172.21.15.156:0/2730614184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e721d0 0x558f71e700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd327fc700 1 -- 172.21.15.156:0/2730614184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e721d0 msgr2=0x558f71e700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd327fc700 1 --2- 172.21.15.156:0/2730614184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e721d0 0x558f71e700f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd327fc700 1 -- 172.21.15.156:0/2730614184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e6f1e0 msgr2=0x558f71e84850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd327fc700 1 --2- 172.21.15.156:0/2730614184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e6f1e0 0x558f71e84850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd327fc700 1 -- 172.21.15.156:0/2730614184 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f71d601f0 con 0x558f71e718c0 2023-11-28T11:11:05.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd327fc700 1 --2- 172.21.15.156:0/2730614184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 0x558f71eae910 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcd24052070 tx=0x7fcd24059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9643b44714d4d733 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd197fa700 1 -- 172.21.15.156:0/2730614184 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2405dea0 con 0x558f71e718c0 2023-11-28T11:11:05.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd197fa700 1 -- 172.21.15.156:0/2730614184 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd240550b0 con 0x558f71e718c0 2023-11-28T11:11:05.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.196+0000 7fcd197fa700 1 -- 172.21.15.156:0/2730614184 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2405d7c0 con 0x558f71e718c0 2023-11-28T11:11:05.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.197+0000 7fcd455e8b80 1 -- 172.21.15.156:0/2730614184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 msgr2=0x558f71eae910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.197+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/2730614184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 0x558f71eae910 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcd24052070 tx=0x7fcd24059000 comp rx=0 tx=0).stop 2023-11-28T11:11:05.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.197+0000 7fcd455e8b80 1 -- 172.21.15.156:0/2730614184 shutdown_connections 2023-11-28T11:11:05.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.197+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/2730614184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e721d0 0x558f71e700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.197+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/2730614184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e6f1e0 0x558f71e84850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.197+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/2730614184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 0x558f71eae910 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.197+0000 7fcd455e8b80 1 -- 172.21.15.156:0/2730614184 >> 172.21.15.156:0/2730614184 conn(0x558f71e44e10 msgr2=0x558f71e41ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.197+0000 7fcd455e8b80 1 -- 172.21.15.156:0/2730614184 shutdown_connections 2023-11-28T11:11:05.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.197+0000 7fcd455e8b80 1 -- 172.21.15.156:0/2730614184 wait complete. 2023-11-28T11:11:05.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.197+0000 7fcd455e8b80 1 -- start start 2023-11-28T11:11:05.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.197+0000 7fcd455e8b80 1 Processor -- start 2023-11-28T11:11:05.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd455e8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e6f1e0 0x558f71e84670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd455e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 0x558f71e41230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd455e8b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e721d0 0x558f71eeb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd455e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f71e6cb40 con 0x558f71e718c0 2023-11-28T11:11:05.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd455e8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f71e6cfa0 con 0x558f71e6f1e0 2023-11-28T11:11:05.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd455e8b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f71e84bb0 con 0x558f71e721d0 2023-11-28T11:11:05.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd455e8b80 10 client.0 _ll_register_callbacks cb 0x558f71c3f2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:05.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd327fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 0x558f71e41230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd32ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e6f1e0 0x558f71e84670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd327fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 0x558f71e41230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35962/0 (socket says 172.21.15.156:35962) 2023-11-28T11:11:05.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd327fc700 1 -- 172.21.15.156:0/1554539653 learned_addr learned my addr 172.21.15.156:0/1554539653 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd337fe700 1 --2- 172.21.15.156:0/1554539653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e721d0 0x558f71eeb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd327fc700 1 -- 172.21.15.156:0/1554539653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e721d0 msgr2=0x558f71eeb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd327fc700 1 --2- 172.21.15.156:0/1554539653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e721d0 0x558f71eeb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd327fc700 1 -- 172.21.15.156:0/1554539653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e6f1e0 msgr2=0x558f71e84670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd327fc700 1 --2- 172.21.15.156:0/1554539653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e6f1e0 0x558f71e84670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.198+0000 7fcd327fc700 1 -- 172.21.15.156:0/1554539653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f71c1b690 con 0x558f71e718c0 2023-11-28T11:11:05.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd327fc700 1 --2- 172.21.15.156:0/1554539653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 0x558f71e41230 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcd240558c0 tx=0x7fcd24058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2a5e63f28f32b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:11:05.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd11ffb700 1 -- 172.21.15.156:0/1554539653 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2405dea0 con 0x558f71e718c0 2023-11-28T11:11:05.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd11ffb700 1 -- 172.21.15.156:0/1554539653 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fcd2405d9c0 con 0x558f71e718c0 2023-11-28T11:11:05.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd455e8b80 1 -- 172.21.15.156:0/1554539653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f71d57210 con 0x558f71e718c0 2023-11-28T11:11:05.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd11ffb700 1 -- 172.21.15.156:0/1554539653 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd24051f00 con 0x558f71e718c0 2023-11-28T11:11:05.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd455e8b80 10 client.6234 fetch_fsmap learned FSMap version 31 2023-11-28T11:11:05.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd455e8b80 1 -- 172.21.15.156:0/1554539653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x558f71bee2b0 con 0x558f71e718c0 2023-11-28T11:11:05.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd11ffb700 1 -- 172.21.15.156:0/1554539653 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f71d57210 con 0x558f71e718c0 2023-11-28T11:11:05.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd11ffb700 1 -- 172.21.15.156:0/1554539653 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fcd24059b20 con 0x558f71e718c0 2023-11-28T11:11:05.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd455e8b80 10 client.6234 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T11:11:05.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd455e8b80 10 client.6234 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-11-28T11:11:05.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd455e8b80 20 client.6234 populate_metadata read hostname 'smithi156' 2023-11-28T11:11:05.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/1554539653 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x558f71eebd90 0x558f71e8ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd455e8b80 4 client.6234 mds_command: new command op to 6021 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2023-11-28T11:11:05.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd455e8b80 1 -- 172.21.15.156:0/1554539653 --> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x558f71d41220 con 0x558f71eebd90 2023-11-28T11:11:05.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.199+0000 7fcd32ffd700 1 --2- 172.21.15.156:0/1554539653 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x558f71eebd90 0x558f71e8ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.200+0000 7fcd11ffb700 10 client.6234 ms_handle_connect on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:05.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.200+0000 7fcd32ffd700 1 --2- 172.21.15.156:0/1554539653 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x558f71eebd90 0x558f71e8ef40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a6fbf8713503143 server_cookie=3b3d7ae995ac05b5 in_seq=0 out_seq=0 2023-11-28T11:11:05.207 INFO:tasks.ceph.mds.d.smithi186.stderr:2023-11-28T11:11:05.200+0000 7f634a59b700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6186=0-4194304@1} caps={6186=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6186 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55dbdd779600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:11:05.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.200+0000 7fcd11ffb700 1 -- 172.21.15.156:0/1554539653 <== mds.0 v2:172.21.15.186:6833/3811791267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558f71d41220 con 0x558f71eebd90 2023-11-28T11:11:05.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.200+0000 7fcd11ffb700 10 client.6234 handle_command_reply: tid=0 2023-11-28T11:11:05.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.200+0000 7fcd455e8b80 1 client.6234 shutdown 2023-11-28T11:11:05.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.200+0000 7fcd455e8b80 1 -- 172.21.15.156:0/1554539653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 msgr2=0x558f71e41230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.200+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/1554539653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 0x558f71e41230 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcd240558c0 tx=0x7fcd24058040 comp rx=0 tx=0).stop 2023-11-28T11:11:05.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd455e8b80 1 -- 172.21.15.156:0/1554539653 shutdown_connections 2023-11-28T11:11:05.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/1554539653 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x558f71eebd90 0x558f71e8ef40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/1554539653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558f71e721d0 0x558f71eeb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/1554539653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558f71e6f1e0 0x558f71e84670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd455e8b80 1 --2- 172.21.15.156:0/1554539653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f71e718c0 0x558f71e41230 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd455e8b80 1 -- 172.21.15.156:0/1554539653 >> 172.21.15.156:0/1554539653 conn(0x558f71e44e10 msgr2=0x558f71ea5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd11ffb700 0 client.6234 ms_handle_reset on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:05.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd455e8b80 1 -- 172.21.15.156:0/1554539653 shutdown_connections 2023-11-28T11:11:05.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd455e8b80 1 -- 172.21.15.156:0/1554539653 wait complete. 2023-11-28T11:11:05.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd455e8b80 20 client.6234 trim_cache size 0 max 16384 2023-11-28T11:11:05.210 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:11:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: "return_code": 0, 2023-11-28T11:11:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_tag": "3a339b0e-123e-4bba-811f-329f50a8bfa3", 2023-11-28T11:11:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: "mode": "asynchronous" 2023-11-28T11:11:05.210 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:11:05.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd3945f700 1 -- 172.21.15.156:0/3341086736 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcd20068320 msgr2=0x7fcd2006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd3945f700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcd20068320 0x7fcd2006a7d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcd280090b0 tx=0x7fcd28008030 comp rx=0 tx=0).stop 2023-11-28T11:11:05.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd3945f700 1 -- 172.21.15.156:0/3341086736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3415a760 msgr2=0x7fcd340ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd3945f700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3415a760 0x7fcd340ccb00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400c550 tx=0x7fcd2400c860 comp rx=0 tx=0).stop 2023-11-28T11:11:05.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd3945f700 1 -- 172.21.15.156:0/3341086736 shutdown_connections 2023-11-28T11:11:05.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd3945f700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3415dc30 0x7fcd340b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd3945f700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fcd20068320 0x7fcd2006a7d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd3945f700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd340cded0 0x7fcd3414b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd3945f700 1 --2- 172.21.15.156:0/3341086736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3415a760 0x7fcd340ccb00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd3945f700 1 -- 172.21.15.156:0/3341086736 >> 172.21.15.156:0/3341086736 conn(0x7fcd34072620 msgr2=0x7fcd3415ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd3945f700 1 -- 172.21.15.156:0/3341086736 shutdown_connections 2023-11-28T11:11:05.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.201+0000 7fcd3945f700 1 -- 172.21.15.156:0/3341086736 wait complete. 2023-11-28T11:11:05.215 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T11:11:05.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.336+0000 7fec6fcb1700 1 Processor -- start 2023-11-28T11:11:05.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.337+0000 7fec6fcb1700 1 -- start start 2023-11-28T11:11:05.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.337+0000 7fec6fcb1700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fec6815cef0 0x7fec680af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.337+0000 7fec6fcb1700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec6806ba90 0x7fec680ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.337+0000 7fec6fcb1700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec680cc5f0 con 0x7fec680cbdf0 2023-11-28T11:11:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.337+0000 7fec6fcb1700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6815d470 con 0x7fec6815cef0 2023-11-28T11:11:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.337+0000 7fec6fcb1700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6815d750 con 0x7fec6806ba90 2023-11-28T11:11:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6e24e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec6806ba90 0x7fec680ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6da4d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fec6815cef0 0x7fec680af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6e24e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec6806ba90 0x7fec680ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:39834/0 (socket says 172.21.15.156:39834) 2023-11-28T11:11:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6e24e700 1 -- 172.21.15.156:0/3855352430 learned_addr learned my addr 172.21.15.156:0/3855352430 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6d24c700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fec680cbdf0 0x7fec6806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:32974/0 (socket says 172.21.15.156:32974) 2023-11-28T11:11:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6e24e700 1 -- 172.21.15.156:0/3855352430 >> v2:172.21.15.186:3300/0 conn(0x7fec6815cef0 msgr2=0x7fec680af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6e24e700 1 --2- 172.21.15.156:0/3855352430 >> v2:172.21.15.186:3300/0 conn(0x7fec6815cef0 0x7fec680af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6e24e700 1 -- 172.21.15.156:0/3855352430 >> v1:172.21.15.186:6789/0 conn(0x7fec680cbdf0 legacy=0x7fec6806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6ca4b700 1 -- 172.21.15.156:0/3855352430 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042956110 0 0) 0x7fec680cc5f0 con 0x7fec680cbdf0 2023-11-28T11:11:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6e24e700 1 -- 172.21.15.156:0/3855352430 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6815da30 con 0x7fec6806ba90 2023-11-28T11:11:05.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6e24e700 1 --2- 172.21.15.156:0/3855352430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec6806ba90 0x7fec680ca650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fec60009e80 tx=0x7fec6000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ce356008ebb79b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6ca4b700 1 -- 172.21.15.156:0/3855352430 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6000e070 con 0x7fec6806ba90 2023-11-28T11:11:05.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6ca4b700 1 -- 172.21.15.156:0/3855352430 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6000b8d0 con 0x7fec6806ba90 2023-11-28T11:11:05.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.338+0000 7fec6ca4b700 1 -- 172.21.15.156:0/3855352430 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6000bc80 con 0x7fec6806ba90 2023-11-28T11:11:05.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.339+0000 7fec6fcb1700 1 -- 172.21.15.156:0/3855352430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec6806ba90 msgr2=0x7fec680ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.339+0000 7fec6fcb1700 1 --2- 172.21.15.156:0/3855352430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec6806ba90 0x7fec680ca650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fec60009e80 tx=0x7fec6000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:05.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.339+0000 7fec6fcb1700 1 -- 172.21.15.156:0/3855352430 shutdown_connections 2023-11-28T11:11:05.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.339+0000 7fec6fcb1700 1 --2- 172.21.15.156:0/3855352430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec6806ba90 0x7fec680ca650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.339+0000 7fec6fcb1700 1 --2- 172.21.15.156:0/3855352430 >> v2:172.21.15.186:3300/0 conn(0x7fec6815cef0 0x7fec680af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.339+0000 7fec6fcb1700 1 -- 172.21.15.156:0/3855352430 >> 172.21.15.156:0/3855352430 conn(0x7fec680d0000 msgr2=0x7fec680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.339+0000 7fec6fcb1700 1 -- 172.21.15.156:0/3855352430 shutdown_connections 2023-11-28T11:11:05.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.339+0000 7fec6fcb1700 1 -- 172.21.15.156:0/3855352430 wait complete. 2023-11-28T11:11:05.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.340+0000 7fec6fcb1700 1 Processor -- start 2023-11-28T11:11:05.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.340+0000 7fec6fcb1700 1 -- start start 2023-11-28T11:11:05.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6fcb1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec6806ba90 0x7fec680b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6fcb1700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec680cbdf0 0x7fec68156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6fcb1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec6815cef0 0x7fec681565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6fcb1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec68158d60 con 0x7fec6815cef0 2023-11-28T11:11:05.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6fcb1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec68158430 con 0x7fec6806ba90 2023-11-28T11:11:05.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6fcb1700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec68158710 con 0x7fec680cbdf0 2023-11-28T11:11:05.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6e24e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec6815cef0 0x7fec681565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6e24e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec6815cef0 0x7fec681565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35976/0 (socket says 172.21.15.156:35976) 2023-11-28T11:11:05.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6e24e700 1 -- 172.21.15.156:0/1527879756 learned_addr learned my addr 172.21.15.156:0/1527879756 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6da4d700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec6806ba90 0x7fec680b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6d24c700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec680cbdf0 0x7fec68156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6e24e700 1 -- 172.21.15.156:0/1527879756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec680cbdf0 msgr2=0x7fec68156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6e24e700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec680cbdf0 0x7fec68156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6e24e700 1 -- 172.21.15.156:0/1527879756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec6806ba90 msgr2=0x7fec680b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6e24e700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec6806ba90 0x7fec680b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6e24e700 1 -- 172.21.15.156:0/1527879756 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60009b30 con 0x7fec6815cef0 2023-11-28T11:11:05.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6da4d700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec6806ba90 0x7fec680b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:05.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec6e24e700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec6815cef0 0x7fec681565a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fec60012000 tx=0x7fec60004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.341+0000 7fec5e7fc700 1 -- 172.21.15.156:0/1527879756 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60004880 con 0x7fec6815cef0 2023-11-28T11:11:05.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.342+0000 7fec5e7fc700 1 -- 172.21.15.156:0/1527879756 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec600099b0 con 0x7fec6815cef0 2023-11-28T11:11:05.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.342+0000 7fec6fcb1700 1 -- 172.21.15.156:0/1527879756 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6815dff0 con 0x7fec6815cef0 2023-11-28T11:11:05.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.342+0000 7fec5e7fc700 1 -- 172.21.15.156:0/1527879756 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6001c680 con 0x7fec6815cef0 2023-11-28T11:11:05.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.342+0000 7fec6fcb1700 1 -- 172.21.15.156:0/1527879756 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6815dd10 con 0x7fec6815cef0 2023-11-28T11:11:05.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.343+0000 7fec5e7fc700 1 -- 172.21.15.156:0/1527879756 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fec60022030 con 0x7fec6815cef0 2023-11-28T11:11:05.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.343+0000 7fec5e7fc700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fec4c0682f0 0x7fec4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.343+0000 7fec5e7fc700 1 -- 172.21.15.156:0/1527879756 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fec60087550 con 0x7fec6815cef0 2023-11-28T11:11:05.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.343+0000 7fec6da4d700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fec4c0682f0 0x7fec4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.344+0000 7fec6da4d700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fec4c0682f0 0x7fec4c06a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fec58009f30 tx=0x7fec58009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.346+0000 7fec7be3ab80 1 Processor -- start 2023-11-28T11:11:05.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.346+0000 7fec7be3ab80 1 -- start start 2023-11-28T11:11:05.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.346+0000 7fec7be3ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c5860 0x55a6ef89ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.346+0000 7fec7be3ab80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef8c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec7be3ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c6170 0x55a6ef8e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec7be3ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ef785c10 con 0x55a6ef8c5860 2023-11-28T11:11:05.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec7be3ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ef7f43a0 con 0x55a6ef8c6170 2023-11-28T11:11:05.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec7be3ab80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6ef7ed8d0 con 0x55a6ef8c3180 2023-11-28T11:11:05.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec6da4d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c5860 0x55a6ef89ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec6da4d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c5860 0x55a6ef89ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35980/0 (socket says 172.21.15.156:35980) 2023-11-28T11:11:05.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec6da4d700 1 -- 172.21.15.156:0/1232873010 learned_addr learned my addr 172.21.15.156:0/1232873010 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec6d24c700 1 --2- 172.21.15.156:0/1232873010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef8c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec6e24e700 1 --2- 172.21.15.156:0/1232873010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c6170 0x55a6ef8e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec6da4d700 1 -- 172.21.15.156:0/1232873010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 msgr2=0x55a6ef8c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec6da4d700 1 --2- 172.21.15.156:0/1232873010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef8c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec6da4d700 1 -- 172.21.15.156:0/1232873010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c6170 msgr2=0x55a6ef8e17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec6da4d700 1 --2- 172.21.15.156:0/1232873010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c6170 0x55a6ef8e17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec6da4d700 1 -- 172.21.15.156:0/1232873010 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6ef66f630 con 0x55a6ef8c5860 2023-11-28T11:11:05.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec6da4d700 1 --2- 172.21.15.156:0/1232873010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c5860 0x55a6ef89ba70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fec58004160 tx=0x7fec58014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c34e9a7daf75b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec4b7fe700 1 -- 172.21.15.156:0/1232873010 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58004520 con 0x55a6ef8c5860 2023-11-28T11:11:05.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec4b7fe700 1 -- 172.21.15.156:0/1232873010 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5801b070 con 0x55a6ef8c5860 2023-11-28T11:11:05.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.347+0000 7fec4b7fe700 1 -- 172.21.15.156:0/1232873010 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58022440 con 0x55a6ef8c5860 2023-11-28T11:11:05.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.348+0000 7fec7be3ab80 1 -- 172.21.15.156:0/1232873010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c5860 msgr2=0x55a6ef89ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.348+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/1232873010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c5860 0x55a6ef89ba70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fec58004160 tx=0x7fec58014000 comp rx=0 tx=0).stop 2023-11-28T11:11:05.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.348+0000 7fec7be3ab80 1 -- 172.21.15.156:0/1232873010 shutdown_connections 2023-11-28T11:11:05.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.348+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/1232873010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c6170 0x55a6ef8e17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.348+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/1232873010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c5860 0x55a6ef89ba70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.348+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/1232873010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef8c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.348+0000 7fec7be3ab80 1 -- 172.21.15.156:0/1232873010 >> 172.21.15.156:0/1232873010 conn(0x55a6ef898db0 msgr2=0x55a6ef899740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.348+0000 7fec7be3ab80 1 -- 172.21.15.156:0/1232873010 shutdown_connections 2023-11-28T11:11:05.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.348+0000 7fec7be3ab80 1 -- 172.21.15.156:0/1232873010 wait complete. 2023-11-28T11:11:05.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.349+0000 7fec7be3ab80 1 -- start start 2023-11-28T11:11:05.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.349+0000 7fec7be3ab80 1 Processor -- start 2023-11-28T11:11:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.349+0000 7fec7be3ab80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef8c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec7be3ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8d7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec7be3ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c6170 0x55a6ef8b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec7be3ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ef671920 con 0x55a6ef8c6170 2023-11-28T11:11:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec7be3ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ef8b7d40 con 0x55a6ef8c5860 2023-11-28T11:11:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec7be3ab80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6ef8d8860 con 0x55a6ef8c3180 2023-11-28T11:11:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec7be3ab80 10 client.0 _ll_register_callbacks cb 0x55a6ef693290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec6e24e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c6170 0x55a6ef8b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec6e24e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c6170 0x55a6ef8b6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35990/0 (socket says 172.21.15.156:35990) 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec6e24e700 1 -- 172.21.15.156:0/3034348210 learned_addr learned my addr 172.21.15.156:0/3034348210 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec6d24c700 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8d7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec6da4d700 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef8c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec6e24e700 1 -- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 msgr2=0x55a6ef8c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec6e24e700 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef8c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec6e24e700 1 -- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 msgr2=0x55a6ef8d7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec6e24e700 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8d7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec6e24e700 1 -- 172.21.15.156:0/3034348210 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec58011da0 con 0x55a6ef8c6170 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec6d24c700 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8d7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec6e24e700 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c6170 0x55a6ef8b6ff0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fec6000b370 tx=0x7fec60057370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9caa86a68c700a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:11:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec48ff9700 1 -- 172.21.15.156:0/3034348210 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec600092e0 con 0x55a6ef8c6170 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3034348210 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6ef7bb190 con 0x55a6ef8c6170 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec48ff9700 1 -- 172.21.15.156:0/3034348210 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fec600870c0 con 0x55a6ef8c6170 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec48ff9700 1 -- 172.21.15.156:0/3034348210 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec60087280 con 0x55a6ef8c6170 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec7be3ab80 10 client.6243 fetch_fsmap learned FSMap version 31 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.350+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3034348210 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55a6ef61bde0 con 0x55a6ef8c6170 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.351+0000 7fec48ff9700 1 -- 172.21.15.156:0/3034348210 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec60063080 con 0x55a6ef8c6170 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.351+0000 7fec48ff9700 1 -- 172.21.15.156:0/3034348210 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fec60058d20 con 0x55a6ef8c6170 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.351+0000 7fec7be3ab80 10 client.6243 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.351+0000 7fec7be3ab80 10 client.6243 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.351+0000 7fec7be3ab80 20 client.6243 populate_metadata read hostname 'smithi156' 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.351+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55a6ef8b58c0 0x55a6ef8b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.351+0000 7fec7be3ab80 4 client.6243 mds_command: new command op to 6021 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.351+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3034348210 --> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6ef671f60 con 0x55a6ef8b58c0 2023-11-28T11:11:05.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.351+0000 7fec6da4d700 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55a6ef8b58c0 0x55a6ef8b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.351+0000 7fec48ff9700 10 client.6243 ms_handle_connect on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:05.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.351+0000 7fec6da4d700 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55a6ef8b58c0 0x55a6ef8b5c90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0876b2ed32a979 server_cookie=f0733340f48adc91 in_seq=0 out_seq=0 2023-11-28T11:11:05.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec48ff9700 1 -- 172.21.15.156:0/3034348210 <== mds.0 v2:172.21.15.186:6833/3811791267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55a6ef671f60 con 0x55a6ef8b58c0 2023-11-28T11:11:05.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec48ff9700 10 client.6243 handle_command_reply: tid=0 2023-11-28T11:11:05.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec7be3ab80 1 client.6243 shutdown 2023-11-28T11:11:05.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3034348210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c6170 msgr2=0x55a6ef8b6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c6170 0x55a6ef8b6ff0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fec6000b370 tx=0x7fec60057370 comp rx=0 tx=0).stop 2023-11-28T11:11:05.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3034348210 shutdown_connections 2023-11-28T11:11:05.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55a6ef8b58c0 0x55a6ef8b5c90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c6170 0x55a6ef8b6ff0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec48ff9700 0 client.6243 ms_handle_reset on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:05.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef8c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3034348210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8d7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3034348210 >> 172.21.15.156:0/3034348210 conn(0x55a6ef898db0 msgr2=0x55a6ef89a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3034348210 shutdown_connections 2023-11-28T11:11:05.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3034348210 wait complete. 2023-11-28T11:11:05.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.352+0000 7fec7be3ab80 20 client.6243 trim_cache size 0 max 16384 2023-11-28T11:11:05.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.369+0000 7fec7be3ab80 1 Processor -- start 2023-11-28T11:11:05.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.369+0000 7fec7be3ab80 1 -- start start 2023-11-28T11:11:05.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.369+0000 7fec7be3ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8b58c0 0x55a6ef8b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.369+0000 7fec7be3ab80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef89cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.369+0000 7fec7be3ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8d7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.369+0000 7fec7be3ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ef8be660 con 0x55a6ef8b58c0 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.369+0000 7fec7be3ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ef8da590 con 0x55a6ef8c5860 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.369+0000 7fec7be3ab80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6ef8c4e50 con 0x55a6ef8c3180 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.369+0000 7fec6da4d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8b58c0 0x55a6ef8b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.369+0000 7fec6da4d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8b58c0 0x55a6ef8b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35998/0 (socket says 172.21.15.156:35998) 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.369+0000 7fec6da4d700 1 -- 172.21.15.156:0/3869139503 learned_addr learned my addr 172.21.15.156:0/3869139503 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.370+0000 7fec6e24e700 1 --2- 172.21.15.156:0/3869139503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8d7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.370+0000 7fec6d24c700 1 --2- 172.21.15.156:0/3869139503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef89cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.370+0000 7fec6da4d700 1 -- 172.21.15.156:0/3869139503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 msgr2=0x55a6ef89cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.370+0000 7fec6da4d700 1 --2- 172.21.15.156:0/3869139503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef89cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.370+0000 7fec6da4d700 1 -- 172.21.15.156:0/3869139503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 msgr2=0x55a6ef8d7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.370+0000 7fec6da4d700 1 --2- 172.21.15.156:0/3869139503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8d7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.370+0000 7fec6da4d700 1 -- 172.21.15.156:0/3869139503 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6ef7b4190 con 0x55a6ef8b58c0 2023-11-28T11:11:05.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.370+0000 7fec6d24c700 1 --2- 172.21.15.156:0/3869139503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef89cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.370+0000 7fec6da4d700 1 --2- 172.21.15.156:0/3869139503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8b58c0 0x55a6ef8b61a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fec58023000 tx=0x7fec58020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19181b2396db5e27 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.370+0000 7fec4bfff700 1 -- 172.21.15.156:0/3869139503 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58032040 con 0x55a6ef8b58c0 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.370+0000 7fec4bfff700 1 -- 172.21.15.156:0/3869139503 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5801b040 con 0x55a6ef8b58c0 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.370+0000 7fec4bfff700 1 -- 172.21.15.156:0/3869139503 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58034660 con 0x55a6ef8b58c0 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.371+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3869139503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8b58c0 msgr2=0x55a6ef8b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.371+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3869139503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8b58c0 0x55a6ef8b61a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fec58023000 tx=0x7fec58020000 comp rx=0 tx=0).stop 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.371+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3869139503 shutdown_connections 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.371+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3869139503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8d7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.371+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3869139503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8b58c0 0x55a6ef8b61a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.371+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3869139503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8c3180 0x55a6ef89cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.371+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3869139503 >> 172.21.15.156:0/3869139503 conn(0x55a6ef898db0 msgr2=0x55a6ef8b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.371+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3869139503 shutdown_connections 2023-11-28T11:11:05.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.371+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3869139503 wait complete. 2023-11-28T11:11:05.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.371+0000 7fec7be3ab80 1 -- start start 2023-11-28T11:11:05.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.371+0000 7fec7be3ab80 1 Processor -- start 2023-11-28T11:11:05.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.372+0000 7fec7be3ab80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8b58c0 0x55a6ef895e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.372+0000 7fec7be3ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c3180 0x55a6ef8963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.372+0000 7fec7be3ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.372+0000 7fec7be3ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ef8968f0 con 0x55a6ef8c3180 2023-11-28T11:11:05.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.372+0000 7fec7be3ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ef8b6720 con 0x55a6ef8c5860 2023-11-28T11:11:05.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.372+0000 7fec7be3ab80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6ef946400 con 0x55a6ef8b58c0 2023-11-28T11:11:05.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.372+0000 7fec7be3ab80 10 client.0 _ll_register_callbacks cb 0x55a6ef693290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:05.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.372+0000 7fec6d24c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c3180 0x55a6ef8963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.372+0000 7fec6d24c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c3180 0x55a6ef8963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36002/0 (socket says 172.21.15.156:36002) 2023-11-28T11:11:05.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.372+0000 7fec6d24c700 1 -- 172.21.15.156:0/3157202950 learned_addr learned my addr 172.21.15.156:0/3157202950 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:05.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.372+0000 7fec6da4d700 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8b58c0 0x55a6ef895e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.372+0000 7fec6e24e700 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec6d24c700 1 -- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8b58c0 msgr2=0x55a6ef895e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec6d24c700 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8b58c0 0x55a6ef895e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec6d24c700 1 -- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 msgr2=0x55a6ef8e2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec6d24c700 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8e2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec6d24c700 1 -- 172.21.15.156:0/3157202950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6ef66f630 con 0x55a6ef8c3180 2023-11-28T11:11:05.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec6e24e700 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:05.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec6d24c700 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c3180 0x55a6ef8963b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fec64000f60 tx=0x7fec64001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684f2f4bac190190 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:11:05.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec48ff9700 1 -- 172.21.15.156:0/3157202950 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64017430 con 0x55a6ef8c3180 2023-11-28T11:11:05.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec48ff9700 1 -- 172.21.15.156:0/3157202950 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fec6401ba00 con 0x55a6ef8c3180 2023-11-28T11:11:05.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3157202950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6ef7ab1b0 con 0x55a6ef8c3180 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec48ff9700 1 -- 172.21.15.156:0/3157202950 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec6401bbc0 con 0x55a6ef8c3180 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec7be3ab80 10 client.6249 fetch_fsmap learned FSMap version 31 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3157202950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55a6ef642250 con 0x55a6ef8c3180 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec48ff9700 1 -- 172.21.15.156:0/3157202950 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a6ef7ab1b0 con 0x55a6ef8c3180 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec48ff9700 1 -- 172.21.15.156:0/3157202950 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fec6401bd20 con 0x55a6ef8c3180 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec7be3ab80 10 client.6249 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec7be3ab80 10 client.6249 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec7be3ab80 20 client.6249 populate_metadata read hostname 'smithi156' 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55a6ef8c6170 0x55a6ef8e3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec7be3ab80 4 client.6249 mds_command: new command op to 6021 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.373+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3157202950 --> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a6ef7951c0 con 0x55a6ef8c6170 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec6da4d700 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55a6ef8c6170 0x55a6ef8e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec48ff9700 10 client.6249 ms_handle_connect on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:05.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec6da4d700 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55a6ef8c6170 0x55a6ef8e3200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=315d483780ebfc7a server_cookie=852ec499ee21f6ac in_seq=0 out_seq=0 2023-11-28T11:11:05.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec48ff9700 1 -- 172.21.15.156:0/3157202950 <== mds.0 v2:172.21.15.186:6833/3811791267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x55a6ef7951c0 con 0x55a6ef8c6170 2023-11-28T11:11:05.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec48ff9700 10 client.6249 handle_command_reply: tid=0 2023-11-28T11:11:05.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec7be3ab80 1 client.6249 shutdown 2023-11-28T11:11:05.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3157202950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c3180 msgr2=0x55a6ef8963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c3180 0x55a6ef8963b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fec64000f60 tx=0x7fec64001650 comp rx=0 tx=0).stop 2023-11-28T11:11:05.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3157202950 shutdown_connections 2023-11-28T11:11:05.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55a6ef8c6170 0x55a6ef8e3200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6ef8c5860 0x55a6ef8e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6ef8b58c0 0x55a6ef895e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec48ff9700 0 client.6249 ms_handle_reset on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:05.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec7be3ab80 1 --2- 172.21.15.156:0/3157202950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6ef8c3180 0x55a6ef8963b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.374+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3157202950 >> 172.21.15.156:0/3157202950 conn(0x55a6ef898db0 msgr2=0x55a6ef897150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3157202950 shutdown_connections 2023-11-28T11:11:05.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec7be3ab80 1 -- 172.21.15.156:0/3157202950 wait complete. 2023-11-28T11:11:05.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec7be3ab80 20 client.6249 trim_cache size 0 max 16384 2023-11-28T11:11:05.383 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:11:05.383 INFO:teuthology.orchestra.run.smithi156.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T11:11:05.383 INFO:teuthology.orchestra.run.smithi156.stdout: "scrubs": { 2023-11-28T11:11:05.383 INFO:teuthology.orchestra.run.smithi156.stdout: "3a339b0e-123e-4bba-811f-329f50a8bfa3": { 2023-11-28T11:11:05.383 INFO:teuthology.orchestra.run.smithi156.stdout: "path": "/", 2023-11-28T11:11:05.383 INFO:teuthology.orchestra.run.smithi156.stdout: "tag": "3a339b0e-123e-4bba-811f-329f50a8bfa3", 2023-11-28T11:11:05.384 INFO:teuthology.orchestra.run.smithi156.stdout: "options": "recursive" 2023-11-28T11:11:05.384 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:11:05.384 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:11:05.384 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:11:05.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec6fcb1700 1 -- 172.21.15.156:0/1527879756 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fec4c0682f0 msgr2=0x7fec4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec6fcb1700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fec4c0682f0 0x7fec4c06a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fec58009f30 tx=0x7fec58009210 comp rx=0 tx=0).stop 2023-11-28T11:11:05.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec6fcb1700 1 -- 172.21.15.156:0/1527879756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec6815cef0 msgr2=0x7fec681565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec6fcb1700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec6815cef0 0x7fec681565a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fec60012000 tx=0x7fec60004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec6fcb1700 1 -- 172.21.15.156:0/1527879756 shutdown_connections 2023-11-28T11:11:05.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec6fcb1700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec6815cef0 0x7fec681565a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec6fcb1700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec6806ba90 0x7fec680b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec6fcb1700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fec4c0682f0 0x7fec4c06a7a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec6fcb1700 1 --2- 172.21.15.156:0/1527879756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec680cbdf0 0x7fec68156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec6fcb1700 1 -- 172.21.15.156:0/1527879756 >> 172.21.15.156:0/1527879756 conn(0x7fec680d0000 msgr2=0x7fec6806a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec6fcb1700 1 -- 172.21.15.156:0/1527879756 shutdown_connections 2023-11-28T11:11:05.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:05.375+0000 7fec6fcb1700 1 -- 172.21.15.156:0/1527879756 wait complete. 2023-11-28T11:11:05.388 INFO:tasks.cephfs.filesystem:scrub status for tag:3a339b0e-123e-4bba-811f-329f50a8bfa3 - {'path': '/', 'tag': '3a339b0e-123e-4bba-811f-329f50a8bfa3', 'options': 'recursive'} 2023-11-28T11:11:08.809 INFO:tasks.ceph.mon.a.smithi156.stderr:2023-11-28T11:11:08.808+0000 7f6a4eb7c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-28T11:11:21.371 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:21.374 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:35.389 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T11:11:35.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.524+0000 7f2c378cc700 1 Processor -- start 2023-11-28T11:11:35.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.524+0000 7f2c378cc700 1 -- start start 2023-11-28T11:11:35.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c378cc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c3006e500 0x7f2c3015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c378cc700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c300aee40 con 0x7f2c3006a5e0 2023-11-28T11:11:35.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c378cc700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c3006c930 con 0x7f2c300ae9d0 2023-11-28T11:11:35.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c378cc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3015bf10 con 0x7f2c3006e500 2023-11-28T11:11:35.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c35e69700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2c3006a5e0 0x7f2c3006aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50890/0 (socket says 172.21.15.156:50890) 2023-11-28T11:11:35.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c35e69700 1 -- 172.21.15.156:0/1078913031 learned_addr learned my addr 172.21.15.156:0/1078913031 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:35.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c35668700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c3006e500 0x7f2c3015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245235408 0 0) 0x7f2c300aee40 con 0x7f2c3006a5e0 2023-11-28T11:11:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c1c0035b0 con 0x7f2c3006a5e0 2023-11-28T11:11:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755609065 0 0) 0x7f2c3006c930 con 0x7f2c300ae9d0 2023-11-28T11:11:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c300aee40 con 0x7f2c300ae9d0 2023-11-28T11:11:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4006163619 0 0) 0x7f2c1c0035b0 con 0x7f2c3006a5e0 2023-11-28T11:11:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c3006e500 msgr2=0x7f2c3015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c3006e500 0x7f2c3015b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 >> v1:172.21.15.186:6789/0 conn(0x7f2c300ae9d0 legacy=0x7f2c3006e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3006cc10 con 0x7f2c3006a5e0 2023-11-28T11:11:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f2c280019f0 con 0x7f2c3006a5e0 2023-11-28T11:11:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c35668700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c3006e500 0x7f2c3015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 >> v1:172.21.15.156:6789/0 conn(0x7f2c3006a5e0 legacy=0x7f2c3006aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c1c003bc0 0x7f2c1c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c1c004850 0x7f2c1c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c3006e500 0x7f2c1c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3015bf10 con 0x7f2c1c003bc0 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c00bba0 con 0x7f2c1c004850 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.525+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c00bd30 con 0x7f2c3006e500 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c35668700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c1c003bc0 0x7f2c1c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c34e67700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c1c004850 0x7f2c1c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c35e69700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c3006e500 0x7f2c1c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c35668700 1 -- 172.21.15.156:0/1078913031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c3006e500 msgr2=0x7f2c1c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c35668700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c3006e500 0x7f2c1c005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c35668700 1 -- 172.21.15.156:0/1078913031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c1c004850 msgr2=0x7f2c1c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:35.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c35668700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c1c004850 0x7f2c1c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c35668700 1 -- 172.21.15.156:0/1078913031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c200094a0 con 0x7f2c1c003bc0 2023-11-28T11:11:35.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c35668700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c1c003bc0 0x7f2c1c003fb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009d30 tx=0x7f2c1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2870254913eb84 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:35.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20009e80 con 0x7f2c1c003bc0 2023-11-28T11:11:35.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2000e9a0 con 0x7f2c1c003bc0 2023-11-28T11:11:35.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c27fff700 1 -- 172.21.15.156:0/1078913031 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2000ed10 con 0x7f2c1c003bc0 2023-11-28T11:11:35.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c378cc700 1 -- 172.21.15.156:0/1078913031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c1c003bc0 msgr2=0x7f2c1c003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.526+0000 7f2c378cc700 1 --2- 172.21.15.156:0/1078913031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c1c003bc0 0x7f2c1c003fb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009d30 tx=0x7f2c1c005880 comp rx=0 tx=0).stop 2023-11-28T11:11:35.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.527+0000 7f2c35e69700 1 -- 172.21.15.156:0/1078913031 reap_dead start 2023-11-28T11:11:35.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.527+0000 7f2c378cc700 1 -- 172.21.15.156:0/1078913031 shutdown_connections 2023-11-28T11:11:35.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.527+0000 7f2c378cc700 1 -- 172.21.15.156:0/1078913031 >> 172.21.15.156:0/1078913031 conn(0x7f2c300b12b0 msgr2=0x7f2c300b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:35.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.527+0000 7f2c378cc700 1 -- 172.21.15.156:0/1078913031 shutdown_connections 2023-11-28T11:11:35.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.527+0000 7f2c378cc700 1 -- 172.21.15.156:0/1078913031 wait complete. 2023-11-28T11:11:35.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.528+0000 7f2c378cc700 1 Processor -- start 2023-11-28T11:11:35.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.528+0000 7f2c378cc700 1 -- start start 2023-11-28T11:11:35.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c378cc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c1c003bc0 0x7f2c3014ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c378cc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c300b36b0 0x7f2c300ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c378cc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c300b0360 0x7f2c300b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c378cc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c300b0d40 con 0x7f2c300b0360 2023-11-28T11:11:35.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c378cc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c300cd390 con 0x7f2c1c003bc0 2023-11-28T11:11:35.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c378cc700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3015cf10 con 0x7f2c300b36b0 2023-11-28T11:11:35.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c35e69700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c300b0360 0x7f2c300b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c35e69700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c300b0360 0x7f2c300b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36024/0 (socket says 172.21.15.156:36024) 2023-11-28T11:11:35.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c35e69700 1 -- 172.21.15.156:0/1783915453 learned_addr learned my addr 172.21.15.156:0/1783915453 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:35.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c34e67700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c300b36b0 0x7f2c300ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c35668700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c1c003bc0 0x7f2c3014ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c35e69700 1 -- 172.21.15.156:0/1783915453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c300b36b0 msgr2=0x7f2c300ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c35e69700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c300b36b0 0x7f2c300ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c35e69700 1 -- 172.21.15.156:0/1783915453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c1c003bc0 msgr2=0x7f2c3014ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c35e69700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c1c003bc0 0x7f2c3014ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c35e69700 1 -- 172.21.15.156:0/1783915453 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3006cef0 con 0x7f2c300b0360 2023-11-28T11:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c35e69700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c300b0360 0x7f2c300b0800 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2c3006dbd0 tx=0x7f2c28007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c25ffb700 1 -- 172.21.15.156:0/1783915453 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28017070 con 0x7f2c300b0360 2023-11-28T11:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c25ffb700 1 -- 172.21.15.156:0/1783915453 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2800ec80 con 0x7f2c300b0360 2023-11-28T11:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c25ffb700 1 -- 172.21.15.156:0/1783915453 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800ee10 con 0x7f2c300b0360 2023-11-28T11:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.529+0000 7f2c378cc700 1 -- 172.21.15.156:0/1783915453 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c300664c0 con 0x7f2c300b0360 2023-11-28T11:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.530+0000 7f2c378cc700 1 -- 172.21.15.156:0/1783915453 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3015ac80 con 0x7f2c300b0360 2023-11-28T11:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.531+0000 7f2c25ffb700 1 -- 172.21.15.156:0/1783915453 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2c28013420 con 0x7f2c300b0360 2023-11-28T11:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.531+0000 7f2c25ffb700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2c140682f0 0x7f2c1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.531+0000 7f2c25ffb700 1 -- 172.21.15.156:0/1783915453 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f2c28085bb0 con 0x7f2c300b0360 2023-11-28T11:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.531+0000 7f2c35668700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2c140682f0 0x7f2c1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.532+0000 7f2c35668700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2c140682f0 0x7f2c1406a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2c200090b0 tx=0x7f2c20008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.533+0000 7f2c43a55b80 1 Processor -- start 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.533+0000 7f2c43a55b80 1 -- start start 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.534+0000 7f2c43a55b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5df860 0x56127b5b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.534+0000 7f2c43a55b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5dd180 0x56127b5dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.534+0000 7f2c43a55b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 0x56127b5fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.534+0000 7f2c43a55b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127b49fc10 con 0x56127b5dd180 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.534+0000 7f2c43a55b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127b50e3a0 con 0x56127b5df860 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.534+0000 7f2c43a55b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56127b5078d0 con 0x56127b5e0170 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.534+0000 7f2c34e67700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5dd180 0x56127b5dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.534+0000 7f2c34e67700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5dd180 0x56127b5dfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36030/0 (socket says 172.21.15.156:36030) 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.534+0000 7f2c34e67700 1 -- 172.21.15.156:0/2199397758 learned_addr learned my addr 172.21.15.156:0/2199397758 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.534+0000 7f2c35e69700 1 --2- 172.21.15.156:0/2199397758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 0x56127b5fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.534+0000 7f2c35668700 1 --2- 172.21.15.156:0/2199397758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5df860 0x56127b5b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.535+0000 7f2c34e67700 1 -- 172.21.15.156:0/2199397758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 msgr2=0x56127b5fb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.535+0000 7f2c34e67700 1 --2- 172.21.15.156:0/2199397758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 0x56127b5fb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.535+0000 7f2c34e67700 1 -- 172.21.15.156:0/2199397758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5df860 msgr2=0x56127b5b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.535+0000 7f2c34e67700 1 --2- 172.21.15.156:0/2199397758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5df860 0x56127b5b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.535+0000 7f2c34e67700 1 -- 172.21.15.156:0/2199397758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56127b389630 con 0x56127b5dd180 2023-11-28T11:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.535+0000 7f2c35668700 1 --2- 172.21.15.156:0/2199397758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5df860 0x56127b5b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.535+0000 7f2c34e67700 1 --2- 172.21.15.156:0/2199397758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5dd180 0x56127b5dfc30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c005320 tx=0x7f2c2c005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cef8436ade6254c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.535+0000 7f2c12ffd700 1 -- 172.21.15.156:0/2199397758 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c001680 con 0x56127b5dd180 2023-11-28T11:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.535+0000 7f2c12ffd700 1 -- 172.21.15.156:0/2199397758 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2c0017e0 con 0x56127b5dd180 2023-11-28T11:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.535+0000 7f2c12ffd700 1 -- 172.21.15.156:0/2199397758 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c001b50 con 0x56127b5dd180 2023-11-28T11:11:35.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.536+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2199397758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5dd180 msgr2=0x56127b5dfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.536+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/2199397758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5dd180 0x56127b5dfc30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c005320 tx=0x7f2c2c005630 comp rx=0 tx=0).stop 2023-11-28T11:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.536+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2199397758 shutdown_connections 2023-11-28T11:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.536+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/2199397758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 0x56127b5fb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.536+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/2199397758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5df860 0x56127b5b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.536+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/2199397758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5dd180 0x56127b5dfc30 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.536+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2199397758 >> 172.21.15.156:0/2199397758 conn(0x56127b5b2db0 msgr2=0x56127b5b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.536+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2199397758 shutdown_connections 2023-11-28T11:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.536+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2199397758 wait complete. 2023-11-28T11:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.536+0000 7f2c43a55b80 1 -- start start 2023-11-28T11:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.537+0000 7f2c43a55b80 1 Processor -- start 2023-11-28T11:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c43a55b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5dd180 0x56127b5d62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c43a55b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b5d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c43a55b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 0x56127b5aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c43a55b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127b38b920 con 0x56127b5df860 2023-11-28T11:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c43a55b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127b5c4160 con 0x56127b5dd180 2023-11-28T11:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c43a55b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56127b5d7190 con 0x56127b5e0170 2023-11-28T11:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c43a55b80 10 client.0 _ll_register_callbacks cb 0x56127b3ad290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c34e67700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b5d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c34e67700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b5d6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36036/0 (socket says 172.21.15.156:36036) 2023-11-28T11:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c34e67700 1 -- 172.21.15.156:0/2270777287 learned_addr learned my addr 172.21.15.156:0/2270777287 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c35668700 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5dd180 0x56127b5d62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c35e69700 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 0x56127b5aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c34e67700 1 -- 172.21.15.156:0/2270777287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 msgr2=0x56127b5aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:35.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c34e67700 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 0x56127b5aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c34e67700 1 -- 172.21.15.156:0/2270777287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5dd180 msgr2=0x56127b5d62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:35.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c34e67700 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5dd180 0x56127b5d62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.538+0000 7f2c34e67700 1 -- 172.21.15.156:0/2270777287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c300aee40 con 0x56127b5df860 2023-11-28T11:11:35.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c34e67700 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b5d6870 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c003a40 tx=0x7f2c2c001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106c599db12dd286 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:35.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:11:35.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c0bfff700 1 -- 172.21.15.156:0/2270777287 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c006a60 con 0x56127b5df860 2023-11-28T11:11:35.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c0bfff700 1 -- 172.21.15.156:0/2270777287 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f2c2c006bc0 con 0x56127b5df860 2023-11-28T11:11:35.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2270777287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56127b4d5190 con 0x56127b5df860 2023-11-28T11:11:35.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c0bfff700 1 -- 172.21.15.156:0/2270777287 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c2c0078f0 con 0x56127b5df860 2023-11-28T11:11:35.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c43a55b80 10 client.6261 fetch_fsmap learned FSMap version 32 2023-11-28T11:11:35.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2270777287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x56127b335de0 con 0x56127b5df860 2023-11-28T11:11:35.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c0bfff700 1 -- 172.21.15.156:0/2270777287 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56127b4d5190 con 0x56127b5df860 2023-11-28T11:11:35.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c0bfff700 1 -- 172.21.15.156:0/2270777287 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f2c2c005350 con 0x56127b5df860 2023-11-28T11:11:35.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c43a55b80 10 client.6261 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:11:35.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c43a55b80 10 client.6261 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-11-28T11:11:35.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c43a55b80 20 client.6261 populate_metadata read hostname 'smithi156' 2023-11-28T11:11:35.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x56127b5c2e20 0x56127b5c3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c43a55b80 4 client.6261 mds_command: new command op to 6021 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:35.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2270777287 --> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56127b38bf60 con 0x56127b5c2e20 2023-11-28T11:11:35.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.539+0000 7f2c35668700 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x56127b5c2e20 0x56127b5c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.540+0000 7f2c0bfff700 10 client.6261 ms_handle_connect on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:35.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.540+0000 7f2c35668700 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x56127b5c2e20 0x56127b5c3220 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb15944a3f221c31 server_cookie=5edcbb8cdfc8c5a9 in_seq=0 out_seq=0 2023-11-28T11:11:35.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.540+0000 7f2c0bfff700 1 -- 172.21.15.156:0/2270777287 <== mds.0 v2:172.21.15.186:6833/3811791267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x56127b38bf60 con 0x56127b5c2e20 2023-11-28T11:11:35.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.540+0000 7f2c0bfff700 10 client.6261 handle_command_reply: tid=0 2023-11-28T11:11:35.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.540+0000 7f2c43a55b80 1 client.6261 shutdown 2023-11-28T11:11:35.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.541+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2270777287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 msgr2=0x56127b5d6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.541+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b5d6870 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c003a40 tx=0x7f2c2c001310 comp rx=0 tx=0).stop 2023-11-28T11:11:35.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.541+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2270777287 shutdown_connections 2023-11-28T11:11:35.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.541+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x56127b5c2e20 0x56127b5c3220 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.541+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 0x56127b5aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.541+0000 7f2c0bfff700 0 client.6261 ms_handle_reset on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:35.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.541+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5dd180 0x56127b5d62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.541+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/2270777287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b5d6870 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.541+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2270777287 >> 172.21.15.156:0/2270777287 conn(0x56127b5b2db0 msgr2=0x56127b5b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:35.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.541+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2270777287 shutdown_connections 2023-11-28T11:11:35.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.541+0000 7f2c43a55b80 1 -- 172.21.15.156:0/2270777287 wait complete. 2023-11-28T11:11:35.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.541+0000 7f2c43a55b80 20 client.6261 trim_cache size 0 max 16384 2023-11-28T11:11:35.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.557+0000 7f2c43a55b80 1 Processor -- start 2023-11-28T11:11:35.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.557+0000 7f2c43a55b80 1 -- start start 2023-11-28T11:11:35.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.558+0000 7f2c43a55b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5dd180 0x56127b602200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.558+0000 7f2c43a55b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b602740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.558+0000 7f2c43a55b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5e0170 0x56127b5ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.558+0000 7f2c43a55b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127b5ac0b0 con 0x56127b5df860 2023-11-28T11:11:35.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.558+0000 7f2c43a55b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127b5b1f50 con 0x56127b5e0170 2023-11-28T11:11:35.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.558+0000 7f2c43a55b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56127b5fe4b0 con 0x56127b5dd180 2023-11-28T11:11:35.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.558+0000 7f2c34e67700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b602740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.558+0000 7f2c34e67700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b602740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36042/0 (socket says 172.21.15.156:36042) 2023-11-28T11:11:35.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.558+0000 7f2c34e67700 1 -- 172.21.15.156:0/1949391276 learned_addr learned my addr 172.21.15.156:0/1949391276 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:35.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.558+0000 7f2c35668700 1 --2- 172.21.15.156:0/1949391276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5dd180 0x56127b602200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.558+0000 7f2c35e69700 1 --2- 172.21.15.156:0/1949391276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5e0170 0x56127b5ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c34e67700 1 -- 172.21.15.156:0/1949391276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5dd180 msgr2=0x56127b602200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:35.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c34e67700 1 --2- 172.21.15.156:0/1949391276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5dd180 0x56127b602200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c34e67700 1 -- 172.21.15.156:0/1949391276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5e0170 msgr2=0x56127b5ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:35.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c34e67700 1 --2- 172.21.15.156:0/1949391276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5e0170 0x56127b5ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c34e67700 1 -- 172.21.15.156:0/1949391276 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3015e570 con 0x56127b5df860 2023-11-28T11:11:35.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c34e67700 1 --2- 172.21.15.156:0/1949391276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b602740 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c000c00 tx=0x7f2c2c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4324295aed8771 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:35.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c137fe700 1 -- 172.21.15.156:0/1949391276 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c001c90 con 0x56127b5df860 2023-11-28T11:11:35.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c137fe700 1 -- 172.21.15.156:0/1949391276 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2c006fb0 con 0x56127b5df860 2023-11-28T11:11:35.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c137fe700 1 -- 172.21.15.156:0/1949391276 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c006aa0 con 0x56127b5df860 2023-11-28T11:11:35.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c43a55b80 1 -- 172.21.15.156:0/1949391276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 msgr2=0x56127b602740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/1949391276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b602740 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c000c00 tx=0x7f2c2c01f000 comp rx=0 tx=0).stop 2023-11-28T11:11:35.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c43a55b80 1 -- 172.21.15.156:0/1949391276 shutdown_connections 2023-11-28T11:11:35.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/1949391276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5e0170 0x56127b5ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/1949391276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5dd180 0x56127b602200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/1949391276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b602740 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c43a55b80 1 -- 172.21.15.156:0/1949391276 >> 172.21.15.156:0/1949391276 conn(0x56127b5b2db0 msgr2=0x56127b5d60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:35.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c43a55b80 1 -- 172.21.15.156:0/1949391276 shutdown_connections 2023-11-28T11:11:35.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.559+0000 7f2c43a55b80 1 -- 172.21.15.156:0/1949391276 wait complete. 2023-11-28T11:11:35.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.560+0000 7f2c43a55b80 1 -- start start 2023-11-28T11:11:35.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.560+0000 7f2c43a55b80 1 Processor -- start 2023-11-28T11:11:35.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c43a55b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5dd180 0x56127b5f0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c43a55b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b5e2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c43a55b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 0x56127b6021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c43a55b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127b5f0af0 con 0x56127b5df860 2023-11-28T11:11:35.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c43a55b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127b5ae530 con 0x56127b5dd180 2023-11-28T11:11:35.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c43a55b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56127b5ae780 con 0x56127b5e0170 2023-11-28T11:11:35.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c43a55b80 10 client.0 _ll_register_callbacks cb 0x56127b3ad290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:35.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c34e67700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b5e2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c34e67700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b5e2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36048/0 (socket says 172.21.15.156:36048) 2023-11-28T11:11:35.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c34e67700 1 -- 172.21.15.156:0/3548791926 learned_addr learned my addr 172.21.15.156:0/3548791926 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:35.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c35668700 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5dd180 0x56127b5f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c35e69700 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 0x56127b6021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c34e67700 1 -- 172.21.15.156:0/3548791926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 msgr2=0x56127b6021d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:35.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c34e67700 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 0x56127b6021d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c34e67700 1 -- 172.21.15.156:0/3548791926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5dd180 msgr2=0x56127b5f0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:35.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c34e67700 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5dd180 0x56127b5f0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c34e67700 1 -- 172.21.15.156:0/3548791926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56127b335de0 con 0x56127b5df860 2023-11-28T11:11:35.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c34e67700 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b5e2ff0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c000fb0 tx=0x7f2c2c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460a985dab0931f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:35.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.561+0000 7f2c0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:11:35.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c0bfff700 1 -- 172.21.15.156:0/3548791926 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c006a60 con 0x56127b5df860 2023-11-28T11:11:35.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c0bfff700 1 -- 172.21.15.156:0/3548791926 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f2c2c006bc0 con 0x56127b5df860 2023-11-28T11:11:35.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c43a55b80 1 -- 172.21.15.156:0/3548791926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56127b4c51b0 con 0x56127b5df860 2023-11-28T11:11:35.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c0bfff700 1 -- 172.21.15.156:0/3548791926 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c2c0071d0 con 0x56127b5df860 2023-11-28T11:11:35.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c43a55b80 10 client.6267 fetch_fsmap learned FSMap version 32 2023-11-28T11:11:35.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c43a55b80 1 -- 172.21.15.156:0/3548791926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x56127b31ec60 con 0x56127b5df860 2023-11-28T11:11:35.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c0bfff700 1 -- 172.21.15.156:0/3548791926 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56127b4c51b0 con 0x56127b5df860 2023-11-28T11:11:35.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c0bfff700 1 -- 172.21.15.156:0/3548791926 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f2c2c007330 con 0x56127b5df860 2023-11-28T11:11:35.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c43a55b80 10 client.6267 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:11:35.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c43a55b80 10 client.6267 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-11-28T11:11:35.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c43a55b80 20 client.6267 populate_metadata read hostname 'smithi156' 2023-11-28T11:11:35.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x56127b5b3180 0x56127b602710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c43a55b80 4 client.6267 mds_command: new command op to 6021 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:11:35.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c43a55b80 1 -- 172.21.15.156:0/3548791926 --> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56127b4af1c0 con 0x56127b5b3180 2023-11-28T11:11:35.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.562+0000 7f2c35668700 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x56127b5b3180 0x56127b602710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c0bfff700 10 client.6267 ms_handle_connect on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:35.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c35668700 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x56127b5b3180 0x56127b602710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d1a7456b33a725 server_cookie=7205bc8ce52664f6 in_seq=0 out_seq=0 2023-11-28T11:11:35.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c0bfff700 1 -- 172.21.15.156:0/3548791926 <== mds.0 v2:172.21.15.186:6833/3811791267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56127b4af1c0 con 0x56127b5b3180 2023-11-28T11:11:35.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c0bfff700 10 client.6267 handle_command_reply: tid=0 2023-11-28T11:11:35.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c43a55b80 1 client.6267 shutdown 2023-11-28T11:11:35.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c43a55b80 1 -- 172.21.15.156:0/3548791926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 msgr2=0x56127b5e2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b5e2ff0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c000fb0 tx=0x7f2c2c009040 comp rx=0 tx=0).stop 2023-11-28T11:11:35.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c43a55b80 1 -- 172.21.15.156:0/3548791926 shutdown_connections 2023-11-28T11:11:35.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x56127b5b3180 0x56127b602710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56127b5e0170 0x56127b6021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c0bfff700 0 client.6267 ms_handle_reset on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:35.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56127b5dd180 0x56127b5f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c43a55b80 1 --2- 172.21.15.156:0/3548791926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56127b5df860 0x56127b5e2ff0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c43a55b80 1 -- 172.21.15.156:0/3548791926 >> 172.21.15.156:0/3548791926 conn(0x56127b5b2db0 msgr2=0x56127b5d60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:35.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c43a55b80 1 -- 172.21.15.156:0/3548791926 shutdown_connections 2023-11-28T11:11:35.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c43a55b80 1 -- 172.21.15.156:0/3548791926 wait complete. 2023-11-28T11:11:35.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.563+0000 7f2c43a55b80 20 client.6267 trim_cache size 0 max 16384 2023-11-28T11:11:35.581 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:11:35.581 INFO:teuthology.orchestra.run.smithi156.stdout: "status": "no active scrubs running", 2023-11-28T11:11:35.581 INFO:teuthology.orchestra.run.smithi156.stdout: "scrubs": {} 2023-11-28T11:11:35.581 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:11:35.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.564+0000 7f2c378cc700 1 -- 172.21.15.156:0/1783915453 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2c140682f0 msgr2=0x7f2c1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.564+0000 7f2c378cc700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2c140682f0 0x7f2c1406a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2c200090b0 tx=0x7f2c20008030 comp rx=0 tx=0).stop 2023-11-28T11:11:35.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.564+0000 7f2c378cc700 1 -- 172.21.15.156:0/1783915453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c300b0360 msgr2=0x7f2c300b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.564+0000 7f2c378cc700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c300b0360 0x7f2c300b0800 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2c3006dbd0 tx=0x7f2c28007c90 comp rx=0 tx=0).stop 2023-11-28T11:11:35.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.564+0000 7f2c378cc700 1 -- 172.21.15.156:0/1783915453 shutdown_connections 2023-11-28T11:11:35.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.564+0000 7f2c378cc700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c300b0360 0x7f2c300b0800 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.564+0000 7f2c378cc700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c1c003bc0 0x7f2c3014ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.564+0000 7f2c378cc700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2c140682f0 0x7f2c1406a7a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.564+0000 7f2c378cc700 1 --2- 172.21.15.156:0/1783915453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c300b36b0 0x7f2c300ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.564+0000 7f2c378cc700 1 -- 172.21.15.156:0/1783915453 >> 172.21.15.156:0/1783915453 conn(0x7f2c300b12b0 msgr2=0x7f2c300af220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:35.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.564+0000 7f2c378cc700 1 -- 172.21.15.156:0/1783915453 shutdown_connections 2023-11-28T11:11:35.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.564+0000 7f2c378cc700 1 -- 172.21.15.156:0/1783915453 wait complete. 2023-11-28T11:11:35.584 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T11:11:35.584 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:35.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.703+0000 7f2066236700 1 Processor -- start 2023-11-28T11:11:35.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.703+0000 7f2066236700 1 -- start start 2023-11-28T11:11:35.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f2066236700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f206006dee0 0x7f206006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f2066236700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2060160540 0x7f20601609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f2066236700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206015d060 con 0x7f2060071040 2023-11-28T11:11:35.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f2066236700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206015d340 con 0x7f206006dee0 2023-11-28T11:11:35.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f2066236700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206015d620 con 0x7f2060160540 2023-11-28T11:11:35.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f205effd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f206006dee0 0x7f206006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f205ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2060160540 0x7f20601609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f205f7fe700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f2060071040 0x7f2060071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:33060/0 (socket says 172.21.15.156:33060) 2023-11-28T11:11:35.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f205effd700 1 --2- 172.21.15.156:0/1436648633 >> v2:172.21.15.186:3300/0 conn(0x7f206006dee0 0x7f206006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:47070/0 (socket says 172.21.15.156:47070) 2023-11-28T11:11:35.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f205f7fe700 1 -- 172.21.15.156:0/1436648633 learned_addr learned my addr 172.21.15.156:0/1436648633 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:35.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f205ffff700 1 -- 172.21.15.156:0/1436648633 >> v2:172.21.15.186:3300/0 conn(0x7f206006dee0 msgr2=0x7f206006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:35.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f205ffff700 1 --2- 172.21.15.156:0/1436648633 >> v2:172.21.15.186:3300/0 conn(0x7f206006dee0 0x7f206006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.704+0000 7f205ffff700 1 -- 172.21.15.156:0/1436648633 >> v1:172.21.15.186:6789/0 conn(0x7f2060071040 legacy=0x7f2060071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.705+0000 7f205ffff700 1 -- 172.21.15.156:0/1436648633 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206015d900 con 0x7f2060160540 2023-11-28T11:11:35.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.705+0000 7f205ffff700 1 --2- 172.21.15.156:0/1436648633 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2060160540 0x7f20601609c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2058009d90 tx=0x7f205800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b4c089fcc65c27f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:35.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.705+0000 7f205e7fc700 1 -- 172.21.15.156:0/1436648633 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205800e070 con 0x7f2060160540 2023-11-28T11:11:35.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.705+0000 7f205e7fc700 1 -- 172.21.15.156:0/1436648633 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205800bb20 con 0x7f2060160540 2023-11-28T11:11:35.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.705+0000 7f205e7fc700 1 -- 172.21.15.156:0/1436648633 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058012430 con 0x7f2060160540 2023-11-28T11:11:35.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.705+0000 7f2066236700 1 -- 172.21.15.156:0/1436648633 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2060160540 msgr2=0x7f20601609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.705+0000 7f2066236700 1 --2- 172.21.15.156:0/1436648633 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2060160540 0x7f20601609c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2058009d90 tx=0x7f205800b330 comp rx=0 tx=0).stop 2023-11-28T11:11:35.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.705+0000 7f2066236700 1 -- 172.21.15.156:0/1436648633 shutdown_connections 2023-11-28T11:11:35.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.705+0000 7f2066236700 1 --2- 172.21.15.156:0/1436648633 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2060160540 0x7f20601609c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.705+0000 7f2066236700 1 --2- 172.21.15.156:0/1436648633 >> v2:172.21.15.186:3300/0 conn(0x7f206006dee0 0x7f206006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.705+0000 7f2066236700 1 -- 172.21.15.156:0/1436648633 >> 172.21.15.156:0/1436648633 conn(0x7f20600d0000 msgr2=0x7f20600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:35.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.705+0000 7f2066236700 1 -- 172.21.15.156:0/1436648633 shutdown_connections 2023-11-28T11:11:35.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.706+0000 7f2066236700 1 -- 172.21.15.156:0/1436648633 wait complete. 2023-11-28T11:11:35.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.707+0000 7f2066236700 1 Processor -- start 2023-11-28T11:11:35.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.707+0000 7f2066236700 1 -- start start 2023-11-28T11:11:35.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f2066236700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f206006dee0 0x7f20600c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f2066236700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2060071040 0x7f20601f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f2066236700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2060160540 0x7f20601f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f2066236700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20600c68f0 con 0x7f2060160540 2023-11-28T11:11:35.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f2066236700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20600c4990 con 0x7f206006dee0 2023-11-28T11:11:35.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f2066236700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20600c3aa0 con 0x7f2060071040 2023-11-28T11:11:35.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f205ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2060160540 0x7f20601f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f205ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2060160540 0x7f20601f6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36064/0 (socket says 172.21.15.156:36064) 2023-11-28T11:11:35.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f205ffff700 1 -- 172.21.15.156:0/898874692 learned_addr learned my addr 172.21.15.156:0/898874692 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:35.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f205f7fe700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f206006dee0 0x7f20600c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f205effd700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2060071040 0x7f20601f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f205ffff700 1 -- 172.21.15.156:0/898874692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2060071040 msgr2=0x7f20601f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f205ffff700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2060071040 0x7f20601f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f205ffff700 1 -- 172.21.15.156:0/898874692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f206006dee0 msgr2=0x7f20600c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f205ffff700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f206006dee0 0x7f20600c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f205ffff700 1 -- 172.21.15.156:0/898874692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2058009a70 con 0x7f2060160540 2023-11-28T11:11:35.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f205f7fe700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f206006dee0 0x7f20600c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:35.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f205ffff700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2060160540 0x7f20601f6710 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2058014040 tx=0x7f2058004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:35.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f2047fff700 1 -- 172.21.15.156:0/898874692 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058003ea0 con 0x7f2060160540 2023-11-28T11:11:35.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f2047fff700 1 -- 172.21.15.156:0/898874692 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2058020070 con 0x7f2060160540 2023-11-28T11:11:35.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f2066236700 1 -- 172.21.15.156:0/898874692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206015ccd0 con 0x7f2060160540 2023-11-28T11:11:35.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f2047fff700 1 -- 172.21.15.156:0/898874692 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20580276c0 con 0x7f2060160540 2023-11-28T11:11:35.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.708+0000 7f2066236700 1 -- 172.21.15.156:0/898874692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206006e870 con 0x7f2060160540 2023-11-28T11:11:35.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.709+0000 7f2066236700 1 -- 172.21.15.156:0/898874692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206006a270 con 0x7f2060160540 2023-11-28T11:11:35.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.710+0000 7f2047fff700 1 -- 172.21.15.156:0/898874692 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f205802e070 con 0x7f2060160540 2023-11-28T11:11:35.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.710+0000 7f2047fff700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f204c0682f0 0x7f204c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:35.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.710+0000 7f2047fff700 1 -- 172.21.15.156:0/898874692 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f2058022070 con 0x7f2060160540 2023-11-28T11:11:35.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.710+0000 7f205f7fe700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f204c0682f0 0x7f204c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:35.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.711+0000 7f205f7fe700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f204c0682f0 0x7f204c06a7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2054007fd0 tx=0x7f2054009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:35.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.715+0000 7f2047fff700 1 -- 172.21.15.156:0/898874692 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f20580541c0 con 0x7f2060160540 2023-11-28T11:11:35.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.940+0000 7f2066236700 1 -- 172.21.15.156:0/898874692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20600720a0 con 0x7f2060160540 2023-11-28T11:11:35.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.941+0000 7f2047fff700 1 -- 172.21.15.156:0/898874692 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f205802c090 con 0x7f2060160540 2023-11-28T11:11:35.942 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:35.942 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6006,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1991065152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1991065152},{"type":"v1","addr":"172.21.15.156:6836","nonce":1991065152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/43543030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":43543030},{"type":"v1","addr":"172.21.15.156:6837","nonce":43543030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/2480189069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2480189069},{"type":"v1","addr":"172.21.15.186:6834","nonce":2480189069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:52.367543+0000","modified":"2023-11-28T11:11:08.808905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6021":{"gid":6021,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6835/3811791267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3811791267},{"type":"v1","addr":"172.21.15.186:6835","nonce":3811791267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:11:35.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.943+0000 7f2066236700 1 -- 172.21.15.156:0/898874692 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f204c0682f0 msgr2=0x7f204c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.943+0000 7f2066236700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f204c0682f0 0x7f204c06a7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2054007fd0 tx=0x7f2054009040 comp rx=0 tx=0).stop 2023-11-28T11:11:35.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.943+0000 7f2066236700 1 -- 172.21.15.156:0/898874692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2060160540 msgr2=0x7f20601f6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:35.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.943+0000 7f2066236700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2060160540 0x7f20601f6710 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2058014040 tx=0x7f2058004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.943+0000 7f2066236700 1 -- 172.21.15.156:0/898874692 shutdown_connections 2023-11-28T11:11:35.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.943+0000 7f2066236700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2060160540 0x7f20601f6710 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.943+0000 7f2066236700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f206006dee0 0x7f20600c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.943+0000 7f2066236700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f204c0682f0 0x7f204c06a7a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.943+0000 7f2066236700 1 --2- 172.21.15.156:0/898874692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2060071040 0x7f20601f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:35.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.943+0000 7f2066236700 1 -- 172.21.15.156:0/898874692 >> 172.21.15.156:0/898874692 conn(0x7f20600d0000 msgr2=0x7f206015e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:35.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.943+0000 7f2066236700 1 -- 172.21.15.156:0/898874692 shutdown_connections 2023-11-28T11:11:35.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:35.943+0000 7f2066236700 1 -- 172.21.15.156:0/898874692 wait complete. 2023-11-28T11:11:35.946 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 32 2023-11-28T11:11:35.956 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2023-11-28T11:11:36.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.077+0000 7f90c8153700 1 Processor -- start 2023-11-28T11:11:36.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.078+0000 7f90c8153700 1 -- start start 2023-11-28T11:11:36.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.078+0000 7f90c8153700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f90c0071040 0x7f90c0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.078+0000 7f90c8153700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c0071950 con 0x7f90c006dee0 2023-11-28T11:11:36.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.078+0000 7f90c8153700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c015d060 con 0x7f90c0160540 2023-11-28T11:11:36.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.078+0000 7f90c8153700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c015d340 con 0x7f90c0071040 2023-11-28T11:11:36.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.079+0000 7f90c56ee700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f90c006dee0 0x7f90c006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50948/0 (socket says 172.21.15.156:50948) 2023-11-28T11:11:36.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.079+0000 7f90c56ee700 1 -- 172.21.15.156:0/2546123517 learned_addr learned my addr 172.21.15.156:0/2546123517 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:36.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.079+0000 7f90c5eef700 1 --2- 172.21.15.156:0/2546123517 >> v2:172.21.15.186:3300/0 conn(0x7f90c0071040 0x7f90c0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.079+0000 7f90c4eed700 1 -- 172.21.15.156:0/2546123517 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523130310 0 0) 0x7f90c0071950 con 0x7f90c006dee0 2023-11-28T11:11:36.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.079+0000 7f90c4eed700 1 -- 172.21.15.156:0/2546123517 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90a80035b0 con 0x7f90c006dee0 2023-11-28T11:11:36.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.079+0000 7f90c5eef700 1 -- 172.21.15.156:0/2546123517 >> v1:172.21.15.186:6789/0 conn(0x7f90c0160540 legacy=0x7f90c0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.079+0000 7f90c5eef700 1 -- 172.21.15.156:0/2546123517 >> v1:172.21.15.156:6789/0 conn(0x7f90c006dee0 legacy=0x7f90c006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.079+0000 7f90c5eef700 1 -- 172.21.15.156:0/2546123517 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c015d620 con 0x7f90c0071040 2023-11-28T11:11:36.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.079+0000 7f90c5eef700 1 --2- 172.21.15.156:0/2546123517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0071040 0x7f90c0071410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009d90 tx=0x7f90b800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10da3c4a0bef9e54 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.079+0000 7f90c4eed700 1 -- 172.21.15.156:0/2546123517 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b800e070 con 0x7f90c0071040 2023-11-28T11:11:36.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.079+0000 7f90c4eed700 1 -- 172.21.15.156:0/2546123517 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b800bb20 con 0x7f90c0071040 2023-11-28T11:11:36.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.079+0000 7f90c4eed700 1 -- 172.21.15.156:0/2546123517 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b800be90 con 0x7f90c0071040 2023-11-28T11:11:36.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.080+0000 7f90c8153700 1 -- 172.21.15.156:0/2546123517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0071040 msgr2=0x7f90c0071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.080+0000 7f90c8153700 1 --2- 172.21.15.156:0/2546123517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0071040 0x7f90c0071410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009d90 tx=0x7f90b800b330 comp rx=0 tx=0).stop 2023-11-28T11:11:36.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.080+0000 7f90c8153700 1 -- 172.21.15.156:0/2546123517 shutdown_connections 2023-11-28T11:11:36.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.080+0000 7f90c8153700 1 --2- 172.21.15.156:0/2546123517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0071040 0x7f90c0071410 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.080+0000 7f90c8153700 1 -- 172.21.15.156:0/2546123517 >> 172.21.15.156:0/2546123517 conn(0x7f90c00d0000 msgr2=0x7f90c00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:36.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.080+0000 7f90c8153700 1 -- 172.21.15.156:0/2546123517 shutdown_connections 2023-11-28T11:11:36.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.080+0000 7f90c8153700 1 -- 172.21.15.156:0/2546123517 wait complete. 2023-11-28T11:11:36.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.081+0000 7f90c8153700 1 Processor -- start 2023-11-28T11:11:36.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.081+0000 7f90c8153700 1 -- start start 2023-11-28T11:11:36.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c8153700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90c006dee0 0x7f90c015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c8153700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c0071040 0x7f90c01f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c8153700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0160540 0x7f90c00b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c8153700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c01f6ab0 con 0x7f90c0071040 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c8153700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c00b3840 con 0x7f90c0160540 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c8153700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c00b3260 con 0x7f90c006dee0 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c56ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c0071040 0x7f90c01f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c56ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c0071040 0x7f90c01f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36076/0 (socket says 172.21.15.156:36076) 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c56ee700 1 -- 172.21.15.156:0/1884425980 learned_addr learned my addr 172.21.15.156:0/1884425980 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c5eef700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90c006dee0 0x7f90c015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c66f0700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0160540 0x7f90c00b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c56ee700 1 -- 172.21.15.156:0/1884425980 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90c006dee0 msgr2=0x7f90c015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c56ee700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90c006dee0 0x7f90c015b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c56ee700 1 -- 172.21.15.156:0/1884425980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0160540 msgr2=0x7f90c00b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c56ee700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0160540 0x7f90c00b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c56ee700 1 -- 172.21.15.156:0/1884425980 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b8009a70 con 0x7f90c0071040 2023-11-28T11:11:36.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.082+0000 7f90c66f0700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0160540 0x7f90c00b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:36.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.083+0000 7f90c56ee700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c0071040 0x7f90c01f5b30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f90b400c4d0 tx=0x7f90b400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.083+0000 7f90be7fc700 1 -- 172.21.15.156:0/1884425980 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4005d70 con 0x7f90c0071040 2023-11-28T11:11:36.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.083+0000 7f90be7fc700 1 -- 172.21.15.156:0/1884425980 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b40070d0 con 0x7f90c0071040 2023-11-28T11:11:36.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.083+0000 7f90c8153700 1 -- 172.21.15.156:0/1884425980 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c015ccd0 con 0x7f90c0071040 2023-11-28T11:11:36.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.083+0000 7f90be7fc700 1 -- 172.21.15.156:0/1884425980 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4007460 con 0x7f90c0071040 2023-11-28T11:11:36.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.083+0000 7f90c8153700 1 -- 172.21.15.156:0/1884425980 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c015d900 con 0x7f90c0071040 2023-11-28T11:11:36.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.084+0000 7f90be7fc700 1 -- 172.21.15.156:0/1884425980 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f90b4019030 con 0x7f90c0071040 2023-11-28T11:11:36.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.084+0000 7f90be7fc700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90ac0682f0 0x7f90ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.085+0000 7f90be7fc700 1 -- 172.21.15.156:0/1884425980 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f90b4085060 con 0x7f90c0071040 2023-11-28T11:11:36.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.085+0000 7f90c5eef700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90ac0682f0 0x7f90ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.085+0000 7f90c5eef700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90ac0682f0 0x7f90ac06a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f90b800b370 tx=0x7f90b8011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.087+0000 7f90d42dcb80 1 Processor -- start 2023-11-28T11:11:36.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.087+0000 7f90d42dcb80 1 -- start start 2023-11-28T11:11:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.087+0000 7f90d42dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 0x55ebfb2feac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.087+0000 7f90d42dcb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 0x55ebfb328c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90d42dcb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3291c0 0x55ebfb344810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90d42dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb1e8c60 con 0x55ebfb3288b0 2023-11-28T11:11:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90d42dcb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb2573f0 con 0x55ebfb3261d0 2023-11-28T11:11:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90d42dcb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb250920 con 0x55ebfb3291c0 2023-11-28T11:11:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90c5eef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 0x55ebfb2feac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90c5eef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 0x55ebfb2feac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36082/0 (socket says 172.21.15.156:36082) 2023-11-28T11:11:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90c5eef700 1 -- 172.21.15.156:0/2619129397 learned_addr learned my addr 172.21.15.156:0/2619129397 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90c56ee700 1 --2- 172.21.15.156:0/2619129397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 0x55ebfb328c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90c66f0700 1 --2- 172.21.15.156:0/2619129397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3291c0 0x55ebfb344810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90c5eef700 1 -- 172.21.15.156:0/2619129397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3291c0 msgr2=0x55ebfb344810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90c5eef700 1 --2- 172.21.15.156:0/2619129397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3291c0 0x55ebfb344810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90c5eef700 1 -- 172.21.15.156:0/2619129397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 msgr2=0x55ebfb328c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90c5eef700 1 --2- 172.21.15.156:0/2619129397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 0x55ebfb328c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90c5eef700 1 -- 172.21.15.156:0/2619129397 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebfb0d2680 con 0x55ebfb3288b0 2023-11-28T11:11:36.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90c66f0700 1 --2- 172.21.15.156:0/2619129397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3291c0 0x55ebfb344810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:36.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90c5eef700 1 --2- 172.21.15.156:0/2619129397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 0x55ebfb2feac0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009d60 tx=0x7f90b8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87599a0479b9292e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90a77fe700 1 -- 172.21.15.156:0/2619129397 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8033680 con 0x55ebfb3288b0 2023-11-28T11:11:36.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90a77fe700 1 -- 172.21.15.156:0/2619129397 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b802c070 con 0x55ebfb3288b0 2023-11-28T11:11:36.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.088+0000 7f90a77fe700 1 -- 172.21.15.156:0/2619129397 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8033a20 con 0x55ebfb3288b0 2023-11-28T11:11:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.089+0000 7f90d42dcb80 1 -- 172.21.15.156:0/2619129397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 msgr2=0x55ebfb2feac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.089+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/2619129397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 0x55ebfb2feac0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009d60 tx=0x7f90b8010000 comp rx=0 tx=0).stop 2023-11-28T11:11:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.089+0000 7f90d42dcb80 1 -- 172.21.15.156:0/2619129397 shutdown_connections 2023-11-28T11:11:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.089+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/2619129397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3291c0 0x55ebfb344810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.089+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/2619129397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 0x55ebfb2feac0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.089+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/2619129397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 0x55ebfb328c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.089+0000 7f90d42dcb80 1 -- 172.21.15.156:0/2619129397 >> 172.21.15.156:0/2619129397 conn(0x55ebfb2fbe00 msgr2=0x55ebfb2fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:36.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.089+0000 7f90d42dcb80 1 -- 172.21.15.156:0/2619129397 shutdown_connections 2023-11-28T11:11:36.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.089+0000 7f90d42dcb80 1 -- 172.21.15.156:0/2619129397 wait complete. 2023-11-28T11:11:36.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.090+0000 7f90d42dcb80 1 -- start start 2023-11-28T11:11:36.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.090+0000 7f90d42dcb80 1 Processor -- start 2023-11-28T11:11:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90d42dcb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3261d0 0x55ebfb31af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90d42dcb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3288b0 0x55ebfb33d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90d42dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 0x55ebfb326680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90d42dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb0d4970 con 0x55ebfb3291c0 2023-11-28T11:11:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90d42dcb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb327290 con 0x55ebfb3288b0 2023-11-28T11:11:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90d42dcb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb31b470 con 0x55ebfb3261d0 2023-11-28T11:11:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90d42dcb80 10 client.0 _ll_register_callbacks cb 0x55ebfb0f62e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90c66f0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 0x55ebfb326680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90c66f0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 0x55ebfb326680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36092/0 (socket says 172.21.15.156:36092) 2023-11-28T11:11:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90c66f0700 1 -- 172.21.15.156:0/1243413552 learned_addr learned my addr 172.21.15.156:0/1243413552 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90c56ee700 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3288b0 0x55ebfb33d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90c5eef700 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3261d0 0x55ebfb31af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90c66f0700 1 -- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3261d0 msgr2=0x55ebfb31af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90c66f0700 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3261d0 0x55ebfb31af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90c66f0700 1 -- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3288b0 msgr2=0x55ebfb33d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90c66f0700 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3288b0 0x55ebfb33d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90c66f0700 1 -- 172.21.15.156:0/1243413552 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b8023070 con 0x55ebfb3291c0 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90c5eef700 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3261d0 0x55ebfb31af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90c66f0700 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 0x55ebfb326680 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f90b0003fb0 tx=0x7f90b0001460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc9ae6a1c8f0392 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90a4ff9700 1 -- 172.21.15.156:0/1243413552 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0001c80 con 0x55ebfb3291c0 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90a4ff9700 1 -- 172.21.15.156:0/1243413552 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f90b0015460 con 0x55ebfb3291c0 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1243413552 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebfb21e1e0 con 0x55ebfb3291c0 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90a4ff9700 1 -- 172.21.15.156:0/1243413552 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90b0015620 con 0x55ebfb3291c0 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.091+0000 7f90d42dcb80 10 client.6282 fetch_fsmap learned FSMap version 32 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.092+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1243413552 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55ebfb07ee30 con 0x55ebfb3291c0 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.092+0000 7f90a4ff9700 1 -- 172.21.15.156:0/1243413552 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ebfb21e1e0 con 0x55ebfb3291c0 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.092+0000 7f90a4ff9700 1 -- 172.21.15.156:0/1243413552 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f90b0015780 con 0x55ebfb3291c0 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.092+0000 7f90d42dcb80 10 client.6282 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:11:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.092+0000 7f90d42dcb80 10 client.6282 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:11:36.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.092+0000 7f90d42dcb80 20 client.6282 populate_metadata read hostname 'smithi156' 2023-11-28T11:11:36.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.092+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55ebfb326bc0 0x55ebfb33ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.092+0000 7f90d42dcb80 4 client.6282 mds_command: new command op to 6021 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:36.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.092+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1243413552 --> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebfb0d4fb0 con 0x55ebfb326bc0 2023-11-28T11:11:36.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.092+0000 7f90c5eef700 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55ebfb326bc0 0x55ebfb33ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.092+0000 7f90a4ff9700 10 client.6282 ms_handle_connect on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:36.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.092+0000 7f90c5eef700 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55ebfb326bc0 0x55ebfb33ad60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=306e5d2d7995eca1 server_cookie=3249ab3abbeb66f0 in_seq=0 out_seq=0 2023-11-28T11:11:36.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90a4ff9700 1 -- 172.21.15.156:0/1243413552 <== mds.0 v2:172.21.15.186:6833/3811791267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55ebfb0d4fb0 con 0x55ebfb326bc0 2023-11-28T11:11:36.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90a4ff9700 10 client.6282 handle_command_reply: tid=0 2023-11-28T11:11:36.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90d42dcb80 1 client.6282 shutdown 2023-11-28T11:11:36.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1243413552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 msgr2=0x55ebfb326680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 0x55ebfb326680 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f90b0003fb0 tx=0x7f90b0001460 comp rx=0 tx=0).stop 2023-11-28T11:11:36.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1243413552 shutdown_connections 2023-11-28T11:11:36.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55ebfb326bc0 0x55ebfb33ad60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 0x55ebfb326680 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3261d0 0x55ebfb31af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90a4ff9700 0 client.6282 ms_handle_reset on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:36.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/1243413552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3288b0 0x55ebfb33d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1243413552 >> 172.21.15.156:0/1243413552 conn(0x55ebfb2fbe00 msgr2=0x55ebfb2fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:36.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1243413552 shutdown_connections 2023-11-28T11:11:36.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.093+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1243413552 wait complete. 2023-11-28T11:11:36.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.094+0000 7f90d42dcb80 20 client.6282 trim_cache size 0 max 16384 2023-11-28T11:11:36.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.106+0000 7f90d42dcb80 1 Processor -- start 2023-11-28T11:11:36.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.106+0000 7f90d42dcb80 1 -- start start 2023-11-28T11:11:36.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90d42dcb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 0x55ebfb346e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90d42dcb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3288b0 0x55ebfb33d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90d42dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 0x55ebfb3195a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90d42dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb2f7f90 con 0x55ebfb3291c0 2023-11-28T11:11:36.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90d42dcb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb316be0 con 0x55ebfb3261d0 2023-11-28T11:11:36.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90d42dcb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb326a10 con 0x55ebfb3288b0 2023-11-28T11:11:36.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90c66f0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 0x55ebfb3195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90c66f0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 0x55ebfb3195a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36100/0 (socket says 172.21.15.156:36100) 2023-11-28T11:11:36.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90c66f0700 1 -- 172.21.15.156:0/2676947609 learned_addr learned my addr 172.21.15.156:0/2676947609 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:36.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90c5eef700 1 --2- 172.21.15.156:0/2676947609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 0x55ebfb346e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90c56ee700 1 --2- 172.21.15.156:0/2676947609 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3288b0 0x55ebfb33d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90c66f0700 1 -- 172.21.15.156:0/2676947609 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3288b0 msgr2=0x55ebfb33d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:36.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90c66f0700 1 --2- 172.21.15.156:0/2676947609 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3288b0 0x55ebfb33d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90c66f0700 1 -- 172.21.15.156:0/2676947609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 msgr2=0x55ebfb346e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:36.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90c66f0700 1 --2- 172.21.15.156:0/2676947609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 0x55ebfb346e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90c66f0700 1 -- 172.21.15.156:0/2676947609 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebfb2171e0 con 0x55ebfb3291c0 2023-11-28T11:11:36.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90c66f0700 1 --2- 172.21.15.156:0/2676947609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 0x55ebfb3195a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f90b001e040 tx=0x7f90b0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3461f0f142d6299 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90a7fff700 1 -- 172.21.15.156:0/2676947609 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0005150 con 0x55ebfb3291c0 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.107+0000 7f90a7fff700 1 -- 172.21.15.156:0/2676947609 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b00143c0 con 0x55ebfb3291c0 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.108+0000 7f90a7fff700 1 -- 172.21.15.156:0/2676947609 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00052b0 con 0x55ebfb3291c0 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.108+0000 7f90d42dcb80 1 -- 172.21.15.156:0/2676947609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 msgr2=0x55ebfb3195a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.108+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/2676947609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 0x55ebfb3195a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f90b001e040 tx=0x7f90b0010000 comp rx=0 tx=0).stop 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.108+0000 7f90d42dcb80 1 -- 172.21.15.156:0/2676947609 shutdown_connections 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.108+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/2676947609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3291c0 0x55ebfb3195a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.108+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/2676947609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 0x55ebfb346e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.108+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/2676947609 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3288b0 0x55ebfb33d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.108+0000 7f90d42dcb80 1 -- 172.21.15.156:0/2676947609 >> 172.21.15.156:0/2676947609 conn(0x55ebfb2fbe00 msgr2=0x55ebfb316340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.108+0000 7f90d42dcb80 1 -- 172.21.15.156:0/2676947609 shutdown_connections 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.108+0000 7f90d42dcb80 1 -- 172.21.15.156:0/2676947609 wait complete. 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.109+0000 7f90d42dcb80 1 -- start start 2023-11-28T11:11:36.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.109+0000 7f90d42dcb80 1 Processor -- start 2023-11-28T11:11:36.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.109+0000 7f90d42dcb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 0x55ebfb3a80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.109+0000 7f90d42dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 0x55ebfb345cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.109+0000 7f90d42dcb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3291c0 0x55ebfb2fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.109+0000 7f90d42dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb324fa0 con 0x55ebfb3288b0 2023-11-28T11:11:36.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.109+0000 7f90d42dcb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb346310 con 0x55ebfb3261d0 2023-11-28T11:11:36.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.109+0000 7f90d42dcb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebfb346560 con 0x55ebfb3291c0 2023-11-28T11:11:36.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90d42dcb80 10 client.0 _ll_register_callbacks cb 0x55ebfb0f62e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:36.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90c56ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 0x55ebfb345cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90c56ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 0x55ebfb345cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36104/0 (socket says 172.21.15.156:36104) 2023-11-28T11:11:36.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90c56ee700 1 -- 172.21.15.156:0/1601464525 learned_addr learned my addr 172.21.15.156:0/1601464525 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:36.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90c5eef700 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 0x55ebfb3a80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90c66f0700 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3291c0 0x55ebfb2fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90c56ee700 1 -- 172.21.15.156:0/1601464525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3291c0 msgr2=0x55ebfb2fc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:36.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90c56ee700 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3291c0 0x55ebfb2fc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90c56ee700 1 -- 172.21.15.156:0/1601464525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 msgr2=0x55ebfb3a80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:36.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90c56ee700 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 0x55ebfb3a80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90c56ee700 1 -- 172.21.15.156:0/1601464525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebfb07ee30 con 0x55ebfb3288b0 2023-11-28T11:11:36.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90c56ee700 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 0x55ebfb345cc0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f90b4001eb0 tx=0x7f90b405b1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef952b891669787d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T11:11:36.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90a4ff9700 1 -- 172.21.15.156:0/1601464525 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4054cc0 con 0x55ebfb3288b0 2023-11-28T11:11:36.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90a4ff9700 1 -- 172.21.15.156:0/1601464525 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f90b405d300 con 0x55ebfb3288b0 2023-11-28T11:11:36.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1601464525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebfb20e200 con 0x55ebfb3288b0 2023-11-28T11:11:36.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90a4ff9700 1 -- 172.21.15.156:0/1601464525 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90b4083250 con 0x55ebfb3288b0 2023-11-28T11:11:36.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90d42dcb80 10 client.6288 fetch_fsmap learned FSMap version 32 2023-11-28T11:11:36.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1601464525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55ebfb0a52a0 con 0x55ebfb3288b0 2023-11-28T11:11:36.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90a4ff9700 1 -- 172.21.15.156:0/1601464525 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ebfb20e200 con 0x55ebfb3288b0 2023-11-28T11:11:36.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90a4ff9700 1 -- 172.21.15.156:0/1601464525 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f90b40512f0 con 0x55ebfb3288b0 2023-11-28T11:11:36.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.110+0000 7f90d42dcb80 10 client.6288 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:11:36.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 10 client.6288 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:11:36.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 20 client.6288 populate_metadata read hostname 'smithi156' 2023-11-28T11:11:36.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55ebfb3394d0 0x55ebfb3398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 4 client.6288 mds_command: new command op to 6021 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-28T11:11:36.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1601464525 --> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55ebfb1f8210 con 0x55ebfb3394d0 2023-11-28T11:11:36.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90c5eef700 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55ebfb3394d0 0x55ebfb3398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90a4ff9700 10 client.6288 ms_handle_connect on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:36.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90c5eef700 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55ebfb3394d0 0x55ebfb3398a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=230796ac824c7a98 server_cookie=ed61df7d33ab56c3 in_seq=0 out_seq=0 2023-11-28T11:11:36.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90a4ff9700 1 -- 172.21.15.156:0/1601464525 <== mds.0 v2:172.21.15.186:6833/3811791267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x55ebfb1f8210 con 0x55ebfb3394d0 2023-11-28T11:11:36.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90a4ff9700 10 client.6288 handle_command_reply: tid=0 2023-11-28T11:11:36.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 1 client.6288 shutdown 2023-11-28T11:11:36.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1601464525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 msgr2=0x55ebfb345cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 0x55ebfb345cc0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f90b4001eb0 tx=0x7f90b405b1a0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1601464525 shutdown_connections 2023-11-28T11:11:36.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.186:6833/3811791267,v1:172.21.15.186:6835/3811791267] conn(0x55ebfb3394d0 0x55ebfb3398a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ebfb3291c0 0x55ebfb2fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90a4ff9700 0 client.6288 ms_handle_reset on v2:172.21.15.186:6833/3811791267 2023-11-28T11:11:36.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ebfb3261d0 0x55ebfb3a80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 1 --2- 172.21.15.156:0/1601464525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebfb3288b0 0x55ebfb345cc0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.111+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1601464525 >> 172.21.15.156:0/1601464525 conn(0x55ebfb2fbe00 msgr2=0x55ebfb2f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:36.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1601464525 shutdown_connections 2023-11-28T11:11:36.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90d42dcb80 1 -- 172.21.15.156:0/1601464525 wait complete. 2023-11-28T11:11:36.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90d42dcb80 20 client.6288 trim_cache size 0 max 16384 2023-11-28T11:11:36.121 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:36.121 INFO:teuthology.orchestra.run.smithi156.stdout:[ 2023-11-28T11:11:36.121 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T11:11:36.121 INFO:teuthology.orchestra.run.smithi156.stdout: "damage_type": "backtrace", 2023-11-28T11:11:36.121 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 2245098610, 2023-11-28T11:11:36.121 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 1099511627777, 2023-11-28T11:11:36.121 INFO:teuthology.orchestra.run.smithi156.stdout: "path": "/dir_test/file" 2023-11-28T11:11:36.121 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T11:11:36.122 INFO:teuthology.orchestra.run.smithi156.stdout:] 2023-11-28T11:11:36.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90c8153700 1 -- 172.21.15.156:0/1884425980 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90ac0682f0 msgr2=0x7f90ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90c8153700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90ac0682f0 0x7f90ac06a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f90b800b370 tx=0x7f90b8011000 comp rx=0 tx=0).stop 2023-11-28T11:11:36.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90c8153700 1 -- 172.21.15.156:0/1884425980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c0071040 msgr2=0x7f90c01f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90c8153700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c0071040 0x7f90c01f5b30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f90b400c4d0 tx=0x7f90b400c7e0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90c8153700 1 -- 172.21.15.156:0/1884425980 shutdown_connections 2023-11-28T11:11:36.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90c8153700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0160540 0x7f90c00b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90c8153700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f90ac0682f0 0x7f90ac06a7a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90c8153700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90c006dee0 0x7f90c015b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90c8153700 1 --2- 172.21.15.156:0/1884425980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c0071040 0x7f90c01f5b30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90c8153700 1 -- 172.21.15.156:0/1884425980 >> 172.21.15.156:0/1884425980 conn(0x7f90c00d0000 msgr2=0x7f90c015f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:36.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90c8153700 1 -- 172.21.15.156:0/1884425980 shutdown_connections 2023-11-28T11:11:36.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.112+0000 7f90c8153700 1 -- 172.21.15.156:0/1884425980 wait complete. 2023-11-28T11:11:36.125 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:11:36.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.243+0000 7ff2495d3700 1 Processor -- start 2023-11-28T11:11:36.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.243+0000 7ff2495d3700 1 -- start start 2023-11-28T11:11:36.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff2495d3700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff24406ba90 0x7ff2440cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff2495d3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2440ccc40 0x7ff2440cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff2495d3700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2440cf5e0 con 0x7ff24415ca50 2023-11-28T11:11:36.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff2495d3700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2440cd470 con 0x7ff24406ba90 2023-11-28T11:11:36.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff2495d3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440cecb0 con 0x7ff2440ccc40 2023-11-28T11:11:36.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff242ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff24415ca50 0x7ff24406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50988/0 (socket says 172.21.15.156:50988) 2023-11-28T11:11:36.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff242ffd700 1 -- 172.21.15.156:0/2797467088 learned_addr learned my addr 172.21.15.156:0/2797467088 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:36.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff2427fc700 1 --2- 172.21.15.156:0/2797467088 >> v2:172.21.15.186:3300/0 conn(0x7ff24406ba90 0x7ff2440cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff2437fe700 1 --2- 172.21.15.156:0/2797467088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2440ccc40 0x7ff2440cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff241ffb700 1 -- 172.21.15.156:0/2797467088 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321971091 0 0) 0x7ff2440cf5e0 con 0x7ff24415ca50 2023-11-28T11:11:36.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff241ffb700 1 -- 172.21.15.156:0/2797467088 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff22c0035b0 con 0x7ff24415ca50 2023-11-28T11:11:36.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff2437fe700 1 -- 172.21.15.156:0/2797467088 >> v2:172.21.15.186:3300/0 conn(0x7ff24406ba90 msgr2=0x7ff2440cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:36.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff2437fe700 1 --2- 172.21.15.156:0/2797467088 >> v2:172.21.15.186:3300/0 conn(0x7ff24406ba90 0x7ff2440cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff2437fe700 1 -- 172.21.15.156:0/2797467088 >> v1:172.21.15.156:6789/0 conn(0x7ff24415ca50 legacy=0x7ff24406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff2437fe700 1 -- 172.21.15.156:0/2797467088 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2440cef90 con 0x7ff2440ccc40 2023-11-28T11:11:36.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff2437fe700 1 --2- 172.21.15.156:0/2797467088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2440ccc40 0x7ff2440cfa70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009d90 tx=0x7ff23c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=985351ceaaf7ccd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff241ffb700 1 -- 172.21.15.156:0/2797467088 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c00e070 con 0x7ff2440ccc40 2023-11-28T11:11:36.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff241ffb700 1 -- 172.21.15.156:0/2797467088 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c00bb20 con 0x7ff2440ccc40 2023-11-28T11:11:36.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.244+0000 7ff241ffb700 1 -- 172.21.15.156:0/2797467088 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c012430 con 0x7ff2440ccc40 2023-11-28T11:11:36.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.245+0000 7ff2495d3700 1 -- 172.21.15.156:0/2797467088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2440ccc40 msgr2=0x7ff2440cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.245+0000 7ff2495d3700 1 --2- 172.21.15.156:0/2797467088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2440ccc40 0x7ff2440cfa70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009d90 tx=0x7ff23c00b330 comp rx=0 tx=0).stop 2023-11-28T11:11:36.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.245+0000 7ff2495d3700 1 -- 172.21.15.156:0/2797467088 shutdown_connections 2023-11-28T11:11:36.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.245+0000 7ff2495d3700 1 --2- 172.21.15.156:0/2797467088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2440ccc40 0x7ff2440cfa70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.245+0000 7ff2495d3700 1 --2- 172.21.15.156:0/2797467088 >> v2:172.21.15.186:3300/0 conn(0x7ff24406ba90 0x7ff2440cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.245+0000 7ff2495d3700 1 -- 172.21.15.156:0/2797467088 >> 172.21.15.156:0/2797467088 conn(0x7ff24409a230 msgr2=0x7ff24408afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:36.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.245+0000 7ff2495d3700 1 -- 172.21.15.156:0/2797467088 shutdown_connections 2023-11-28T11:11:36.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.245+0000 7ff2495d3700 1 -- 172.21.15.156:0/2797467088 wait complete. 2023-11-28T11:11:36.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.246+0000 7ff2495d3700 1 Processor -- start 2023-11-28T11:11:36.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.246+0000 7ff2495d3700 1 -- start start 2023-11-28T11:11:36.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.247+0000 7ff2495d3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff24406ba90 0x7ff2440af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.247+0000 7ff2495d3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2440ccc40 0x7ff2441e63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.247+0000 7ff2495d3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff24415ca50 0x7ff2441e6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.247+0000 7ff2495d3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440b28e0 con 0x7ff24406ba90 2023-11-28T11:11:36.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.247+0000 7ff2495d3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440adf30 con 0x7ff24415ca50 2023-11-28T11:11:36.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.247+0000 7ff2495d3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440ae210 con 0x7ff2440ccc40 2023-11-28T11:11:36.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.247+0000 7ff242ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff24406ba90 0x7ff2440af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.247+0000 7ff242ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff24406ba90 0x7ff2440af070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36116/0 (socket says 172.21.15.156:36116) 2023-11-28T11:11:36.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.247+0000 7ff242ffd700 1 -- 172.21.15.156:0/3912601066 learned_addr learned my addr 172.21.15.156:0/3912601066 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:36.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff2437fe700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff24415ca50 0x7ff2441e6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff2427fc700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2440ccc40 0x7ff2441e63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff242ffd700 1 -- 172.21.15.156:0/3912601066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2440ccc40 msgr2=0x7ff2441e63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff242ffd700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2440ccc40 0x7ff2441e63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff242ffd700 1 -- 172.21.15.156:0/3912601066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff24415ca50 msgr2=0x7ff2441e6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff242ffd700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff24415ca50 0x7ff2441e6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff242ffd700 1 -- 172.21.15.156:0/3912601066 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c009a70 con 0x7ff24406ba90 2023-11-28T11:11:36.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff242ffd700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff24406ba90 0x7ff2440af070 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff234001eb0 tx=0x7ff23400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff22b7fe700 1 -- 172.21.15.156:0/3912601066 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234005d70 con 0x7ff24406ba90 2023-11-28T11:11:36.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff22b7fe700 1 -- 172.21.15.156:0/3912601066 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2340070d0 con 0x7ff24406ba90 2023-11-28T11:11:36.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff22b7fe700 1 -- 172.21.15.156:0/3912601066 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234007460 con 0x7ff24406ba90 2023-11-28T11:11:36.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff2495d3700 1 -- 172.21.15.156:0/3912601066 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2440cd0a0 con 0x7ff24406ba90 2023-11-28T11:11:36.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.248+0000 7ff2495d3700 1 -- 172.21.15.156:0/3912601066 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2440cf270 con 0x7ff24406ba90 2023-11-28T11:11:36.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.249+0000 7ff2495d3700 1 -- 172.21.15.156:0/3912601066 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2440cd230 con 0x7ff24406ba90 2023-11-28T11:11:36.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.249+0000 7ff22b7fe700 1 -- 172.21.15.156:0/3912601066 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff234019030 con 0x7ff24406ba90 2023-11-28T11:11:36.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.249+0000 7ff22b7fe700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff2300682f0 0x7ff23006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:36.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.250+0000 7ff22b7fe700 1 -- 172.21.15.156:0/3912601066 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7ff234085160 con 0x7ff24406ba90 2023-11-28T11:11:36.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.250+0000 7ff2427fc700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff2300682f0 0x7ff23006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:36.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.250+0000 7ff2427fc700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff2300682f0 0x7ff23006a7a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff2440b1a60 tx=0x7ff238008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:36.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.254+0000 7ff22b7fe700 1 -- 172.21.15.156:0/3912601066 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff2440cd230 con 0x7ff24406ba90 2023-11-28T11:11:36.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.477+0000 7ff2495d3700 1 -- 172.21.15.156:0/3912601066 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff2440cd230 con 0x7ff24406ba90 2023-11-28T11:11:36.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.477+0000 7ff22b7fe700 1 -- 172.21.15.156:0/3912601066 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7ff2440cd230 con 0x7ff24406ba90 2023-11-28T11:11:36.479 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:36.479 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:11:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.479+0000 7ff2495d3700 1 -- 172.21.15.156:0/3912601066 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff2300682f0 msgr2=0x7ff23006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.479+0000 7ff2495d3700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff2300682f0 0x7ff23006a7a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff2440b1a60 tx=0x7ff238008040 comp rx=0 tx=0).stop 2023-11-28T11:11:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.479+0000 7ff2495d3700 1 -- 172.21.15.156:0/3912601066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff24406ba90 msgr2=0x7ff2440af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.479+0000 7ff2495d3700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff24406ba90 0x7ff2440af070 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff234001eb0 tx=0x7ff23400c860 comp rx=0 tx=0).stop 2023-11-28T11:11:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.479+0000 7ff2495d3700 1 -- 172.21.15.156:0/3912601066 shutdown_connections 2023-11-28T11:11:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.479+0000 7ff2495d3700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff24415ca50 0x7ff2441e6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.479+0000 7ff2495d3700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff24406ba90 0x7ff2440af070 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.479+0000 7ff2495d3700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff2300682f0 0x7ff23006a7a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.479+0000 7ff2495d3700 1 --2- 172.21.15.156:0/3912601066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2440ccc40 0x7ff2441e63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.479+0000 7ff2495d3700 1 -- 172.21.15.156:0/3912601066 >> 172.21.15.156:0/3912601066 conn(0x7ff24409a230 msgr2=0x7ff24415b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.480+0000 7ff2495d3700 1 -- 172.21.15.156:0/3912601066 shutdown_connections 2023-11-28T11:11:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:36.480+0000 7ff2495d3700 1 -- 172.21.15.156:0/3912601066 wait complete. 2023-11-28T11:11:36.493 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:11:41.497 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:11:41.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.630+0000 7fd90048e700 1 Processor -- start 2023-11-28T11:11:41.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.630+0000 7fd90048e700 1 -- start start 2023-11-28T11:11:41.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.631+0000 7fd90048e700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd8f8071040 0x7fd8f8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:41.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.631+0000 7fd90048e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f8071950 con 0x7fd8f806dee0 2023-11-28T11:11:41.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.631+0000 7fd90048e700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f815d060 con 0x7fd8f8160540 2023-11-28T11:11:41.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.631+0000 7fd90048e700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f815d340 con 0x7fd8f8071040 2023-11-28T11:11:41.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fda29700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd8f806dee0 0x7fd8f806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51004/0 (socket says 172.21.15.156:51004) 2023-11-28T11:11:41.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fda29700 1 -- 172.21.15.156:0/1728647129 learned_addr learned my addr 172.21.15.156:0/1728647129 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:41.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fe22a700 1 --2- 172.21.15.156:0/1728647129 >> v2:172.21.15.186:3300/0 conn(0x7fd8f8071040 0x7fd8f8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:41.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577183164 0 0) 0x7fd8f8071950 con 0x7fd8f806dee0 2023-11-28T11:11:41.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e40035b0 con 0x7fd8f806dee0 2023-11-28T11:11:41.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 951641244 0 0) 0x7fd8e40035b0 con 0x7fd8f806dee0 2023-11-28T11:11:41.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 >> v2:172.21.15.186:3300/0 conn(0x7fd8f8071040 msgr2=0x7fd8f8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:41.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 --2- 172.21.15.156:0/1728647129 >> v2:172.21.15.186:3300/0 conn(0x7fd8f8071040 0x7fd8f8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:41.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 >> v1:172.21.15.186:6789/0 conn(0x7fd8f8160540 legacy=0x7fd8f8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:41.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f815d620 con 0x7fd8f806dee0 2023-11-28T11:11:41.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fd8f4001670 con 0x7fd8f806dee0 2023-11-28T11:11:41.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 >> v1:172.21.15.156:6789/0 conn(0x7fd8f806dee0 legacy=0x7fd8f806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:41.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 --2- 172.21.15.156:0/1728647129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8e4003a00 0x7fd8e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:41.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 --2- 172.21.15.156:0/1728647129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8e4004850 0x7fd8e4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:41.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 --2- 172.21.15.156:0/1728647129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8e4005320 0x7fd8e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:41.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f815d340 con 0x7fd8e4003a00 2023-11-28T11:11:41.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e400c250 con 0x7fd8e4005320 2023-11-28T11:11:41.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e400c410 con 0x7fd8e4004850 2023-11-28T11:11:41.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fe22a700 1 --2- 172.21.15.156:0/1728647129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8e4003a00 0x7fd8e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:41.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.632+0000 7fd8fea2b700 1 --2- 172.21.15.156:0/1728647129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8e4005320 0x7fd8e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:41.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd8fda29700 1 --2- 172.21.15.156:0/1728647129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8e4004850 0x7fd8e4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:41.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd8fe22a700 1 -- 172.21.15.156:0/1728647129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8e4004850 msgr2=0x7fd8e4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:41.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd8fe22a700 1 --2- 172.21.15.156:0/1728647129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8e4004850 0x7fd8e4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:41.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd8fe22a700 1 -- 172.21.15.156:0/1728647129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8e4005320 msgr2=0x7fd8e400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:41.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd8fe22a700 1 --2- 172.21.15.156:0/1728647129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8e4005320 0x7fd8e400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:41.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd8fea2b700 1 -- 172.21.15.156:0/1728647129 reap_dead start 2023-11-28T11:11:41.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd8fe22a700 1 -- 172.21.15.156:0/1728647129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e40035b0 con 0x7fd8e4003a00 2023-11-28T11:11:41.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd8fda29700 1 --2- 172.21.15.156:0/1728647129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8e4004850 0x7fd8e4004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:41.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd8fe22a700 1 --2- 172.21.15.156:0/1728647129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8e4003a00 0x7fd8e4003e10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009b60 tx=0x7fd8e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9728d60d9789e84 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:41.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800e6c0 con 0x7fd8e4003a00 2023-11-28T11:11:41.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e800e820 con 0x7fd8e4003a00 2023-11-28T11:11:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd8fd228700 1 -- 172.21.15.156:0/1728647129 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800eb90 con 0x7fd8e4003a00 2023-11-28T11:11:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd90048e700 1 -- 172.21.15.156:0/1728647129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8e4003a00 msgr2=0x7fd8e4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.633+0000 7fd90048e700 1 --2- 172.21.15.156:0/1728647129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8e4003a00 0x7fd8e4003e10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009b60 tx=0x7fd8e4005880 comp rx=0 tx=0).stop 2023-11-28T11:11:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.634+0000 7fd90048e700 1 -- 172.21.15.156:0/1728647129 shutdown_connections 2023-11-28T11:11:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.634+0000 7fd90048e700 1 --2- 172.21.15.156:0/1728647129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8e4003a00 0x7fd8e4003e10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.634+0000 7fd90048e700 1 -- 172.21.15.156:0/1728647129 >> 172.21.15.156:0/1728647129 conn(0x7fd8f80d0000 msgr2=0x7fd8f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.634+0000 7fd90048e700 1 -- 172.21.15.156:0/1728647129 shutdown_connections 2023-11-28T11:11:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.634+0000 7fd90048e700 1 -- 172.21.15.156:0/1728647129 wait complete. 2023-11-28T11:11:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.635+0000 7fd90048e700 1 Processor -- start 2023-11-28T11:11:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.635+0000 7fd90048e700 1 -- start start 2023-11-28T11:11:41.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd90048e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8e4003a00 0x7fd8f8152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:41.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd90048e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8f8155d30 0x7fd8f80c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:41.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd90048e700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8f81561a0 0x7fd8f80c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:41.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd90048e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80b4e00 con 0x7fd8f8155d30 2023-11-28T11:11:41.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd90048e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80c5050 con 0x7fd8e4003a00 2023-11-28T11:11:41.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd90048e700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f8157fa0 con 0x7fd8f81561a0 2023-11-28T11:11:41.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fda29700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8f8155d30 0x7fd8f80c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:41.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fda29700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8f8155d30 0x7fd8f80c5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36138/0 (socket says 172.21.15.156:36138) 2023-11-28T11:11:41.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fda29700 1 -- 172.21.15.156:0/1198093202 learned_addr learned my addr 172.21.15.156:0/1198093202 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:41.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fe22a700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8e4003a00 0x7fd8f8152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:41.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fea2b700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8f81561a0 0x7fd8f80c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:41.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fda29700 1 -- 172.21.15.156:0/1198093202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8f81561a0 msgr2=0x7fd8f80c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:41.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fda29700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8f81561a0 0x7fd8f80c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:41.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fda29700 1 -- 172.21.15.156:0/1198093202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8e4003a00 msgr2=0x7fd8f8152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:41.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fda29700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8e4003a00 0x7fd8f8152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:41.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fda29700 1 -- 172.21.15.156:0/1198093202 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f815ccd0 con 0x7fd8f8155d30 2023-11-28T11:11:41.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fea2b700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8f81561a0 0x7fd8f80c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:41.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fe22a700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8e4003a00 0x7fd8f8152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:41.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8fda29700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8f8155d30 0x7fd8f80c5ba0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400d7a0 tx=0x7fd8f400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:41.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8eeffd700 1 -- 172.21.15.156:0/1198093202 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4004c80 con 0x7fd8f8155d30 2023-11-28T11:11:41.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8eeffd700 1 -- 172.21.15.156:0/1198093202 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f4004de0 con 0x7fd8f8155d30 2023-11-28T11:11:41.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd90048e700 1 -- 172.21.15.156:0/1198093202 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f815d900 con 0x7fd8f8155d30 2023-11-28T11:11:41.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd8eeffd700 1 -- 172.21.15.156:0/1198093202 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40072d0 con 0x7fd8f8155d30 2023-11-28T11:11:41.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.636+0000 7fd90048e700 1 -- 172.21.15.156:0/1198093202 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f806a270 con 0x7fd8f8155d30 2023-11-28T11:11:41.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.637+0000 7fd90048e700 1 -- 172.21.15.156:0/1198093202 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f80b4f40 con 0x7fd8f8155d30 2023-11-28T11:11:41.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.638+0000 7fd8eeffd700 1 -- 172.21.15.156:0/1198093202 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd8f401b030 con 0x7fd8f8155d30 2023-11-28T11:11:41.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.638+0000 7fd8eeffd700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd8dc0682f0 0x7fd8dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:41.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.638+0000 7fd8eeffd700 1 -- 172.21.15.156:0/1198093202 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fd8f4085550 con 0x7fd8f8155d30 2023-11-28T11:11:41.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.638+0000 7fd8fe22a700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd8dc0682f0 0x7fd8dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:41.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.638+0000 7fd8fe22a700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd8dc0682f0 0x7fd8dc06a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80090b0 tx=0x7fd8e8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:41.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.642+0000 7fd8eeffd700 1 -- 172.21.15.156:0/1198093202 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd8f4051d30 con 0x7fd8f8155d30 2023-11-28T11:11:41.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.867+0000 7fd90048e700 1 -- 172.21.15.156:0/1198093202 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd8f815e880 con 0x7fd8f8155d30 2023-11-28T11:11:41.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.867+0000 7fd8eeffd700 1 -- 172.21.15.156:0/1198093202 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fd8f4055350 con 0x7fd8f8155d30 2023-11-28T11:11:41.869 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:41.869 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:11:41.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.869+0000 7fd90048e700 1 -- 172.21.15.156:0/1198093202 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd8dc0682f0 msgr2=0x7fd8dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:41.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.869+0000 7fd90048e700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd8dc0682f0 0x7fd8dc06a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80090b0 tx=0x7fd8e8008030 comp rx=0 tx=0).stop 2023-11-28T11:11:41.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.869+0000 7fd90048e700 1 -- 172.21.15.156:0/1198093202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8f8155d30 msgr2=0x7fd8f80c5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:41.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.869+0000 7fd90048e700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8f8155d30 0x7fd8f80c5ba0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400d7a0 tx=0x7fd8f400db60 comp rx=0 tx=0).stop 2023-11-28T11:11:41.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.869+0000 7fd90048e700 1 -- 172.21.15.156:0/1198093202 shutdown_connections 2023-11-28T11:11:41.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.869+0000 7fd90048e700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd8dc0682f0 0x7fd8dc06a7a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:41.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.869+0000 7fd90048e700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8f81561a0 0x7fd8f80c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:41.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.869+0000 7fd90048e700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8e4003a00 0x7fd8f8152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:41.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.869+0000 7fd90048e700 1 --2- 172.21.15.156:0/1198093202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8f8155d30 0x7fd8f80c5ba0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:41.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.869+0000 7fd90048e700 1 -- 172.21.15.156:0/1198093202 >> 172.21.15.156:0/1198093202 conn(0x7fd8f80d0000 msgr2=0x7fd8f815e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:41.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.869+0000 7fd90048e700 1 -- 172.21.15.156:0/1198093202 shutdown_connections 2023-11-28T11:11:41.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:41.869+0000 7fd90048e700 1 -- 172.21.15.156:0/1198093202 wait complete. 2023-11-28T11:11:41.883 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:11:46.884 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:11:47.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.016+0000 7fc883d03700 1 Processor -- start 2023-11-28T11:11:47.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.016+0000 7fc883d03700 1 -- start start 2023-11-28T11:11:47.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc883d03700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c15cef0 0x7fc87c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc883d03700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87c0cc5f0 con 0x7fc87c0cbdf0 2023-11-28T11:11:47.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc883d03700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87c15d470 con 0x7fc87c06ba90 2023-11-28T11:11:47.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc883d03700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c15d750 con 0x7fc87c15cef0 2023-11-28T11:11:47.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc88129e700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc87c0cbdf0 0x7fc87c06b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51022/0 (socket says 172.21.15.156:51022) 2023-11-28T11:11:47.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc88129e700 1 -- 172.21.15.156:0/3512446377 learned_addr learned my addr 172.21.15.156:0/3512446377 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:47.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc881a9f700 1 --2- 172.21.15.156:0/3512446377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c15cef0 0x7fc87c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23418511 0 0) 0x7fc87c0cc5f0 con 0x7fc87c0cbdf0 2023-11-28T11:11:47.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8680035b0 con 0x7fc87c0cbdf0 2023-11-28T11:11:47.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2351914249 0 0) 0x7fc8680035b0 con 0x7fc87c0cbdf0 2023-11-28T11:11:47.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c15cef0 msgr2=0x7fc87c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:47.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc880a9d700 1 --2- 172.21.15.156:0/3512446377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c15cef0 0x7fc87c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.017+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 >> v1:172.21.15.186:6789/0 conn(0x7fc87c06ba90 legacy=0x7fc87c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c15da30 con 0x7fc87c0cbdf0 2023-11-28T11:11:47.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fc878001670 con 0x7fc87c0cbdf0 2023-11-28T11:11:47.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 >> v1:172.21.15.156:6789/0 conn(0x7fc87c0cbdf0 legacy=0x7fc87c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc880a9d700 1 --2- 172.21.15.156:0/3512446377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c15cef0 0x7fc868003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc880a9d700 1 --2- 172.21.15.156:0/3512446377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc868003f70 0x7fc868004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc880a9d700 1 --2- 172.21.15.156:0/3512446377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868004d90 0x7fc868005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c15d750 con 0x7fc868004d90 2023-11-28T11:11:47.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86800ba40 con 0x7fc868003f70 2023-11-28T11:11:47.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86800bc00 con 0x7fc87c15cef0 2023-11-28T11:11:47.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc8822a0700 1 --2- 172.21.15.156:0/3512446377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868004d90 0x7fc868005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc88129e700 1 --2- 172.21.15.156:0/3512446377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc868003f70 0x7fc868004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc881a9f700 1 --2- 172.21.15.156:0/3512446377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c15cef0 0x7fc868003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc8822a0700 1 -- 172.21.15.156:0/3512446377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c15cef0 msgr2=0x7fc868003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:47.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc8822a0700 1 --2- 172.21.15.156:0/3512446377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c15cef0 0x7fc868003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc8822a0700 1 -- 172.21.15.156:0/3512446377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc868003f70 msgr2=0x7fc868004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:47.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc8822a0700 1 --2- 172.21.15.156:0/3512446377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc868003f70 0x7fc868004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc8822a0700 1 -- 172.21.15.156:0/3512446377 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680035b0 con 0x7fc868004d90 2023-11-28T11:11:47.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc8822a0700 1 --2- 172.21.15.156:0/3512446377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868004d90 0x7fc868005200 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c290 tx=0x7fc868005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c411202ef06eb431 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:47.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874001e70 con 0x7fc868004d90 2023-11-28T11:11:47.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87400ce50 con 0x7fc868004d90 2023-11-28T11:11:47.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.018+0000 7fc880a9d700 1 -- 172.21.15.156:0/3512446377 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874012660 con 0x7fc868004d90 2023-11-28T11:11:47.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.019+0000 7fc883d03700 1 -- 172.21.15.156:0/3512446377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868004d90 msgr2=0x7fc868005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.019+0000 7fc883d03700 1 --2- 172.21.15.156:0/3512446377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868004d90 0x7fc868005200 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c290 tx=0x7fc868005880 comp rx=0 tx=0).stop 2023-11-28T11:11:47.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.019+0000 7fc8822a0700 1 -- 172.21.15.156:0/3512446377 reap_dead start 2023-11-28T11:11:47.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.019+0000 7fc883d03700 1 -- 172.21.15.156:0/3512446377 shutdown_connections 2023-11-28T11:11:47.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.019+0000 7fc883d03700 1 -- 172.21.15.156:0/3512446377 >> 172.21.15.156:0/3512446377 conn(0x7fc87c0d0000 msgr2=0x7fc87c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:47.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.019+0000 7fc883d03700 1 -- 172.21.15.156:0/3512446377 shutdown_connections 2023-11-28T11:11:47.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.019+0000 7fc883d03700 1 -- 172.21.15.156:0/3512446377 wait complete. 2023-11-28T11:11:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.020+0000 7fc883d03700 1 Processor -- start 2023-11-28T11:11:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.020+0000 7fc883d03700 1 -- start start 2023-11-28T11:11:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc883d03700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868004d90 0x7fc87c0c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc883d03700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc87c0ae6a0 0x7fc87c1515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc883d03700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c0b0670 0x7fc87c0b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc883d03700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c0aeb10 con 0x7fc868004d90 2023-11-28T11:11:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc883d03700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c0b1020 con 0x7fc87c0ae6a0 2023-11-28T11:11:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc883d03700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c158980 con 0x7fc87c0b0670 2023-11-28T11:11:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc881a9f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868004d90 0x7fc87c0c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc881a9f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868004d90 0x7fc87c0c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36152/0 (socket says 172.21.15.156:36152) 2023-11-28T11:11:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc881a9f700 1 -- 172.21.15.156:0/2879817597 learned_addr learned my addr 172.21.15.156:0/2879817597 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc88129e700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc87c0ae6a0 0x7fc87c1515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc8822a0700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c0b0670 0x7fc87c0b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc881a9f700 1 -- 172.21.15.156:0/2879817597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c0b0670 msgr2=0x7fc87c0b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc881a9f700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c0b0670 0x7fc87c0b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc881a9f700 1 -- 172.21.15.156:0/2879817597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc87c0ae6a0 msgr2=0x7fc87c1515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc881a9f700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc87c0ae6a0 0x7fc87c1515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc881a9f700 1 -- 172.21.15.156:0/2879817597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c15dff0 con 0x7fc868004d90 2023-11-28T11:11:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.021+0000 7fc8822a0700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c0b0670 0x7fc87c0b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.022+0000 7fc881a9f700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868004d90 0x7fc87c0c79e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc86c011aa0 tx=0x7fc86c011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.022+0000 7fc8727fc700 1 -- 172.21.15.156:0/2879817597 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00f940 con 0x7fc868004d90 2023-11-28T11:11:47.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.022+0000 7fc8727fc700 1 -- 172.21.15.156:0/2879817597 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c00faa0 con 0x7fc868004d90 2023-11-28T11:11:47.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.022+0000 7fc883d03700 1 -- 172.21.15.156:0/2879817597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87c15dd10 con 0x7fc868004d90 2023-11-28T11:11:47.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.022+0000 7fc8727fc700 1 -- 172.21.15.156:0/2879817597 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00fe50 con 0x7fc868004d90 2023-11-28T11:11:47.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.022+0000 7fc883d03700 1 -- 172.21.15.156:0/2879817597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87c0af400 con 0x7fc868004d90 2023-11-28T11:11:47.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.023+0000 7fc883d03700 1 -- 172.21.15.156:0/2879817597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87c0cace0 con 0x7fc868004d90 2023-11-28T11:11:47.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.023+0000 7fc8727fc700 1 -- 172.21.15.156:0/2879817597 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc86c014440 con 0x7fc868004d90 2023-11-28T11:11:47.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.023+0000 7fc8727fc700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc8600682f0 0x7fc86006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.024+0000 7fc8727fc700 1 -- 172.21.15.156:0/2879817597 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fc86c086780 con 0x7fc868004d90 2023-11-28T11:11:47.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.024+0000 7fc88129e700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc8600682f0 0x7fc86006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.024+0000 7fc88129e700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc8600682f0 0x7fc86006a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc87c151cc0 tx=0x7fc878000f80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:47.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.028+0000 7fc8727fc700 1 -- 172.21.15.156:0/2879817597 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc86c052f00 con 0x7fc868004d90 2023-11-28T11:11:47.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.253+0000 7fc883d03700 1 -- 172.21.15.156:0/2879817597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc87c0d0e60 con 0x7fc868004d90 2023-11-28T11:11:47.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.253+0000 7fc8727fc700 1 -- 172.21.15.156:0/2879817597 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fc86c056520 con 0x7fc868004d90 2023-11-28T11:11:47.255 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:47.255 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:11:47.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.255+0000 7fc883d03700 1 -- 172.21.15.156:0/2879817597 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc8600682f0 msgr2=0x7fc86006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.255+0000 7fc883d03700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc8600682f0 0x7fc86006a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc87c151cc0 tx=0x7fc878000f80 comp rx=0 tx=0).stop 2023-11-28T11:11:47.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.255+0000 7fc883d03700 1 -- 172.21.15.156:0/2879817597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868004d90 msgr2=0x7fc87c0c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.255+0000 7fc883d03700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868004d90 0x7fc87c0c79e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc86c011aa0 tx=0x7fc86c011db0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.255+0000 7fc883d03700 1 -- 172.21.15.156:0/2879817597 shutdown_connections 2023-11-28T11:11:47.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.255+0000 7fc883d03700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc8600682f0 0x7fc86006a7a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.255+0000 7fc883d03700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc87c0b0670 0x7fc87c0b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.255+0000 7fc883d03700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868004d90 0x7fc87c0c79e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.255+0000 7fc883d03700 1 --2- 172.21.15.156:0/2879817597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc87c0ae6a0 0x7fc87c1515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.255+0000 7fc883d03700 1 -- 172.21.15.156:0/2879817597 >> 172.21.15.156:0/2879817597 conn(0x7fc87c0d0000 msgr2=0x7fc87c15c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:47.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.256+0000 7fc883d03700 1 -- 172.21.15.156:0/2879817597 shutdown_connections 2023-11-28T11:11:47.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:47.256+0000 7fc883d03700 1 -- 172.21.15.156:0/2879817597 wait complete. 2023-11-28T11:11:47.269 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:11:51.518 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:51.522 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:52.270 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:11:52.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.403+0000 7f879301b700 1 Processor -- start 2023-11-28T11:11:52.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.403+0000 7f879301b700 1 -- start start 2023-11-28T11:11:52.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.403+0000 7f879301b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f878c071040 0x7f878c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:52.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.403+0000 7f879301b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878c071950 con 0x7f878c06dee0 2023-11-28T11:11:52.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.403+0000 7f879301b700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878c15d060 con 0x7f878c160540 2023-11-28T11:11:52.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.403+0000 7f879301b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878c15d340 con 0x7f878c071040 2023-11-28T11:11:52.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.404+0000 7f8790db7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f878c071040 0x7f878c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:52.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.404+0000 7f878bfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f878c06dee0 0x7f878c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51040/0 (socket says 172.21.15.156:51040) 2023-11-28T11:11:52.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.404+0000 7f878bfff700 1 -- 172.21.15.156:0/4231901321 learned_addr learned my addr 172.21.15.156:0/4231901321 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:52.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.404+0000 7f8790db7700 1 -- 172.21.15.156:0/4231901321 >> v1:172.21.15.186:6789/0 conn(0x7f878c160540 legacy=0x7f878c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:52.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.404+0000 7f8790db7700 1 -- 172.21.15.156:0/4231901321 >> v1:172.21.15.156:6789/0 conn(0x7f878c06dee0 legacy=0x7f878c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:52.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.404+0000 7f878b7fe700 1 -- 172.21.15.156:0/4231901321 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600213219 0 0) 0x7f878c071950 con 0x7f878c06dee0 2023-11-28T11:11:52.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.404+0000 7f8790db7700 1 -- 172.21.15.156:0/4231901321 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c15d620 con 0x7f878c071040 2023-11-28T11:11:52.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.404+0000 7f8790db7700 1 --2- 172.21.15.156:0/4231901321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f878c071040 0x7f878c071410 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8780009840 tx=0x7f8780009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a56e3ef53c33777 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:52.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.404+0000 7f878b7fe700 1 -- 172.21.15.156:0/4231901321 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8780004230 con 0x7f878c071040 2023-11-28T11:11:52.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.404+0000 7f878b7fe700 1 -- 172.21.15.156:0/4231901321 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8780004390 con 0x7f878c071040 2023-11-28T11:11:52.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.404+0000 7f878b7fe700 1 -- 172.21.15.156:0/4231901321 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87800036a0 con 0x7f878c071040 2023-11-28T11:11:52.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.405+0000 7f879301b700 1 -- 172.21.15.156:0/4231901321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f878c071040 msgr2=0x7f878c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:52.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.405+0000 7f879301b700 1 --2- 172.21.15.156:0/4231901321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f878c071040 0x7f878c071410 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8780009840 tx=0x7f8780009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:52.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.405+0000 7f879301b700 1 -- 172.21.15.156:0/4231901321 shutdown_connections 2023-11-28T11:11:52.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.405+0000 7f879301b700 1 --2- 172.21.15.156:0/4231901321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f878c071040 0x7f878c071410 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.405+0000 7f879301b700 1 -- 172.21.15.156:0/4231901321 >> 172.21.15.156:0/4231901321 conn(0x7f878c0d0000 msgr2=0x7f878c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:52.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.405+0000 7f879301b700 1 -- 172.21.15.156:0/4231901321 shutdown_connections 2023-11-28T11:11:52.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.405+0000 7f879301b700 1 -- 172.21.15.156:0/4231901321 wait complete. 2023-11-28T11:11:52.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.406+0000 7f879301b700 1 Processor -- start 2023-11-28T11:11:52.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.406+0000 7f879301b700 1 -- start start 2023-11-28T11:11:52.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.407+0000 7f879301b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c06dee0 0x7f878c1f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:52.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.407+0000 7f879301b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f878c071040 0x7f878c0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:52.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.407+0000 7f879301b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f878c160540 0x7f878c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:52.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.407+0000 7f879301b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c1f6c10 con 0x7f878c071040 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.407+0000 7f879301b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0b4340 con 0x7f878c160540 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.407+0000 7f879301b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0b32a0 con 0x7f878c06dee0 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.407+0000 7f878bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f878c071040 0x7f878c0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.407+0000 7f878bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f878c071040 0x7f878c0b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36166/0 (socket says 172.21.15.156:36166) 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.407+0000 7f878bfff700 1 -- 172.21.15.156:0/1756022938 learned_addr learned my addr 172.21.15.156:0/1756022938 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f8790db7700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c06dee0 0x7f878c1f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f87915b8700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f878c160540 0x7f878c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f878bfff700 1 -- 172.21.15.156:0/1756022938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c06dee0 msgr2=0x7f878c1f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f878bfff700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c06dee0 0x7f878c1f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f878bfff700 1 -- 172.21.15.156:0/1756022938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f878c160540 msgr2=0x7f878c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f878bfff700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f878c160540 0x7f878c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f878bfff700 1 -- 172.21.15.156:0/1756022938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87800094a0 con 0x7f878c071040 2023-11-28T11:11:52.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f87915b8700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f878c160540 0x7f878c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:52.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f878bfff700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f878c071040 0x7f878c0b4980 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f877800c550 tx=0x7f877800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:52.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f87897fa700 1 -- 172.21.15.156:0/1756022938 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778004c80 con 0x7f878c071040 2023-11-28T11:11:52.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f87897fa700 1 -- 172.21.15.156:0/1756022938 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8778004e00 con 0x7f878c071040 2023-11-28T11:11:52.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f879301b700 1 -- 172.21.15.156:0/1756022938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c15ccd0 con 0x7f878c071040 2023-11-28T11:11:52.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f87897fa700 1 -- 172.21.15.156:0/1756022938 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778007310 con 0x7f878c071040 2023-11-28T11:11:52.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.408+0000 7f879301b700 1 -- 172.21.15.156:0/1756022938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c15d900 con 0x7f878c071040 2023-11-28T11:11:52.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.409+0000 7f879301b700 1 -- 172.21.15.156:0/1756022938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878c06a270 con 0x7f878c071040 2023-11-28T11:11:52.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.409+0000 7f87897fa700 1 -- 172.21.15.156:0/1756022938 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f877801a020 con 0x7f878c071040 2023-11-28T11:11:52.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.410+0000 7f87897fa700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f87700682f0 0x7f877006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:52.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.410+0000 7f87897fa700 1 -- 172.21.15.156:0/1756022938 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f8778085840 con 0x7f878c071040 2023-11-28T11:11:52.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.410+0000 7f8790db7700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f87700682f0 0x7f877006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:52.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.410+0000 7f8790db7700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f87700682f0 0x7f877006a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8780009b90 tx=0x7f8780013040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:52.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.414+0000 7f87897fa700 1 -- 172.21.15.156:0/1756022938 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8778052020 con 0x7f878c071040 2023-11-28T11:11:52.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.639+0000 7f879301b700 1 -- 172.21.15.156:0/1756022938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f878c0720a0 con 0x7f878c071040 2023-11-28T11:11:52.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.639+0000 7f87897fa700 1 -- 172.21.15.156:0/1756022938 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f8778055640 con 0x7f878c071040 2023-11-28T11:11:52.640 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:52.641 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:11:52.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.641+0000 7f879301b700 1 -- 172.21.15.156:0/1756022938 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f87700682f0 msgr2=0x7f877006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:52.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.641+0000 7f879301b700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f87700682f0 0x7f877006a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8780009b90 tx=0x7f8780013040 comp rx=0 tx=0).stop 2023-11-28T11:11:52.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.641+0000 7f879301b700 1 -- 172.21.15.156:0/1756022938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f878c071040 msgr2=0x7f878c0b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:52.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.641+0000 7f879301b700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f878c071040 0x7f878c0b4980 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f877800c550 tx=0x7f877800c910 comp rx=0 tx=0).stop 2023-11-28T11:11:52.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.641+0000 7f879301b700 1 -- 172.21.15.156:0/1756022938 shutdown_connections 2023-11-28T11:11:52.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.641+0000 7f879301b700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f878c160540 0x7f878c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.641+0000 7f879301b700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f87700682f0 0x7f877006a7a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.641+0000 7f879301b700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c06dee0 0x7f878c1f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.641+0000 7f879301b700 1 --2- 172.21.15.156:0/1756022938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f878c071040 0x7f878c0b4980 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.641+0000 7f879301b700 1 -- 172.21.15.156:0/1756022938 >> 172.21.15.156:0/1756022938 conn(0x7f878c0d0000 msgr2=0x7f878c15e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:52.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.641+0000 7f879301b700 1 -- 172.21.15.156:0/1756022938 shutdown_connections 2023-11-28T11:11:52.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:52.641+0000 7f879301b700 1 -- 172.21.15.156:0/1756022938 wait complete. 2023-11-28T11:11:52.654 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:11:57.655 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:11:57.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.787+0000 7f1a4a9f3700 1 Processor -- start 2023-11-28T11:11:57.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.787+0000 7f1a4a9f3700 1 -- start start 2023-11-28T11:11:57.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.787+0000 7f1a4a9f3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a44140950 0x7f1a440b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:57.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.787+0000 7f1a4a9f3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a440b2e70 0x7f1a4406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:57.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.787+0000 7f1a4a9f3700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4413ff00 con 0x7f1a440cd8c0 2023-11-28T11:11:57.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.787+0000 7f1a4a9f3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a441404c0 con 0x7f1a440b2e70 2023-11-28T11:11:57.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a4a9f3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44140dc0 con 0x7f1a44140950 2023-11-28T11:11:57.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a48f90700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a440b2e70 0x7f1a4406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:57.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a48f90700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a440b2e70 0x7f1a4406f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36178/0 (socket says 172.21.15.156:36178) 2023-11-28T11:11:57.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a48f90700 1 -- 172.21.15.156:0/1997860480 learned_addr learned my addr 172.21.15.156:0/1997860480 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:57.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a437fe700 1 --2- 172.21.15.156:0/1997860480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a44140950 0x7f1a440b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:57.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a48f90700 1 -- 172.21.15.156:0/1997860480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a44140950 msgr2=0x7f1a440b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:57.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a48f90700 1 --2- 172.21.15.156:0/1997860480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a44140950 0x7f1a440b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a48f90700 1 -- 172.21.15.156:0/1997860480 >> v1:172.21.15.186:6789/0 conn(0x7f1a440cd8c0 legacy=0x7f1a440cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:57.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a48f90700 1 -- 172.21.15.156:0/1997860480 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4415aa80 con 0x7f1a440b2e70 2023-11-28T11:11:57.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a48f90700 1 --2- 172.21.15.156:0/1997860480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a440b2e70 0x7f1a4406f850 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009e50 tx=0x7f1a3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba1bc732031ad03 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:57.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a42ffd700 1 -- 172.21.15.156:0/1997860480 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00e070 con 0x7f1a440b2e70 2023-11-28T11:11:57.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a42ffd700 1 -- 172.21.15.156:0/1997860480 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c00b9c0 con 0x7f1a440b2e70 2023-11-28T11:11:57.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.788+0000 7f1a42ffd700 1 -- 172.21.15.156:0/1997860480 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00bd70 con 0x7f1a440b2e70 2023-11-28T11:11:57.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.789+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/1997860480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a440b2e70 msgr2=0x7f1a4406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:57.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.789+0000 7f1a4a9f3700 1 --2- 172.21.15.156:0/1997860480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a440b2e70 0x7f1a4406f850 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009e50 tx=0x7f1a3c00b330 comp rx=0 tx=0).stop 2023-11-28T11:11:57.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.789+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/1997860480 shutdown_connections 2023-11-28T11:11:57.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.789+0000 7f1a4a9f3700 1 --2- 172.21.15.156:0/1997860480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a440b2e70 0x7f1a4406f850 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.789+0000 7f1a4a9f3700 1 --2- 172.21.15.156:0/1997860480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a44140950 0x7f1a440b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.789+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/1997860480 >> 172.21.15.156:0/1997860480 conn(0x7f1a4406aed0 msgr2=0x7f1a4406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:57.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.789+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/1997860480 shutdown_connections 2023-11-28T11:11:57.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.789+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/1997860480 wait complete. 2023-11-28T11:11:57.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.791+0000 7f1a4a9f3700 1 Processor -- start 2023-11-28T11:11:57.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.791+0000 7f1a4a9f3700 1 -- start start 2023-11-28T11:11:57.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a4a9f3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a440b2e70 0x7f1a440b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:57.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a4a9f3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a440cd8c0 0x7f1a440b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:57.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a4a9f3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a44140950 0x7f1a440b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:57.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a4a9f3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440b4ca0 con 0x7f1a440b2e70 2023-11-28T11:11:57.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a4a9f3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44140ff0 con 0x7f1a440cd8c0 2023-11-28T11:11:57.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a4a9f3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440c9110 con 0x7f1a44140950 2023-11-28T11:11:57.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a43fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a440b2e70 0x7f1a440b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:57.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a43fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a440b2e70 0x7f1a440b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36180/0 (socket says 172.21.15.156:36180) 2023-11-28T11:11:57.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a43fff700 1 -- 172.21.15.156:0/2389396022 learned_addr learned my addr 172.21.15.156:0/2389396022 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:57.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a48f90700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a44140950 0x7f1a440b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:57.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a437fe700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a440cd8c0 0x7f1a440b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:57.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a43fff700 1 -- 172.21.15.156:0/2389396022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a44140950 msgr2=0x7f1a440b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:57.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a43fff700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a44140950 0x7f1a440b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a43fff700 1 -- 172.21.15.156:0/2389396022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a440cd8c0 msgr2=0x7f1a440b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:57.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a43fff700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a440cd8c0 0x7f1a440b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a43fff700 1 -- 172.21.15.156:0/2389396022 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c009b30 con 0x7f1a440b2e70 2023-11-28T11:11:57.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a43fff700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a440b2e70 0x7f1a440b2810 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400c4d0 tx=0x7f1a3400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:57.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a40ff9700 1 -- 172.21.15.156:0/2389396022 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a340052d0 con 0x7f1a440b2e70 2023-11-28T11:11:57.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a40ff9700 1 -- 172.21.15.156:0/2389396022 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a34005430 con 0x7f1a440b2e70 2023-11-28T11:11:57.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.792+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/2389396022 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a440b35f0 con 0x7f1a440b2e70 2023-11-28T11:11:57.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.793+0000 7f1a40ff9700 1 -- 172.21.15.156:0/2389396022 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a340038f0 con 0x7f1a440b2e70 2023-11-28T11:11:57.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.793+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/2389396022 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a440b3310 con 0x7f1a440b2e70 2023-11-28T11:11:57.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.793+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/2389396022 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a44140150 con 0x7f1a440b2e70 2023-11-28T11:11:57.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.794+0000 7f1a40ff9700 1 -- 172.21.15.156:0/2389396022 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a34019030 con 0x7f1a440b2e70 2023-11-28T11:11:57.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.794+0000 7f1a40ff9700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1a300682f0 0x7f1a3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:57.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.794+0000 7f1a437fe700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1a300682f0 0x7f1a3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:57.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.794+0000 7f1a40ff9700 1 -- 172.21.15.156:0/2389396022 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f1a34085170 con 0x7f1a440b2e70 2023-11-28T11:11:57.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.795+0000 7f1a437fe700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1a300682f0 0x7f1a3006a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a44157280 tx=0x7f1a38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:57.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:57.799+0000 7f1a40ff9700 1 -- 172.21.15.156:0/2389396022 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1a34051860 con 0x7f1a440b2e70 2023-11-28T11:11:58.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.024+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/2389396022 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1a440c9280 con 0x7f1a440b2e70 2023-11-28T11:11:58.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.024+0000 7f1a40ff9700 1 -- 172.21.15.156:0/2389396022 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f1a340170c0 con 0x7f1a440b2e70 2023-11-28T11:11:58.026 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:58.026 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:11:58.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.026+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/2389396022 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1a300682f0 msgr2=0x7f1a3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.026+0000 7f1a4a9f3700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1a300682f0 0x7f1a3006a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a44157280 tx=0x7f1a38008040 comp rx=0 tx=0).stop 2023-11-28T11:11:58.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.026+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/2389396022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a440b2e70 msgr2=0x7f1a440b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.026+0000 7f1a4a9f3700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a440b2e70 0x7f1a440b2810 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400c4d0 tx=0x7f1a3400c890 comp rx=0 tx=0).stop 2023-11-28T11:11:58.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.026+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/2389396022 shutdown_connections 2023-11-28T11:11:58.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.026+0000 7f1a4a9f3700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1a300682f0 0x7f1a3006a7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.026+0000 7f1a4a9f3700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a44140950 0x7f1a440b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.026+0000 7f1a4a9f3700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a440b2e70 0x7f1a440b2810 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.026+0000 7f1a4a9f3700 1 --2- 172.21.15.156:0/2389396022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a440cd8c0 0x7f1a440b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.026+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/2389396022 >> 172.21.15.156:0/2389396022 conn(0x7f1a4406aed0 msgr2=0x7f1a440cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:58.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.026+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/2389396022 shutdown_connections 2023-11-28T11:11:58.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.026+0000 7f1a4a9f3700 1 -- 172.21.15.156:0/2389396022 wait complete. 2023-11-28T11:11:58.039 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:11:58.039 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:11:58.082 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:11:58.137 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:11:58.137 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:11:58.137 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:58.155 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:11:58.155 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:58.212 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:58.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.372+0000 7f7b1829a700 1 Processor -- start 2023-11-28T11:11:58.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.372+0000 7f7b1829a700 1 -- start start 2023-11-28T11:11:58.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.373+0000 7f7b1829a700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7b100ce040 0x7f7b1015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.373+0000 7f7b1829a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1006b420 con 0x7f7b100b2930 2023-11-28T11:11:58.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.373+0000 7f7b1829a700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1006b700 con 0x7f7b100d0a00 2023-11-28T11:11:58.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.373+0000 7f7b1829a700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1006ff60 con 0x7f7b100ce040 2023-11-28T11:11:58.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.373+0000 7f7b16837700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7b100b2930 0x7f7b1006c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51068/0 (socket says 172.21.15.156:51068) 2023-11-28T11:11:58.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.373+0000 7f7b16837700 1 -- 172.21.15.156:0/3546846335 learned_addr learned my addr 172.21.15.156:0/3546846335 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:58.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b16036700 1 --2- 172.21.15.156:0/3546846335 >> v2:172.21.15.186:3300/0 conn(0x7f7b100ce040 0x7f7b1015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253249767 0 0) 0x7f7b1006b420 con 0x7f7b100b2930 2023-11-28T11:11:58.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7afc0035b0 con 0x7f7b100b2930 2023-11-28T11:11:58.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384126276 0 0) 0x7f7b1006b700 con 0x7f7b100d0a00 2023-11-28T11:11:58.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b1006b420 con 0x7f7b100d0a00 2023-11-28T11:11:58.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1637034824 0 0) 0x7f7afc0035b0 con 0x7f7b100b2930 2023-11-28T11:11:58.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 >> v2:172.21.15.186:3300/0 conn(0x7f7b100ce040 msgr2=0x7f7b1015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 --2- 172.21.15.156:0/3546846335 >> v2:172.21.15.186:3300/0 conn(0x7f7b100ce040 0x7f7b1015df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 >> v1:172.21.15.186:6789/0 conn(0x7f7b100d0a00 legacy=0x7f7b1006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1006b9e0 con 0x7f7b100b2930 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f7b080019f0 con 0x7f7b100b2930 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 >> v1:172.21.15.156:6789/0 conn(0x7f7b100b2930 legacy=0x7f7b1006c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 --2- 172.21.15.156:0/3546846335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7afc003b90 0x7f7afc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 --2- 172.21.15.156:0/3546846335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7afc004850 0x7f7afc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 --2- 172.21.15.156:0/3546846335 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7afc005200 0x7f7afc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1006ff60 con 0x7f7afc003b90 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc00c060 con 0x7f7afc004850 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc00c220 con 0x7f7afc005200 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b16036700 1 --2- 172.21.15.156:0/3546846335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7afc003b90 0x7f7afc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15835700 1 --2- 172.21.15.156:0/3546846335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7afc004850 0x7f7afc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b16837700 1 --2- 172.21.15.156:0/3546846335 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7afc005200 0x7f7afc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b16036700 1 -- 172.21.15.156:0/3546846335 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7afc005200 msgr2=0x7f7afc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:58.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b16036700 1 --2- 172.21.15.156:0/3546846335 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7afc005200 0x7f7afc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b16036700 1 -- 172.21.15.156:0/3546846335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7afc004850 msgr2=0x7f7afc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:58.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b16036700 1 --2- 172.21.15.156:0/3546846335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7afc004850 0x7f7afc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b16837700 1 -- 172.21.15.156:0/3546846335 reap_dead start 2023-11-28T11:11:58.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b16036700 1 -- 172.21.15.156:0/3546846335 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc0035b0 con 0x7f7afc003b90 2023-11-28T11:11:58.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.374+0000 7f7b15835700 1 --2- 172.21.15.156:0/3546846335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7afc004850 0x7f7afc004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:58.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.375+0000 7f7b16036700 1 --2- 172.21.15.156:0/3546846335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7afc003b90 0x7f7afc003f80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009b40 tx=0x7f7afc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517a11e90474c286 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:58.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.375+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00005a20 con 0x7f7afc003b90 2023-11-28T11:11:58.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.375+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b00005b80 con 0x7f7afc003b90 2023-11-28T11:11:58.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.375+0000 7f7b15034700 1 -- 172.21.15.156:0/3546846335 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0000e6c0 con 0x7f7afc003b90 2023-11-28T11:11:58.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.375+0000 7f7b1829a700 1 -- 172.21.15.156:0/3546846335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7afc003b90 msgr2=0x7f7afc003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.375+0000 7f7b1829a700 1 --2- 172.21.15.156:0/3546846335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7afc003b90 0x7f7afc003f80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009b40 tx=0x7f7afc005880 comp rx=0 tx=0).stop 2023-11-28T11:11:58.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.375+0000 7f7b1829a700 1 -- 172.21.15.156:0/3546846335 shutdown_connections 2023-11-28T11:11:58.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.375+0000 7f7b1829a700 1 --2- 172.21.15.156:0/3546846335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7afc003b90 0x7f7afc003f80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.375+0000 7f7b1829a700 1 -- 172.21.15.156:0/3546846335 >> 172.21.15.156:0/3546846335 conn(0x7f7b1006aed0 msgr2=0x7f7b1006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:58.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.375+0000 7f7b1829a700 1 -- 172.21.15.156:0/3546846335 shutdown_connections 2023-11-28T11:11:58.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.375+0000 7f7b1829a700 1 -- 172.21.15.156:0/3546846335 wait complete. 2023-11-28T11:11:58.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.376+0000 7f7b1829a700 1 Processor -- start 2023-11-28T11:11:58.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.377+0000 7f7b1829a700 1 -- start start 2023-11-28T11:11:58.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.377+0000 7f7b1829a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7afc003b90 0x7f7b10154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.377+0000 7f7b1829a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b10155be0 0x7f7b100c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.377+0000 7f7b1829a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b100cecd0 0x7f7b1013ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.377+0000 7f7b1829a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b101557c0 con 0x7f7b100cecd0 2023-11-28T11:11:58.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.377+0000 7f7b1829a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b101f22d0 con 0x7f7b10155be0 2023-11-28T11:11:58.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.377+0000 7f7b1829a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b10154b40 con 0x7f7afc003b90 2023-11-28T11:11:58.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.377+0000 7f7b16837700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b100cecd0 0x7f7b1013ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.377+0000 7f7b16837700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b100cecd0 0x7f7b1013ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36200/0 (socket says 172.21.15.156:36200) 2023-11-28T11:11:58.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.377+0000 7f7b16837700 1 -- 172.21.15.156:0/1003157891 learned_addr learned my addr 172.21.15.156:0/1003157891 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:58.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.378+0000 7f7b15835700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b10155be0 0x7f7b100c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.378+0000 7f7b16837700 1 -- 172.21.15.156:0/1003157891 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7afc003b90 msgr2=0x7f7b10154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.378+0000 7f7b16837700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7afc003b90 0x7f7b10154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.378+0000 7f7b16837700 1 -- 172.21.15.156:0/1003157891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b10155be0 msgr2=0x7f7b100c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.378+0000 7f7b16837700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b10155be0 0x7f7b100c9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.378+0000 7f7b16837700 1 -- 172.21.15.156:0/1003157891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1006bfa0 con 0x7f7b100cecd0 2023-11-28T11:11:58.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.378+0000 7f7b16837700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b100cecd0 0x7f7b1013ed00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7b08000f80 tx=0x7f7b08007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:58.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.378+0000 7f7b06ffd700 1 -- 172.21.15.156:0/1003157891 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800ece0 con 0x7f7b100cecd0 2023-11-28T11:11:58.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.378+0000 7f7b06ffd700 1 -- 172.21.15.156:0/1003157891 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0800ee40 con 0x7f7b100cecd0 2023-11-28T11:11:58.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.378+0000 7f7b1829a700 1 -- 172.21.15.156:0/1003157891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1006bcc0 con 0x7f7b100cecd0 2023-11-28T11:11:58.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.378+0000 7f7b06ffd700 1 -- 172.21.15.156:0/1003157891 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08013660 con 0x7f7b100cecd0 2023-11-28T11:11:58.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.378+0000 7f7b1829a700 1 -- 172.21.15.156:0/1003157891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1015df10 con 0x7f7b100cecd0 2023-11-28T11:11:58.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.379+0000 7f7b1829a700 1 -- 172.21.15.156:0/1003157891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b10068690 con 0x7f7b100cecd0 2023-11-28T11:11:58.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.379+0000 7f7b06ffd700 1 -- 172.21.15.156:0/1003157891 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b080137c0 con 0x7f7b100cecd0 2023-11-28T11:11:58.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.380+0000 7f7b06ffd700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7af40682f0 0x7f7af406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.380+0000 7f7b06ffd700 1 -- 172.21.15.156:0/1003157891 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f7b080864a0 con 0x7f7b100cecd0 2023-11-28T11:11:58.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.380+0000 7f7b16036700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7af40682f0 0x7f7af406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.380+0000 7f7b16036700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7af40682f0 0x7f7af406a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7b00000c00 tx=0x7f7b00013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:58.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.384+0000 7f7b06ffd700 1 -- 172.21.15.156:0/1003157891 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7b08052c80 con 0x7f7b100cecd0 2023-11-28T11:11:58.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.566+0000 7f7b1829a700 1 -- 172.21.15.156:0/1003157891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b1015d7f0 con 0x7f7b100cecd0 2023-11-28T11:11:58.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.567+0000 7f7b06ffd700 1 -- 172.21.15.156:0/1003157891 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18161 (secure 0 0 0) 0x7f7b080562a0 con 0x7f7b100cecd0 2023-11-28T11:11:58.568 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:11:58.569 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":65,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:11:02.788873+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:50.055098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:51.350508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.33","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:58.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.569+0000 7f7b1829a700 1 -- 172.21.15.156:0/1003157891 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7af40682f0 msgr2=0x7f7af406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.569+0000 7f7b1829a700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7af40682f0 0x7f7af406a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7b00000c00 tx=0x7f7b00013000 comp rx=0 tx=0).stop 2023-11-28T11:11:58.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.569+0000 7f7b1829a700 1 -- 172.21.15.156:0/1003157891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b100cecd0 msgr2=0x7f7b1013ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.569+0000 7f7b1829a700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b100cecd0 0x7f7b1013ed00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7b08000f80 tx=0x7f7b08007c90 comp rx=0 tx=0).stop 2023-11-28T11:11:58.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.569+0000 7f7b1829a700 1 -- 172.21.15.156:0/1003157891 shutdown_connections 2023-11-28T11:11:58.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.569+0000 7f7b1829a700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b100cecd0 0x7f7b1013ed00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.569+0000 7f7b1829a700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7af40682f0 0x7f7af406a7a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.569+0000 7f7b1829a700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7afc003b90 0x7f7b10154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.569+0000 7f7b1829a700 1 --2- 172.21.15.156:0/1003157891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b10155be0 0x7f7b100c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.569+0000 7f7b1829a700 1 -- 172.21.15.156:0/1003157891 >> 172.21.15.156:0/1003157891 conn(0x7f7b1006aed0 msgr2=0x7f7b1015f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:58.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.569+0000 7f7b1829a700 1 -- 172.21.15.156:0/1003157891 shutdown_connections 2023-11-28T11:11:58.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.569+0000 7f7b1829a700 1 -- 172.21.15.156:0/1003157891 wait complete. 2023-11-28T11:11:58.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:10:50.055098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:10:51.350508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:11:58.583 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:11:58.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.706+0000 7f4b703ff700 1 Processor -- start 2023-11-28T11:11:58.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.706+0000 7f4b703ff700 1 -- start start 2023-11-28T11:11:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b703ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b6815cef0 0x7f4b680af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b703ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b680cc5f0 con 0x7f4b6806ba90 2023-11-28T11:11:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b703ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b6815d470 con 0x7f4b680cbdf0 2023-11-28T11:11:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b703ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6815d750 con 0x7f4b6815cef0 2023-11-28T11:11:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b6e99c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4b6806ba90 0x7f4b680ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51088/0 (socket says 172.21.15.156:51088) 2023-11-28T11:11:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b6e99c700 1 -- 172.21.15.156:0/1611188115 learned_addr learned my addr 172.21.15.156:0/1611188115 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b6e19b700 1 --2- 172.21.15.156:0/1611188115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b6815cef0 0x7f4b680af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b6d199700 1 -- 172.21.15.156:0/1611188115 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034708428 0 0) 0x7f4b680cc5f0 con 0x7f4b6806ba90 2023-11-28T11:11:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b6d199700 1 -- 172.21.15.156:0/1611188115 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b540035b0 con 0x7f4b6806ba90 2023-11-28T11:11:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b6d199700 1 -- 172.21.15.156:0/1611188115 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876797224 0 0) 0x7f4b6815d470 con 0x7f4b680cbdf0 2023-11-28T11:11:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b6d199700 1 -- 172.21.15.156:0/1611188115 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b680cc5f0 con 0x7f4b680cbdf0 2023-11-28T11:11:58.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b6e19b700 1 -- 172.21.15.156:0/1611188115 >> v1:172.21.15.186:6789/0 conn(0x7f4b680cbdf0 legacy=0x7f4b6806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b6e19b700 1 -- 172.21.15.156:0/1611188115 >> v1:172.21.15.156:6789/0 conn(0x7f4b6806ba90 legacy=0x7f4b680ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.707+0000 7f4b6e19b700 1 -- 172.21.15.156:0/1611188115 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6815da30 con 0x7f4b6815cef0 2023-11-28T11:11:58.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.708+0000 7f4b6e19b700 1 --2- 172.21.15.156:0/1611188115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b6815cef0 0x7f4b680af400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009870 tx=0x7f4b64009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=297bc6c95480fef3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:58.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.708+0000 7f4b6d199700 1 -- 172.21.15.156:0/1611188115 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64011830 con 0x7f4b6815cef0 2023-11-28T11:11:58.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.708+0000 7f4b6d199700 1 -- 172.21.15.156:0/1611188115 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b64011990 con 0x7f4b6815cef0 2023-11-28T11:11:58.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.708+0000 7f4b6d199700 1 -- 172.21.15.156:0/1611188115 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64011d20 con 0x7f4b6815cef0 2023-11-28T11:11:58.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.708+0000 7f4b703ff700 1 -- 172.21.15.156:0/1611188115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b6815cef0 msgr2=0x7f4b680af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.708+0000 7f4b703ff700 1 --2- 172.21.15.156:0/1611188115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b6815cef0 0x7f4b680af400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009870 tx=0x7f4b64009b80 comp rx=0 tx=0).stop 2023-11-28T11:11:58.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.708+0000 7f4b703ff700 1 -- 172.21.15.156:0/1611188115 shutdown_connections 2023-11-28T11:11:58.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.708+0000 7f4b703ff700 1 --2- 172.21.15.156:0/1611188115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b6815cef0 0x7f4b680af400 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.708+0000 7f4b703ff700 1 -- 172.21.15.156:0/1611188115 >> 172.21.15.156:0/1611188115 conn(0x7f4b680d0000 msgr2=0x7f4b680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:58.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.708+0000 7f4b703ff700 1 -- 172.21.15.156:0/1611188115 shutdown_connections 2023-11-28T11:11:58.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.708+0000 7f4b703ff700 1 -- 172.21.15.156:0/1611188115 wait complete. 2023-11-28T11:11:58.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.709+0000 7f4b703ff700 1 Processor -- start 2023-11-28T11:11:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.710+0000 7f4b703ff700 1 -- start start 2023-11-28T11:11:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.710+0000 7f4b703ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6806ba90 0x7f4b6814d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.710+0000 7f4b703ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b680cbdf0 0x7f4b6806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.710+0000 7f4b703ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b6815cef0 0x7f4b6806e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.710+0000 7f4b703ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680b2ea0 con 0x7f4b680cbdf0 2023-11-28T11:11:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.710+0000 7f4b703ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6814da20 con 0x7f4b6806ba90 2023-11-28T11:11:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.710+0000 7f4b703ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680c9580 con 0x7f4b6815cef0 2023-11-28T11:11:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.710+0000 7f4b6d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b680cbdf0 0x7f4b6806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.710+0000 7f4b6d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b680cbdf0 0x7f4b6806dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36212/0 (socket says 172.21.15.156:36212) 2023-11-28T11:11:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.710+0000 7f4b6d99a700 1 -- 172.21.15.156:0/2555943684 learned_addr learned my addr 172.21.15.156:0/2555943684 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b6e99c700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b6815cef0 0x7f4b6806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b6e19b700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6806ba90 0x7f4b6814d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b6d99a700 1 -- 172.21.15.156:0/2555943684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b6815cef0 msgr2=0x7f4b6806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b6d99a700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b6815cef0 0x7f4b6806e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b6d99a700 1 -- 172.21.15.156:0/2555943684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6806ba90 msgr2=0x7f4b6814d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b6d99a700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6806ba90 0x7f4b6814d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b6d99a700 1 -- 172.21.15.156:0/2555943684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b640094a0 con 0x7f4b680cbdf0 2023-11-28T11:11:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b6d99a700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b680cbdf0 0x7f4b6806dfd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800c520 tx=0x7f4b5800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b5effd700 1 -- 172.21.15.156:0/2555943684 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b580052d0 con 0x7f4b680cbdf0 2023-11-28T11:11:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b5effd700 1 -- 172.21.15.156:0/2555943684 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b58005430 con 0x7f4b680cbdf0 2023-11-28T11:11:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b5effd700 1 -- 172.21.15.156:0/2555943684 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b580038c0 con 0x7f4b680cbdf0 2023-11-28T11:11:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b703ff700 1 -- 172.21.15.156:0/2555943684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6815dff0 con 0x7f4b680cbdf0 2023-11-28T11:11:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.711+0000 7f4b703ff700 1 -- 172.21.15.156:0/2555943684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6815dd10 con 0x7f4b680cbdf0 2023-11-28T11:11:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.712+0000 7f4b703ff700 1 -- 172.21.15.156:0/2555943684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b680cad60 con 0x7f4b680cbdf0 2023-11-28T11:11:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.712+0000 7f4b5effd700 1 -- 172.21.15.156:0/2555943684 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4b58019030 con 0x7f4b680cbdf0 2023-11-28T11:11:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.713+0000 7f4b5effd700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4b4c0682f0 0x7f4b4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.713+0000 7f4b5effd700 1 -- 172.21.15.156:0/2555943684 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f4b58085000 con 0x7f4b680cbdf0 2023-11-28T11:11:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.713+0000 7f4b6e19b700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4b4c0682f0 0x7f4b4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.713+0000 7f4b6e19b700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4b4c0682f0 0x7f4b4c06a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4b640090b0 tx=0x7f4b6401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:58.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.717+0000 7f4b5effd700 1 -- 172.21.15.156:0/2555943684 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4b580517e0 con 0x7f4b680cbdf0 2023-11-28T11:11:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.900+0000 7f4b703ff700 1 -- 172.21.15.156:0/2555943684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b680b2fe0 con 0x7f4b680cbdf0 2023-11-28T11:11:58.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.901+0000 7f4b5effd700 1 -- 172.21.15.156:0/2555943684 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b58017020 con 0x7f4b680cbdf0 2023-11-28T11:11:58.902 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:11:58.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.903+0000 7f4b703ff700 1 -- 172.21.15.156:0/2555943684 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4b4c0682f0 msgr2=0x7f4b4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.903+0000 7f4b703ff700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4b4c0682f0 0x7f4b4c06a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4b640090b0 tx=0x7f4b6401c040 comp rx=0 tx=0).stop 2023-11-28T11:11:58.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.903+0000 7f4b703ff700 1 -- 172.21.15.156:0/2555943684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b680cbdf0 msgr2=0x7f4b6806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.903+0000 7f4b703ff700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b680cbdf0 0x7f4b6806dfd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800c520 tx=0x7f4b5800c830 comp rx=0 tx=0).stop 2023-11-28T11:11:58.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.903+0000 7f4b703ff700 1 -- 172.21.15.156:0/2555943684 shutdown_connections 2023-11-28T11:11:58.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.903+0000 7f4b703ff700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4b4c0682f0 0x7f4b4c06a7a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.903+0000 7f4b703ff700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b6815cef0 0x7f4b6806e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.903+0000 7f4b703ff700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6806ba90 0x7f4b6814d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.903+0000 7f4b703ff700 1 --2- 172.21.15.156:0/2555943684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b680cbdf0 0x7f4b6806dfd0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.903+0000 7f4b703ff700 1 -- 172.21.15.156:0/2555943684 >> 172.21.15.156:0/2555943684 conn(0x7f4b680d0000 msgr2=0x7f4b680ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:58.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.903+0000 7f4b703ff700 1 -- 172.21.15.156:0/2555943684 shutdown_connections 2023-11-28T11:11:58.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:58.903+0000 7f4b703ff700 1 -- 172.21.15.156:0/2555943684 wait complete. 2023-11-28T11:11:58.916 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:11:59.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.038+0000 7f18402aa700 1 Processor -- start 2023-11-28T11:11:59.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.038+0000 7f18402aa700 1 -- start start 2023-11-28T11:11:59.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.039+0000 7f18402aa700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f183815cef0 0x7f18380af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.039+0000 7f18402aa700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f183806ba90 0x7f18380ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.039+0000 7f18402aa700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18380cc5f0 con 0x7f18380cbdf0 2023-11-28T11:11:59.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.039+0000 7f18402aa700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183815d470 con 0x7f183815cef0 2023-11-28T11:11:59.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.039+0000 7f18402aa700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183815d750 con 0x7f183806ba90 2023-11-28T11:11:59.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.039+0000 7f183e046700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f183815cef0 0x7f18380af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.039+0000 7f183e046700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f183815cef0 0x7f18380af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36220/0 (socket says 172.21.15.156:36220) 2023-11-28T11:11:59.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.039+0000 7f183e046700 1 -- 172.21.15.156:0/2897457878 learned_addr learned my addr 172.21.15.156:0/2897457878 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:59.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.040+0000 7f183e847700 1 --2- 172.21.15.156:0/2897457878 >> v2:172.21.15.186:3300/0 conn(0x7f183806ba90 0x7f18380ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.040+0000 7f183e046700 1 -- 172.21.15.156:0/2897457878 >> v2:172.21.15.186:3300/0 conn(0x7f183806ba90 msgr2=0x7f18380ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:59.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.040+0000 7f183e046700 1 --2- 172.21.15.156:0/2897457878 >> v2:172.21.15.186:3300/0 conn(0x7f183806ba90 0x7f18380ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.040+0000 7f183e046700 1 -- 172.21.15.156:0/2897457878 >> v1:172.21.15.186:6789/0 conn(0x7f18380cbdf0 legacy=0x7f183806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:59.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.040+0000 7f183e046700 1 -- 172.21.15.156:0/2897457878 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183815da30 con 0x7f183815cef0 2023-11-28T11:11:59.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.040+0000 7f183e847700 1 --2- 172.21.15.156:0/2897457878 >> v2:172.21.15.186:3300/0 conn(0x7f183806ba90 0x7f18380ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:59.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.040+0000 7f183e046700 1 --2- 172.21.15.156:0/2897457878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f183815cef0 0x7f18380af400 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f18280098f0 tx=0x7f1828009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2347b10fe289d9c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:59.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.040+0000 7f183d044700 1 -- 172.21.15.156:0/2897457878 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828004230 con 0x7f183815cef0 2023-11-28T11:11:59.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.040+0000 7f183d044700 1 -- 172.21.15.156:0/2897457878 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1828004390 con 0x7f183815cef0 2023-11-28T11:11:59.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.040+0000 7f183d044700 1 -- 172.21.15.156:0/2897457878 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280036a0 con 0x7f183815cef0 2023-11-28T11:11:59.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.041+0000 7f18402aa700 1 -- 172.21.15.156:0/2897457878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f183815cef0 msgr2=0x7f18380af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.041+0000 7f18402aa700 1 --2- 172.21.15.156:0/2897457878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f183815cef0 0x7f18380af400 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f18280098f0 tx=0x7f1828009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:59.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.041+0000 7f18402aa700 1 -- 172.21.15.156:0/2897457878 shutdown_connections 2023-11-28T11:11:59.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.041+0000 7f18402aa700 1 --2- 172.21.15.156:0/2897457878 >> v2:172.21.15.186:3300/0 conn(0x7f183806ba90 0x7f18380ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.041+0000 7f18402aa700 1 --2- 172.21.15.156:0/2897457878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f183815cef0 0x7f18380af400 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.041+0000 7f18402aa700 1 -- 172.21.15.156:0/2897457878 >> 172.21.15.156:0/2897457878 conn(0x7f18380d0000 msgr2=0x7f18380d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:59.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.041+0000 7f18402aa700 1 -- 172.21.15.156:0/2897457878 shutdown_connections 2023-11-28T11:11:59.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.041+0000 7f18402aa700 1 -- 172.21.15.156:0/2897457878 wait complete. 2023-11-28T11:11:59.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.042+0000 7f18402aa700 1 Processor -- start 2023-11-28T11:11:59.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.042+0000 7f18402aa700 1 -- start start 2023-11-28T11:11:59.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f18402aa700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f183806ba90 0x7f1838152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f18402aa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18380cbdf0 0x7f1838157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f18402aa700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f183815cef0 0x7f18380c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f18402aa700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381f16f0 con 0x7f183815cef0 2023-11-28T11:11:59.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f18402aa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18380ad360 con 0x7f18380cbdf0 2023-11-28T11:11:59.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f18402aa700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18380ad640 con 0x7f183806ba90 2023-11-28T11:11:59.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183e847700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f183815cef0 0x7f18380c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183e847700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f183815cef0 0x7f18380c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36230/0 (socket says 172.21.15.156:36230) 2023-11-28T11:11:59.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183e847700 1 -- 172.21.15.156:0/1973742226 learned_addr learned my addr 172.21.15.156:0/1973742226 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:59.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183d845700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18380cbdf0 0x7f1838157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183e046700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f183806ba90 0x7f1838152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183e847700 1 -- 172.21.15.156:0/1973742226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f183806ba90 msgr2=0x7f1838152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183e847700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f183806ba90 0x7f1838152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183e847700 1 -- 172.21.15.156:0/1973742226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18380cbdf0 msgr2=0x7f1838157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183e847700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18380cbdf0 0x7f1838157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183e847700 1 -- 172.21.15.156:0/1973742226 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1828009550 con 0x7f183815cef0 2023-11-28T11:11:59.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183d845700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18380cbdf0 0x7f1838157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:59.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183e046700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f183806ba90 0x7f1838152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:59.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f183e847700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f183815cef0 0x7f18380c8610 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f183000ce70 tx=0x7f183000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:59.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f182effd700 1 -- 172.21.15.156:0/1973742226 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183000ae10 con 0x7f183815cef0 2023-11-28T11:11:59.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f182effd700 1 -- 172.21.15.156:0/1973742226 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1830003a20 con 0x7f183815cef0 2023-11-28T11:11:59.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f18402aa700 1 -- 172.21.15.156:0/1973742226 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183815dff0 con 0x7f183815cef0 2023-11-28T11:11:59.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f182effd700 1 -- 172.21.15.156:0/1973742226 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830003dd0 con 0x7f183815cef0 2023-11-28T11:11:59.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.043+0000 7f18402aa700 1 -- 172.21.15.156:0/1973742226 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183815dd10 con 0x7f183815cef0 2023-11-28T11:11:59.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.044+0000 7f18402aa700 1 -- 172.21.15.156:0/1973742226 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18380cab90 con 0x7f183815cef0 2023-11-28T11:11:59.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.045+0000 7f182effd700 1 -- 172.21.15.156:0/1973742226 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1830003f30 con 0x7f183815cef0 2023-11-28T11:11:59.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.045+0000 7f182effd700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f181c0682f0 0x7f181c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.045+0000 7f182effd700 1 -- 172.21.15.156:0/1973742226 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f18300862a0 con 0x7f183815cef0 2023-11-28T11:11:59.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.045+0000 7f183e046700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f181c0682f0 0x7f181c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.046+0000 7f183e046700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f181c0682f0 0x7f181c06a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1828009c40 tx=0x7f1828008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:59.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.049+0000 7f182effd700 1 -- 172.21.15.156:0/1973742226 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f183004d5e0 con 0x7f183815cef0 2023-11-28T11:11:59.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.230+0000 7f18402aa700 1 -- 172.21.15.156:0/1973742226 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f18381f1830 con 0x7f183815cef0 2023-11-28T11:11:59.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.231+0000 7f182effd700 1 -- 172.21.15.156:0/1973742226 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1830052a20 con 0x7f183815cef0 2023-11-28T11:11:59.232 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:11:59.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.233+0000 7f18402aa700 1 -- 172.21.15.156:0/1973742226 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f181c0682f0 msgr2=0x7f181c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.233+0000 7f18402aa700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f181c0682f0 0x7f181c06a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1828009c40 tx=0x7f1828008030 comp rx=0 tx=0).stop 2023-11-28T11:11:59.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.233+0000 7f18402aa700 1 -- 172.21.15.156:0/1973742226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f183815cef0 msgr2=0x7f18380c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.233+0000 7f18402aa700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f183815cef0 0x7f18380c8610 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f183000ce70 tx=0x7f183000cf50 comp rx=0 tx=0).stop 2023-11-28T11:11:59.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.233+0000 7f18402aa700 1 -- 172.21.15.156:0/1973742226 shutdown_connections 2023-11-28T11:11:59.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.233+0000 7f18402aa700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f183815cef0 0x7f18380c8610 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.233+0000 7f18402aa700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f181c0682f0 0x7f181c06a7a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.233+0000 7f18402aa700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f183806ba90 0x7f1838152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.233+0000 7f18402aa700 1 --2- 172.21.15.156:0/1973742226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18380cbdf0 0x7f1838157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.233+0000 7f18402aa700 1 -- 172.21.15.156:0/1973742226 >> 172.21.15.156:0/1973742226 conn(0x7f18380d0000 msgr2=0x7f183806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:59.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.233+0000 7f18402aa700 1 -- 172.21.15.156:0/1973742226 shutdown_connections 2023-11-28T11:11:59.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.233+0000 7f18402aa700 1 -- 172.21.15.156:0/1973742226 wait complete. 2023-11-28T11:11:59.246 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:11:59.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.368+0000 7fd1846b3700 1 Processor -- start 2023-11-28T11:11:59.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.368+0000 7fd1846b3700 1 -- start start 2023-11-28T11:11:59.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.369+0000 7fd1846b3700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd17c06ba90 0x7fd17c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.369+0000 7fd1846b3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c0cbdf0 0x7fd17c0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.369+0000 7fd1846b3700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17c0cc5f0 con 0x7fd17c15cef0 2023-11-28T11:11:59.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.369+0000 7fd1846b3700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17c15d470 con 0x7fd17c06ba90 2023-11-28T11:11:59.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.369+0000 7fd1846b3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c15d750 con 0x7fd17c0cbdf0 2023-11-28T11:11:59.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.369+0000 7fd18244f700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd17c15cef0 0x7fd17c06b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51112/0 (socket says 172.21.15.156:51112) 2023-11-28T11:11:59.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.369+0000 7fd18244f700 1 -- 172.21.15.156:0/3222870408 learned_addr learned my addr 172.21.15.156:0/3222870408 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:11:59.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.369+0000 7fd181c4e700 1 --2- 172.21.15.156:0/3222870408 >> v2:172.21.15.186:3300/0 conn(0x7fd17c06ba90 0x7fd17c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.369+0000 7fd182c50700 1 --2- 172.21.15.156:0/3222870408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c0cbdf0 0x7fd17c0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.369+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523168085 0 0) 0x7fd17c0cc5f0 con 0x7fd17c15cef0 2023-11-28T11:11:59.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.369+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1680035b0 con 0x7fd17c15cef0 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1761068969 0 0) 0x7fd1680035b0 con 0x7fd17c15cef0 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c0cbdf0 msgr2=0x7fd17c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 --2- 172.21.15.156:0/3222870408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c0cbdf0 0x7fd17c0ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 >> v2:172.21.15.186:3300/0 conn(0x7fd17c06ba90 msgr2=0x7fd17c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 --2- 172.21.15.156:0/3222870408 >> v2:172.21.15.186:3300/0 conn(0x7fd17c06ba90 0x7fd17c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c15da30 con 0x7fd17c15cef0 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fd16c001670 con 0x7fd17c15cef0 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 >> v1:172.21.15.156:6789/0 conn(0x7fd17c15cef0 legacy=0x7fd17c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 --2- 172.21.15.156:0/3222870408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168003a00 0x7fd168003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 --2- 172.21.15.156:0/3222870408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c0cbdf0 0x7fd168004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 --2- 172.21.15.156:0/3222870408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd168004d90 0x7fd168005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c15d470 con 0x7fd168004d90 2023-11-28T11:11:59.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c15d750 con 0x7fd168003a00 2023-11-28T11:11:59.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16800ba10 con 0x7fd17c0cbdf0 2023-11-28T11:11:59.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd182c50700 1 --2- 172.21.15.156:0/3222870408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd168004d90 0x7fd168005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18244f700 1 --2- 172.21.15.156:0/3222870408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168003a00 0x7fd168003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd181c4e700 1 --2- 172.21.15.156:0/3222870408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c0cbdf0 0x7fd168004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd182c50700 1 -- 172.21.15.156:0/3222870408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c0cbdf0 msgr2=0x7fd168004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:59.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd182c50700 1 --2- 172.21.15.156:0/3222870408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c0cbdf0 0x7fd168004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd182c50700 1 -- 172.21.15.156:0/3222870408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168003a00 msgr2=0x7fd168003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:59.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd182c50700 1 --2- 172.21.15.156:0/3222870408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168003a00 0x7fd168003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd182c50700 1 -- 172.21.15.156:0/3222870408 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1680035b0 con 0x7fd168004d90 2023-11-28T11:11:59.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd182c50700 1 --2- 172.21.15.156:0/3222870408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd168004d90 0x7fd168005200 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd174009fc0 tx=0x7fd168005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d073cfeb04e2c13d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:59.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17400ebd0 con 0x7fd168004d90 2023-11-28T11:11:59.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17400f040 con 0x7fd168004d90 2023-11-28T11:11:59.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.370+0000 7fd18144d700 1 -- 172.21.15.156:0/3222870408 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174014480 con 0x7fd168004d90 2023-11-28T11:11:59.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.371+0000 7fd1846b3700 1 -- 172.21.15.156:0/3222870408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd168004d90 msgr2=0x7fd168005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.371+0000 7fd1846b3700 1 --2- 172.21.15.156:0/3222870408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd168004d90 0x7fd168005200 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd174009fc0 tx=0x7fd168005880 comp rx=0 tx=0).stop 2023-11-28T11:11:59.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.371+0000 7fd182c50700 1 -- 172.21.15.156:0/3222870408 reap_dead start 2023-11-28T11:11:59.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.371+0000 7fd1846b3700 1 -- 172.21.15.156:0/3222870408 shutdown_connections 2023-11-28T11:11:59.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.371+0000 7fd1846b3700 1 -- 172.21.15.156:0/3222870408 >> 172.21.15.156:0/3222870408 conn(0x7fd17c0d0000 msgr2=0x7fd17c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:59.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.371+0000 7fd1846b3700 1 -- 172.21.15.156:0/3222870408 shutdown_connections 2023-11-28T11:11:59.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.371+0000 7fd1846b3700 1 -- 172.21.15.156:0/3222870408 wait complete. 2023-11-28T11:11:59.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.372+0000 7fd1846b3700 1 Processor -- start 2023-11-28T11:11:59.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.372+0000 7fd1846b3700 1 -- start start 2023-11-28T11:11:59.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd1846b3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168004d90 0x7fd17c0c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd1846b3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c14b870 0x7fd17c0c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd1846b3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c159dc0 0x7fd17c15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd1846b3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0c6410 con 0x7fd17c159dc0 2023-11-28T11:11:59.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd1846b3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c15a770 con 0x7fd168004d90 2023-11-28T11:11:59.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd1846b3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0c5790 con 0x7fd17c14b870 2023-11-28T11:11:59.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd182c50700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c159dc0 0x7fd17c15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd182c50700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c159dc0 0x7fd17c15a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36250/0 (socket says 172.21.15.156:36250) 2023-11-28T11:11:59.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd182c50700 1 -- 172.21.15.156:0/4258327274 learned_addr learned my addr 172.21.15.156:0/4258327274 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd18244f700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168004d90 0x7fd17c0c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd181c4e700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c14b870 0x7fd17c0c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd182c50700 1 -- 172.21.15.156:0/4258327274 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c14b870 msgr2=0x7fd17c0c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd182c50700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c14b870 0x7fd17c0c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd182c50700 1 -- 172.21.15.156:0/4258327274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168004d90 msgr2=0x7fd17c0c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd182c50700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168004d90 0x7fd17c0c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd182c50700 1 -- 172.21.15.156:0/4258327274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c15dff0 con 0x7fd17c159dc0 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.373+0000 7fd182c50700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c159dc0 0x7fd17c15a230 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd174016000 tx=0x7fd174009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.374+0000 7fd172ffd700 1 -- 172.21.15.156:0/4258327274 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174007000 con 0x7fd17c159dc0 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.374+0000 7fd172ffd700 1 -- 172.21.15.156:0/4258327274 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd174014890 con 0x7fd17c159dc0 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.374+0000 7fd172ffd700 1 -- 172.21.15.156:0/4258327274 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174014c40 con 0x7fd17c159dc0 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.374+0000 7fd1846b3700 1 -- 172.21.15.156:0/4258327274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c15dd10 con 0x7fd17c159dc0 2023-11-28T11:11:59.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.374+0000 7fd1846b3700 1 -- 172.21.15.156:0/4258327274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c06a270 con 0x7fd17c159dc0 2023-11-28T11:11:59.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.375+0000 7fd1846b3700 1 -- 172.21.15.156:0/4258327274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17c0c6550 con 0x7fd17c159dc0 2023-11-28T11:11:59.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.375+0000 7fd172ffd700 1 -- 172.21.15.156:0/4258327274 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd17401e430 con 0x7fd17c159dc0 2023-11-28T11:11:59.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.376+0000 7fd172ffd700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd160068280 0x7fd16006a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.376+0000 7fd172ffd700 1 -- 172.21.15.156:0/4258327274 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fd174087a60 con 0x7fd17c159dc0 2023-11-28T11:11:59.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.376+0000 7fd18244f700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd160068280 0x7fd16006a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.376+0000 7fd18244f700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd160068280 0x7fd16006a730 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd16c001590 tx=0x7fd16c007980 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:59.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.380+0000 7fd172ffd700 1 -- 172.21.15.156:0/4258327274 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd17c0c6550 con 0x7fd17c159dc0 2023-11-28T11:11:59.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.561+0000 7fd1846b3700 1 -- 172.21.15.156:0/4258327274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd17c0c6550 con 0x7fd17c159dc0 2023-11-28T11:11:59.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.561+0000 7fd172ffd700 1 -- 172.21.15.156:0/4258327274 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd17c0c6550 con 0x7fd17c159dc0 2023-11-28T11:11:59.563 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:11:59.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.563+0000 7fd1846b3700 1 -- 172.21.15.156:0/4258327274 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd160068280 msgr2=0x7fd16006a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.563+0000 7fd1846b3700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd160068280 0x7fd16006a730 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd16c001590 tx=0x7fd16c007980 comp rx=0 tx=0).stop 2023-11-28T11:11:59.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.563+0000 7fd1846b3700 1 -- 172.21.15.156:0/4258327274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c159dc0 msgr2=0x7fd17c15a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.563+0000 7fd1846b3700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c159dc0 0x7fd17c15a230 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd174016000 tx=0x7fd174009750 comp rx=0 tx=0).stop 2023-11-28T11:11:59.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.563+0000 7fd1846b3700 1 -- 172.21.15.156:0/4258327274 shutdown_connections 2023-11-28T11:11:59.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.563+0000 7fd1846b3700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c159dc0 0x7fd17c15a230 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.563+0000 7fd1846b3700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd168004d90 0x7fd17c0c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.563+0000 7fd1846b3700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd160068280 0x7fd16006a730 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.563+0000 7fd1846b3700 1 --2- 172.21.15.156:0/4258327274 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd17c14b870 0x7fd17c0c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.563+0000 7fd1846b3700 1 -- 172.21.15.156:0/4258327274 >> 172.21.15.156:0/4258327274 conn(0x7fd17c0d0000 msgr2=0x7fd17c0b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:59.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.564+0000 7fd1846b3700 1 -- 172.21.15.156:0/4258327274 shutdown_connections 2023-11-28T11:11:59.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.564+0000 7fd1846b3700 1 -- 172.21.15.156:0/4258327274 wait complete. 2023-11-28T11:11:59.577 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:11:59.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.699+0000 7f3bd714c700 1 Processor -- start 2023-11-28T11:11:59.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.699+0000 7f3bd714c700 1 -- start start 2023-11-28T11:11:59.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.699+0000 7f3bd714c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bd006dee0 0x7f3bd006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.699+0000 7f3bd714c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3bd0160540 0x7f3bd01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.699+0000 7f3bd714c700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd015d060 con 0x7f3bd0071040 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.699+0000 7f3bd714c700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd015d340 con 0x7f3bd0160540 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.699+0000 7f3bd714c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd015d620 con 0x7f3bd006dee0 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.700+0000 7f3bcffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bd006dee0 0x7f3bd006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.700+0000 7f3bd56e9700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3bd0160540 0x7f3bd01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.700+0000 7f3bcffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bd006dee0 0x7f3bd006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:40120/0 (socket says 172.21.15.156:40120) 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.700+0000 7f3bcffff700 1 -- 172.21.15.156:0/348719060 learned_addr learned my addr 172.21.15.156:0/348719060 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.700+0000 7f3bd4ee8700 1 --1- 172.21.15.156:0/348719060 >> v1:172.21.15.186:6789/0 conn(0x7f3bd0071040 0x7f3bd0071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:33260/0 (socket says 172.21.15.156:33260) 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.700+0000 7f3bcffff700 1 -- 172.21.15.156:0/348719060 >> v2:172.21.15.186:3300/0 conn(0x7f3bd0160540 msgr2=0x7f3bd01609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.700+0000 7f3bcffff700 1 --2- 172.21.15.156:0/348719060 >> v2:172.21.15.186:3300/0 conn(0x7f3bd0160540 0x7f3bd01609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.700+0000 7f3bcffff700 1 -- 172.21.15.156:0/348719060 >> v1:172.21.15.186:6789/0 conn(0x7f3bd0071040 legacy=0x7f3bd0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.700+0000 7f3bcffff700 1 -- 172.21.15.156:0/348719060 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd015d900 con 0x7f3bd006dee0 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.700+0000 7f3bd56e9700 1 --2- 172.21.15.156:0/348719060 >> v2:172.21.15.186:3300/0 conn(0x7f3bd0160540 0x7f3bd01609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:59.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.700+0000 7f3bcffff700 1 --2- 172.21.15.156:0/348719060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bd006dee0 0x7f3bd006e330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0098f0 tx=0x7f3bbc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26193d99ee0613f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:59.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.700+0000 7f3bcf7fe700 1 -- 172.21.15.156:0/348719060 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc011830 con 0x7f3bd006dee0 2023-11-28T11:11:59.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.701+0000 7f3bcf7fe700 1 -- 172.21.15.156:0/348719060 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc011990 con 0x7f3bd006dee0 2023-11-28T11:11:59.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.701+0000 7f3bcf7fe700 1 -- 172.21.15.156:0/348719060 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc011d20 con 0x7f3bd006dee0 2023-11-28T11:11:59.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.701+0000 7f3bd714c700 1 -- 172.21.15.156:0/348719060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bd006dee0 msgr2=0x7f3bd006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.701+0000 7f3bd714c700 1 --2- 172.21.15.156:0/348719060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bd006dee0 0x7f3bd006e330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0098f0 tx=0x7f3bbc009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:59.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.701+0000 7f3bd714c700 1 -- 172.21.15.156:0/348719060 shutdown_connections 2023-11-28T11:11:59.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.701+0000 7f3bd714c700 1 --2- 172.21.15.156:0/348719060 >> v2:172.21.15.186:3300/0 conn(0x7f3bd0160540 0x7f3bd01609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.701+0000 7f3bd714c700 1 --2- 172.21.15.156:0/348719060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bd006dee0 0x7f3bd006e330 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.701+0000 7f3bd714c700 1 -- 172.21.15.156:0/348719060 >> 172.21.15.156:0/348719060 conn(0x7f3bd00d0000 msgr2=0x7f3bd00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:59.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.701+0000 7f3bd714c700 1 -- 172.21.15.156:0/348719060 shutdown_connections 2023-11-28T11:11:59.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.701+0000 7f3bd714c700 1 -- 172.21.15.156:0/348719060 wait complete. 2023-11-28T11:11:59.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.702+0000 7f3bd714c700 1 Processor -- start 2023-11-28T11:11:59.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.702+0000 7f3bd714c700 1 -- start start 2023-11-28T11:11:59.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.703+0000 7f3bd714c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bd006dee0 0x7f3bd01f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.703+0000 7f3bd714c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bd0071040 0x7f3bd01538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.703+0000 7f3bd714c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bd0160540 0x7f3bd00b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.703+0000 7f3bd714c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00c52a0 con 0x7f3bd0071040 2023-11-28T11:11:59.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.703+0000 7f3bd714c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd01f6f70 con 0x7f3bd006dee0 2023-11-28T11:11:59.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.703+0000 7f3bd714c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd0153e30 con 0x7f3bd0160540 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.703+0000 7f3bcffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bd0071040 0x7f3bd01538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.703+0000 7f3bcffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bd0071040 0x7f3bd01538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36262/0 (socket says 172.21.15.156:36262) 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.703+0000 7f3bcffff700 1 -- 172.21.15.156:0/2428365718 learned_addr learned my addr 172.21.15.156:0/2428365718 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bd56e9700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bd0160540 0x7f3bd00b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bd4ee8700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bd006dee0 0x7f3bd01f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bcffff700 1 -- 172.21.15.156:0/2428365718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bd0160540 msgr2=0x7f3bd00b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bcffff700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bd0160540 0x7f3bd00b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bcffff700 1 -- 172.21.15.156:0/2428365718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bd006dee0 msgr2=0x7f3bd01f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bcffff700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bd006dee0 0x7f3bd01f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bcffff700 1 -- 172.21.15.156:0/2428365718 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8009a70 con 0x7f3bd0071040 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bcffff700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bd0071040 0x7f3bd01538f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc014040 tx=0x7f3bbc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bcd7fa700 1 -- 172.21.15.156:0/2428365718 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0036a0 con 0x7f3bd0071040 2023-11-28T11:11:59.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bcd7fa700 1 -- 172.21.15.156:0/2428365718 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc01e070 con 0x7f3bd0071040 2023-11-28T11:11:59.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bd714c700 1 -- 172.21.15.156:0/2428365718 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc009550 con 0x7f3bd0071040 2023-11-28T11:11:59.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bcd7fa700 1 -- 172.21.15.156:0/2428365718 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc003a40 con 0x7f3bd0071040 2023-11-28T11:11:59.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.704+0000 7f3bd714c700 1 -- 172.21.15.156:0/2428365718 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd015ccd0 con 0x7f3bd0071040 2023-11-28T11:11:59.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.705+0000 7f3bd714c700 1 -- 172.21.15.156:0/2428365718 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd006e870 con 0x7f3bd0071040 2023-11-28T11:11:59.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.705+0000 7f3bcd7fa700 1 -- 172.21.15.156:0/2428365718 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3bbc00f050 con 0x7f3bd0071040 2023-11-28T11:11:59.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.706+0000 7f3bcd7fa700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3bb40682f0 0x7f3bb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.706+0000 7f3bcd7fa700 1 -- 172.21.15.156:0/2428365718 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f3bbc086cf0 con 0x7f3bd0071040 2023-11-28T11:11:59.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.706+0000 7f3bd4ee8700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3bb40682f0 0x7f3bb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.706+0000 7f3bd4ee8700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3bb40682f0 0x7f3bb406a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4000f60 tx=0x7f3bc4007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:59.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.710+0000 7f3bcd7fa700 1 -- 172.21.15.156:0/2428365718 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3bbc053450 con 0x7f3bd0071040 2023-11-28T11:11:59.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.893+0000 7f3bd714c700 1 -- 172.21.15.156:0/2428365718 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3bd006a270 con 0x7f3bd0071040 2023-11-28T11:11:59.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.894+0000 7f3bcd7fa700 1 -- 172.21.15.156:0/2428365718 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3bbc0150a0 con 0x7f3bd0071040 2023-11-28T11:11:59.895 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:11:59.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.896+0000 7f3bd714c700 1 -- 172.21.15.156:0/2428365718 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3bb40682f0 msgr2=0x7f3bb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.896+0000 7f3bd714c700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3bb40682f0 0x7f3bb406a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4000f60 tx=0x7f3bc4007d10 comp rx=0 tx=0).stop 2023-11-28T11:11:59.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.896+0000 7f3bd714c700 1 -- 172.21.15.156:0/2428365718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bd0071040 msgr2=0x7f3bd01538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.896+0000 7f3bd714c700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bd0071040 0x7f3bd01538f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc014040 tx=0x7f3bbc004750 comp rx=0 tx=0).stop 2023-11-28T11:11:59.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.896+0000 7f3bd714c700 1 -- 172.21.15.156:0/2428365718 shutdown_connections 2023-11-28T11:11:59.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.896+0000 7f3bd714c700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3bb40682f0 0x7f3bb406a7a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.896+0000 7f3bd714c700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bd0160540 0x7f3bd00b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.896+0000 7f3bd714c700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bd006dee0 0x7f3bd01f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.896+0000 7f3bd714c700 1 --2- 172.21.15.156:0/2428365718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bd0071040 0x7f3bd01538f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.896+0000 7f3bd714c700 1 -- 172.21.15.156:0/2428365718 >> 172.21.15.156:0/2428365718 conn(0x7f3bd00d0000 msgr2=0x7f3bd00aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:59.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.896+0000 7f3bd714c700 1 -- 172.21.15.156:0/2428365718 shutdown_connections 2023-11-28T11:11:59.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:11:59.896+0000 7f3bd714c700 1 -- 172.21.15.156:0/2428365718 wait complete. 2023-11-28T11:11:59.910 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:12:00.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.032+0000 7f75e764c700 1 Processor -- start 2023-11-28T11:12:00.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.032+0000 7f75e764c700 1 -- start start 2023-11-28T11:12:00.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e764c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75e0071040 0x7f75e0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e764c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f75e0160540 0x7f75e01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e764c700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e0071950 con 0x7f75e006dee0 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e764c700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e015d060 con 0x7f75e0160540 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e764c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e015d340 con 0x7f75e0071040 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e5be9700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f75e0160540 0x7f75e01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e5be9700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f75e0160540 0x7f75e01609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36270/0 (socket says 172.21.15.156:36270) 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e5be9700 1 -- 172.21.15.156:0/2562326433 learned_addr learned my addr 172.21.15.156:0/2562326433 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e53e8700 1 --2- 172.21.15.156:0/2562326433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75e0071040 0x7f75e0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e5be9700 1 -- 172.21.15.156:0/2562326433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75e0071040 msgr2=0x7f75e0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e5be9700 1 --2- 172.21.15.156:0/2562326433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75e0071040 0x7f75e0071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e5be9700 1 -- 172.21.15.156:0/2562326433 >> v1:172.21.15.186:6789/0 conn(0x7f75e006dee0 legacy=0x7f75e006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e5be9700 1 -- 172.21.15.156:0/2562326433 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e015d620 con 0x7f75e0160540 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75e5be9700 1 --2- 172.21.15.156:0/2562326433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0160540 0x7f75e01609c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009e50 tx=0x7f75d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3320e54f1cda6d94 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75d7fff700 1 -- 172.21.15.156:0/2562326433 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d800e070 con 0x7f75e0160540 2023-11-28T11:12:00.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.033+0000 7f75d7fff700 1 -- 172.21.15.156:0/2562326433 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d8003a20 con 0x7f75e0160540 2023-11-28T11:12:00.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.034+0000 7f75d7fff700 1 -- 172.21.15.156:0/2562326433 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8003dd0 con 0x7f75e0160540 2023-11-28T11:12:00.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.034+0000 7f75e764c700 1 -- 172.21.15.156:0/2562326433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0160540 msgr2=0x7f75e01609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.034+0000 7f75e764c700 1 --2- 172.21.15.156:0/2562326433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0160540 0x7f75e01609c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009e50 tx=0x7f75d800b330 comp rx=0 tx=0).stop 2023-11-28T11:12:00.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.034+0000 7f75e764c700 1 -- 172.21.15.156:0/2562326433 shutdown_connections 2023-11-28T11:12:00.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.034+0000 7f75e764c700 1 --2- 172.21.15.156:0/2562326433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0160540 0x7f75e01609c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.034+0000 7f75e764c700 1 --2- 172.21.15.156:0/2562326433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75e0071040 0x7f75e0071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.034+0000 7f75e764c700 1 -- 172.21.15.156:0/2562326433 >> 172.21.15.156:0/2562326433 conn(0x7f75e00d0000 msgr2=0x7f75e00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:00.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.034+0000 7f75e764c700 1 -- 172.21.15.156:0/2562326433 shutdown_connections 2023-11-28T11:12:00.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.034+0000 7f75e764c700 1 -- 172.21.15.156:0/2562326433 wait complete. 2023-11-28T11:12:00.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.035+0000 7f75e764c700 1 Processor -- start 2023-11-28T11:12:00.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.035+0000 7f75e764c700 1 -- start start 2023-11-28T11:12:00.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e764c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75e006dee0 0x7f75e00c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e764c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0071040 0x7f75e00b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e764c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75e0160540 0x7f75e01f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e764c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00c6650 con 0x7f75e0071040 2023-11-28T11:12:00.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e764c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00c5a40 con 0x7f75e006dee0 2023-11-28T11:12:00.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e764c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00c5d20 con 0x7f75e0160540 2023-11-28T11:12:00.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e4be7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0071040 0x7f75e00b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e4be7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0071040 0x7f75e00b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36274/0 (socket says 172.21.15.156:36274) 2023-11-28T11:12:00.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e4be7700 1 -- 172.21.15.156:0/3000766325 learned_addr learned my addr 172.21.15.156:0/3000766325 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:00.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e53e8700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75e006dee0 0x7f75e00c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e5be9700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75e0160540 0x7f75e01f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e4be7700 1 -- 172.21.15.156:0/3000766325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75e0160540 msgr2=0x7f75e01f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e4be7700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75e0160540 0x7f75e01f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e4be7700 1 -- 172.21.15.156:0/3000766325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75e006dee0 msgr2=0x7f75e00c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e4be7700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75e006dee0 0x7f75e00c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.036+0000 7f75e4be7700 1 -- 172.21.15.156:0/3000766325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d8009b30 con 0x7f75e0071040 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.037+0000 7f75e4be7700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0071040 0x7f75e00b4270 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f75d000b4a0 tx=0x7f75d000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.037+0000 7f75d5ffb700 1 -- 172.21.15.156:0/3000766325 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0011830 con 0x7f75e0071040 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.037+0000 7f75d5ffb700 1 -- 172.21.15.156:0/3000766325 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d0011990 con 0x7f75e0071040 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.037+0000 7f75e764c700 1 -- 172.21.15.156:0/3000766325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e015ccd0 con 0x7f75e0071040 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.037+0000 7f75d5ffb700 1 -- 172.21.15.156:0/3000766325 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0011d40 con 0x7f75e0071040 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.037+0000 7f75e764c700 1 -- 172.21.15.156:0/3000766325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e015d900 con 0x7f75e0071040 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.038+0000 7f75e764c700 1 -- 172.21.15.156:0/3000766325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e006a270 con 0x7f75e0071040 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.038+0000 7f75d5ffb700 1 -- 172.21.15.156:0/3000766325 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f75d000f050 con 0x7f75e0071040 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.039+0000 7f75d5ffb700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f75c40682f0 0x7f75c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.039+0000 7f75d5ffb700 1 -- 172.21.15.156:0/3000766325 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f75d0085890 con 0x7f75e0071040 2023-11-28T11:12:00.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.039+0000 7f75e53e8700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f75c40682f0 0x7f75c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.039+0000 7f75e53e8700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f75c40682f0 0x7f75c406a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00a950 tx=0x7f75dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.043+0000 7f75d5ffb700 1 -- 172.21.15.156:0/3000766325 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f75d0052010 con 0x7f75e0071040 2023-11-28T11:12:00.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.241+0000 7f75e764c700 1 -- 172.21.15.156:0/3000766325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f75e00d0680 con 0x7f75e0071040 2023-11-28T11:12:00.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.241+0000 7f75d5ffb700 1 -- 172.21.15.156:0/3000766325 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7f75d0055630 con 0x7f75e0071040 2023-11-28T11:12:00.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.243+0000 7f75e764c700 1 -- 172.21.15.156:0/3000766325 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f75c40682f0 msgr2=0x7f75c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.243+0000 7f75e764c700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f75c40682f0 0x7f75c406a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00a950 tx=0x7f75dc008040 comp rx=0 tx=0).stop 2023-11-28T11:12:00.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.243+0000 7f75e764c700 1 -- 172.21.15.156:0/3000766325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0071040 msgr2=0x7f75e00b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.243+0000 7f75e764c700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0071040 0x7f75e00b4270 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f75d000b4a0 tx=0x7f75d000b860 comp rx=0 tx=0).stop 2023-11-28T11:12:00.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.243+0000 7f75e764c700 1 -- 172.21.15.156:0/3000766325 shutdown_connections 2023-11-28T11:12:00.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.243+0000 7f75e764c700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f75c40682f0 0x7f75c406a7a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.243+0000 7f75e764c700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75e0160540 0x7f75e01f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.243+0000 7f75e764c700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75e006dee0 0x7f75e00c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.243+0000 7f75e764c700 1 --2- 172.21.15.156:0/3000766325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0071040 0x7f75e00b4270 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.243+0000 7f75e764c700 1 -- 172.21.15.156:0/3000766325 >> 172.21.15.156:0/3000766325 conn(0x7f75e00d0000 msgr2=0x7f75e00ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:00.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.243+0000 7f75e764c700 1 -- 172.21.15.156:0/3000766325 shutdown_connections 2023-11-28T11:12:00.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.243+0000 7f75e764c700 1 -- 172.21.15.156:0/3000766325 wait complete. 2023-11-28T11:12:00.246 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T11:12:00.257 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:12:00.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:00.257 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:12:00.332 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:12:00.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:00.332 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:12:00.352 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:12:00.353 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:00.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:12:00.375 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:12:00.375 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:00.410 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2550 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:12:00.410 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:12:00.411 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:00.432 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:12:00.433 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:00.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.572+0000 7f74d2c87700 1 Processor -- start 2023-11-28T11:12:00.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.572+0000 7f74d2c87700 1 -- start start 2023-11-28T11:12:00.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.573+0000 7f74d2c87700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74cc0ce040 0x7f74cc15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.573+0000 7f74d2c87700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74cc06b420 con 0x7f74cc0d0a00 2023-11-28T11:12:00.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.573+0000 7f74d2c87700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74cc06b700 con 0x7f74cc0b2930 2023-11-28T11:12:00.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.573+0000 7f74d2c87700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc06ff60 con 0x7f74cc0ce040 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.573+0000 7f74cbfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f74cc0d0a00 0x7f74cc06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51158/0 (socket says 172.21.15.156:51158) 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.573+0000 7f74cbfff700 1 -- 172.21.15.156:0/15241467 learned_addr learned my addr 172.21.15.156:0/15241467 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.573+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876737824 0 0) 0x7f74cc06b420 con 0x7f74cc0d0a00 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.573+0000 7f74d0a23700 1 --2- 172.21.15.156:0/15241467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74cc0ce040 0x7f74cc15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.573+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74bc0035b0 con 0x7f74cc0d0a00 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1783317674 0 0) 0x7f74bc0035b0 con 0x7f74cc0d0a00 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74cc0ce040 msgr2=0x7f74cc15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 --2- 172.21.15.156:0/15241467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74cc0ce040 0x7f74cc15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 >> v1:172.21.15.186:6789/0 conn(0x7f74cc0b2930 legacy=0x7f74cc06c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc06b9e0 con 0x7f74cc0d0a00 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f74b8001670 con 0x7f74cc0d0a00 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 >> v1:172.21.15.156:6789/0 conn(0x7f74cc0d0a00 legacy=0x7f74cc06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 --2- 172.21.15.156:0/15241467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74bc003a30 0x7f74bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 --2- 172.21.15.156:0/15241467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74bc0043d0 0x7f74bc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 --2- 172.21.15.156:0/15241467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74cc0ce040 0x7f74bc0050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc06ff60 con 0x7f74bc0043d0 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc00ba10 con 0x7f74bc003a30 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc00bba0 con 0x7f74cc0ce040 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cbfff700 1 --2- 172.21.15.156:0/15241467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74bc0043d0 0x7f74bc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74d0a23700 1 --2- 172.21.15.156:0/15241467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74bc003a30 0x7f74bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74d1224700 1 --2- 172.21.15.156:0/15241467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74cc0ce040 0x7f74bc0050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cbfff700 1 -- 172.21.15.156:0/15241467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74cc0ce040 msgr2=0x7f74bc0050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cbfff700 1 --2- 172.21.15.156:0/15241467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74cc0ce040 0x7f74bc0050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cbfff700 1 -- 172.21.15.156:0/15241467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74bc003a30 msgr2=0x7f74bc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cbfff700 1 --2- 172.21.15.156:0/15241467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74bc003a30 0x7f74bc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cbfff700 1 -- 172.21.15.156:0/15241467 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc0035b0 con 0x7f74bc0043d0 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cbfff700 1 --2- 172.21.15.156:0/15241467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74bc0043d0 0x7f74bc004890 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c2f0 tx=0x7f74bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392db3906a199738 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b800ce30 con 0x7f74bc0043d0 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8005960 con 0x7f74bc0043d0 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.574+0000 7f74cb7fe700 1 -- 172.21.15.156:0/15241467 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8005d10 con 0x7f74bc0043d0 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.575+0000 7f74d2c87700 1 -- 172.21.15.156:0/15241467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74bc0043d0 msgr2=0x7f74bc004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.575+0000 7f74d2c87700 1 --2- 172.21.15.156:0/15241467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74bc0043d0 0x7f74bc004890 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c2f0 tx=0x7f74bc005880 comp rx=0 tx=0).stop 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.575+0000 7f74d1224700 1 -- 172.21.15.156:0/15241467 reap_dead start 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.575+0000 7f74d2c87700 1 -- 172.21.15.156:0/15241467 shutdown_connections 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.575+0000 7f74d2c87700 1 -- 172.21.15.156:0/15241467 >> 172.21.15.156:0/15241467 conn(0x7f74cc06aed0 msgr2=0x7f74cc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.575+0000 7f74d2c87700 1 -- 172.21.15.156:0/15241467 shutdown_connections 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.575+0000 7f74d2c87700 1 -- 172.21.15.156:0/15241467 wait complete. 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.576+0000 7f74d2c87700 1 Processor -- start 2023-11-28T11:12:00.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.576+0000 7f74d2c87700 1 -- start start 2023-11-28T11:12:00.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d2c87700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74bc0043d0 0x7f74cc0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d2c87700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74cc0b6100 0x7f74cc0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d2c87700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74cc0ca230 0x7f74cc0ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d2c87700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc0c8400 con 0x7f74cc0ca230 2023-11-28T11:12:00.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d2c87700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc14ce70 con 0x7f74cc0b6100 2023-11-28T11:12:00.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d2c87700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc0cabf0 con 0x7f74bc0043d0 2023-11-28T11:12:00.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d1224700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74cc0ca230 0x7f74cc0ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d1224700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74cc0ca230 0x7f74cc0ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36294/0 (socket says 172.21.15.156:36294) 2023-11-28T11:12:00.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d1224700 1 -- 172.21.15.156:0/3828559765 learned_addr learned my addr 172.21.15.156:0/3828559765 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74cbfff700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74cc0b6100 0x7f74cc0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d0a23700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74bc0043d0 0x7f74cc0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d1224700 1 -- 172.21.15.156:0/3828559765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74bc0043d0 msgr2=0x7f74cc0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d1224700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74bc0043d0 0x7f74cc0ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d1224700 1 -- 172.21.15.156:0/3828559765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74cc0b6100 msgr2=0x7f74cc0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.577+0000 7f74d1224700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74cc0b6100 0x7f74cc0b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.578+0000 7f74d1224700 1 -- 172.21.15.156:0/3828559765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc06bfa0 con 0x7f74cc0ca230 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.578+0000 7f74d0a23700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74bc0043d0 0x7f74cc0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.578+0000 7f74d1224700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74cc0ca230 0x7f74cc0ca6b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0b3980 tx=0x7f74c400d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.578+0000 7f74c97fa700 1 -- 172.21.15.156:0/3828559765 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c400db60 con 0x7f74cc0ca230 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.578+0000 7f74c97fa700 1 -- 172.21.15.156:0/3828559765 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c400dcc0 con 0x7f74cc0ca230 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.578+0000 7f74d2c87700 1 -- 172.21.15.156:0/3828559765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc06bcc0 con 0x7f74cc0ca230 2023-11-28T11:12:00.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.578+0000 7f74c97fa700 1 -- 172.21.15.156:0/3828559765 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4003a00 con 0x7f74cc0ca230 2023-11-28T11:12:00.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.578+0000 7f74d2c87700 1 -- 172.21.15.156:0/3828559765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc15df10 con 0x7f74cc0ca230 2023-11-28T11:12:00.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.579+0000 7f74d2c87700 1 -- 172.21.15.156:0/3828559765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc0c8540 con 0x7f74cc0ca230 2023-11-28T11:12:00.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.579+0000 7f74c97fa700 1 -- 172.21.15.156:0/3828559765 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f74c4003b80 con 0x7f74cc0ca230 2023-11-28T11:12:00.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.580+0000 7f74c97fa700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74b00682f0 0x7f74b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.580+0000 7f74d0a23700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74b00682f0 0x7f74b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.580+0000 7f74c97fa700 1 -- 172.21.15.156:0/3828559765 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f74c40862c0 con 0x7f74cc0ca230 2023-11-28T11:12:00.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.580+0000 7f74d0a23700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74b00682f0 0x7f74b006a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009bc0 tx=0x7f74c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.584+0000 7f74c97fa700 1 -- 172.21.15.156:0/3828559765 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f74cc0c8540 con 0x7f74cc0ca230 2023-11-28T11:12:00.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.810+0000 7f74d2c87700 1 -- 172.21.15.156:0/3828559765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74cc0c8540 con 0x7f74cc0ca230 2023-11-28T11:12:00.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.811+0000 7f74c97fa700 1 -- 172.21.15.156:0/3828559765 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f74cc0c8540 con 0x7f74cc0ca230 2023-11-28T11:12:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:00.812 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6006,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1991065152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1991065152},{"type":"v1","addr":"172.21.15.156:6836","nonce":1991065152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/43543030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":43543030},{"type":"v1","addr":"172.21.15.156:6837","nonce":43543030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/2480189069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2480189069},{"type":"v1","addr":"172.21.15.186:6834","nonce":2480189069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:52.367543+0000","modified":"2023-11-28T11:11:08.808905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6021":{"gid":6021,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6835/3811791267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3811791267},{"type":"v1","addr":"172.21.15.186:6835","nonce":3811791267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:00.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.813+0000 7f74d2c87700 1 -- 172.21.15.156:0/3828559765 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74b00682f0 msgr2=0x7f74b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.813+0000 7f74d2c87700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74b00682f0 0x7f74b006a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009bc0 tx=0x7f74c0008040 comp rx=0 tx=0).stop 2023-11-28T11:12:00.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.813+0000 7f74d2c87700 1 -- 172.21.15.156:0/3828559765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74cc0ca230 msgr2=0x7f74cc0ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.813+0000 7f74d2c87700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74cc0ca230 0x7f74cc0ca6b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0b3980 tx=0x7f74c400d660 comp rx=0 tx=0).stop 2023-11-28T11:12:00.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.813+0000 7f74d2c87700 1 -- 172.21.15.156:0/3828559765 shutdown_connections 2023-11-28T11:12:00.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.813+0000 7f74d2c87700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74cc0ca230 0x7f74cc0ca6b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.813+0000 7f74d2c87700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74b00682f0 0x7f74b006a7a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.813+0000 7f74d2c87700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74bc0043d0 0x7f74cc0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.813+0000 7f74d2c87700 1 --2- 172.21.15.156:0/3828559765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74cc0b6100 0x7f74cc0b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.813+0000 7f74d2c87700 1 -- 172.21.15.156:0/3828559765 >> 172.21.15.156:0/3828559765 conn(0x7f74cc06aed0 msgr2=0x7f74cc15c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:00.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.813+0000 7f74d2c87700 1 -- 172.21.15.156:0/3828559765 shutdown_connections 2023-11-28T11:12:00.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.813+0000 7f74d2c87700 1 -- 172.21.15.156:0/3828559765 wait complete. 2023-11-28T11:12:00.815 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 32 2023-11-28T11:12:00.826 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:00.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.945+0000 7f45cee55700 1 Processor -- start 2023-11-28T11:12:00.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.945+0000 7f45cee55700 1 -- start start 2023-11-28T11:12:00.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.945+0000 7f45cee55700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c806c990 0x7f45c806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.945+0000 7f45cee55700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f45c80ca240 0x7f45c80ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.945+0000 7f45cee55700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c806d320 con 0x7f45c806e9f0 2023-11-28T11:12:00.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45cee55700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c806ae60 con 0x7f45c80ca240 2023-11-28T11:12:00.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45cee55700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c806a270 con 0x7f45c806c990 2023-11-28T11:12:00.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45cd3f2700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f45c80ca240 0x7f45c80ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45cd3f2700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f45c80ca240 0x7f45c80ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36302/0 (socket says 172.21.15.156:36302) 2023-11-28T11:12:00.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45ccbf1700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f45c806e9f0 0x7f45c806b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51176/0 (socket says 172.21.15.156:51176) 2023-11-28T11:12:00.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45cd3f2700 1 -- 172.21.15.156:0/4105752030 learned_addr learned my addr 172.21.15.156:0/4105752030 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:00.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45c7fff700 1 --2- 172.21.15.156:0/4105752030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c806c990 0x7f45c806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45cd3f2700 1 -- 172.21.15.156:0/4105752030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c806c990 msgr2=0x7f45c806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45cd3f2700 1 --2- 172.21.15.156:0/4105752030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c806c990 0x7f45c806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45cd3f2700 1 -- 172.21.15.156:0/4105752030 >> v1:172.21.15.156:6789/0 conn(0x7f45c806e9f0 legacy=0x7f45c806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45cd3f2700 1 -- 172.21.15.156:0/4105752030 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c80cabe0 con 0x7f45c80ca240 2023-11-28T11:12:00.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45c7fff700 1 --2- 172.21.15.156:0/4105752030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c806c990 0x7f45c806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:00.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.946+0000 7f45cd3f2700 1 --2- 172.21.15.156:0/4105752030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c80ca240 0x7f45c80ca6a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009e50 tx=0x7f45c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f38b7970c5a0729 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.947+0000 7f45c77fe700 1 -- 172.21.15.156:0/4105752030 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c000e070 con 0x7f45c80ca240 2023-11-28T11:12:00.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.947+0000 7f45c77fe700 1 -- 172.21.15.156:0/4105752030 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c000b9c0 con 0x7f45c80ca240 2023-11-28T11:12:00.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.947+0000 7f45c77fe700 1 -- 172.21.15.156:0/4105752030 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c000bd70 con 0x7f45c80ca240 2023-11-28T11:12:00.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.948+0000 7f45cee55700 1 -- 172.21.15.156:0/4105752030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c80ca240 msgr2=0x7f45c80ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.948+0000 7f45cee55700 1 --2- 172.21.15.156:0/4105752030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c80ca240 0x7f45c80ca6a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009e50 tx=0x7f45c000b330 comp rx=0 tx=0).stop 2023-11-28T11:12:00.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.948+0000 7f45cee55700 1 -- 172.21.15.156:0/4105752030 shutdown_connections 2023-11-28T11:12:00.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.948+0000 7f45cee55700 1 --2- 172.21.15.156:0/4105752030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c80ca240 0x7f45c80ca6a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.948+0000 7f45cee55700 1 --2- 172.21.15.156:0/4105752030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c806c990 0x7f45c806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.948+0000 7f45cee55700 1 -- 172.21.15.156:0/4105752030 >> 172.21.15.156:0/4105752030 conn(0x7f45c80b12b0 msgr2=0x7f45c80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:00.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.948+0000 7f45cee55700 1 -- 172.21.15.156:0/4105752030 shutdown_connections 2023-11-28T11:12:00.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.948+0000 7f45cee55700 1 -- 172.21.15.156:0/4105752030 wait complete. 2023-11-28T11:12:00.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.949+0000 7f45cee55700 1 Processor -- start 2023-11-28T11:12:00.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.949+0000 7f45cee55700 1 -- start start 2023-11-28T11:12:00.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45cee55700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c806c990 0x7f45c81572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45cee55700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c806e9f0 0x7f45c80c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45cee55700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c80ca240 0x7f45c80c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45cee55700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c81dee80 con 0x7f45c806e9f0 2023-11-28T11:12:00.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45cee55700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c81df160 con 0x7f45c80ca240 2023-11-28T11:12:00.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45cee55700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c81df650 con 0x7f45c806c990 2023-11-28T11:12:00.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45c7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c806e9f0 0x7f45c80c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45c7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c806e9f0 0x7f45c80c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36306/0 (socket says 172.21.15.156:36306) 2023-11-28T11:12:00.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45c7fff700 1 -- 172.21.15.156:0/984922853 learned_addr learned my addr 172.21.15.156:0/984922853 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:00.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45ccbf1700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c806c990 0x7f45c81572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45cd3f2700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c80ca240 0x7f45c80c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45c7fff700 1 -- 172.21.15.156:0/984922853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c806c990 msgr2=0x7f45c81572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45c7fff700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c806c990 0x7f45c81572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45c7fff700 1 -- 172.21.15.156:0/984922853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c80ca240 msgr2=0x7f45c80c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45c7fff700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c80ca240 0x7f45c80c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45c7fff700 1 -- 172.21.15.156:0/984922853 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0009b30 con 0x7f45c806e9f0 2023-11-28T11:12:00.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45c7fff700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c806e9f0 0x7f45c80c8a60 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f45b400e810 tx=0x7f45b400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45c57fa700 1 -- 172.21.15.156:0/984922853 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400c020 con 0x7f45c806e9f0 2023-11-28T11:12:00.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45c57fa700 1 -- 172.21.15.156:0/984922853 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b4004250 con 0x7f45c806e9f0 2023-11-28T11:12:00.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45c57fa700 1 -- 172.21.15.156:0/984922853 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b40036a0 con 0x7f45c806e9f0 2023-11-28T11:12:00.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45cee55700 1 -- 172.21.15.156:0/984922853 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c806a810 con 0x7f45c806e9f0 2023-11-28T11:12:00.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.951+0000 7f45cee55700 1 -- 172.21.15.156:0/984922853 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c806a530 con 0x7f45c806e9f0 2023-11-28T11:12:00.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.952+0000 7f45cee55700 1 -- 172.21.15.156:0/984922853 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c806aaf0 con 0x7f45c806e9f0 2023-11-28T11:12:00.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.953+0000 7f45c57fa700 1 -- 172.21.15.156:0/984922853 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f45b4022070 con 0x7f45c806e9f0 2023-11-28T11:12:00.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.953+0000 7f45c57fa700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f45ac0682f0 0x7f45ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.953+0000 7f45c57fa700 1 -- 172.21.15.156:0/984922853 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f45b4085970 con 0x7f45c806e9f0 2023-11-28T11:12:00.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.953+0000 7f45ccbf1700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f45ac0682f0 0x7f45ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.954+0000 7f45ccbf1700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f45ac0682f0 0x7f45ac06a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000f60 tx=0x7f45bc007f10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:00.957+0000 7f45c57fa700 1 -- 172.21.15.156:0/984922853 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f45b4052150 con 0x7f45c806e9f0 2023-11-28T11:12:01.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.139+0000 7f45cee55700 1 -- 172.21.15.156:0/984922853 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45c80c2d60 con 0x7f45c806e9f0 2023-11-28T11:12:01.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.140+0000 7f45c57fa700 1 -- 172.21.15.156:0/984922853 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18161 (secure 0 0 0) 0x7f45b4055770 con 0x7f45c806e9f0 2023-11-28T11:12:01.142 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:01.142 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":65,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:11:02.788873+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:50.055098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:51.350508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.33","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:01.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.142+0000 7f45cee55700 1 -- 172.21.15.156:0/984922853 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f45ac0682f0 msgr2=0x7f45ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.142+0000 7f45cee55700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f45ac0682f0 0x7f45ac06a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000f60 tx=0x7f45bc007f10 comp rx=0 tx=0).stop 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.142+0000 7f45cee55700 1 -- 172.21.15.156:0/984922853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c806e9f0 msgr2=0x7f45c80c8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.142+0000 7f45cee55700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c806e9f0 0x7f45c80c8a60 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f45b400e810 tx=0x7f45b400eb20 comp rx=0 tx=0).stop 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.142+0000 7f45cee55700 1 -- 172.21.15.156:0/984922853 shutdown_connections 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.142+0000 7f45cee55700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c80ca240 0x7f45c80c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.142+0000 7f45cee55700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f45ac0682f0 0x7f45ac06a7a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.142+0000 7f45cee55700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c806c990 0x7f45c81572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.142+0000 7f45cee55700 1 --2- 172.21.15.156:0/984922853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c806e9f0 0x7f45c80c8a60 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.142+0000 7f45cee55700 1 -- 172.21.15.156:0/984922853 >> 172.21.15.156:0/984922853 conn(0x7f45c80b12b0 msgr2=0x7f45c80b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.142+0000 7f45cee55700 1 -- 172.21.15.156:0/984922853 shutdown_connections 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.142+0000 7f45cee55700 1 -- 172.21.15.156:0/984922853 wait complete. 2023-11-28T11:12:01.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:06.566571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:10.283177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:10:50.055098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:10:51.350508+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:12:01.156 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:12:01.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.278+0000 7f13afdc2700 1 Processor -- start 2023-11-28T11:12:01.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.279+0000 7f13afdc2700 1 -- start start 2023-11-28T11:12:01.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.279+0000 7f13afdc2700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f13a806dee0 0x7f13a806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.279+0000 7f13afdc2700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f13a8160540 0x7f13a8160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.279+0000 7f13afdc2700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a815d060 con 0x7f13a8071040 2023-11-28T11:12:01.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.279+0000 7f13afdc2700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a815d340 con 0x7f13a806dee0 2023-11-28T11:12:01.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.279+0000 7f13afdc2700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a815d620 con 0x7f13a8160540 2023-11-28T11:12:01.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.280+0000 7f13ad35d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f13a806dee0 0x7f13a806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.280+0000 7f13ad35d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f13a806dee0 0x7f13a806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36314/0 (socket says 172.21.15.156:36314) 2023-11-28T11:12:01.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.280+0000 7f13ad35d700 1 -- 172.21.15.156:0/4243417315 learned_addr learned my addr 172.21.15.156:0/4243417315 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:01.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.280+0000 7f13ae35f700 1 --2- 172.21.15.156:0/4243417315 >> v2:172.21.15.186:3300/0 conn(0x7f13a8160540 0x7f13a8160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.280+0000 7f13ad35d700 1 -- 172.21.15.156:0/4243417315 >> v2:172.21.15.186:3300/0 conn(0x7f13a8160540 msgr2=0x7f13a8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:01.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.280+0000 7f13ad35d700 1 --2- 172.21.15.156:0/4243417315 >> v2:172.21.15.186:3300/0 conn(0x7f13a8160540 0x7f13a8160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.280+0000 7f13ad35d700 1 -- 172.21.15.156:0/4243417315 >> v1:172.21.15.186:6789/0 conn(0x7f13a8071040 legacy=0x7f13a8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.280+0000 7f13ad35d700 1 -- 172.21.15.156:0/4243417315 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a815d900 con 0x7f13a806dee0 2023-11-28T11:12:01.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.280+0000 7f13ad35d700 1 --2- 172.21.15.156:0/4243417315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13a806dee0 0x7f13a806e330 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f13a80c3a10 tx=0x7f1398009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e7312016f9ea32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.280+0000 7f13acb5c700 1 -- 172.21.15.156:0/4243417315 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398011830 con 0x7f13a806dee0 2023-11-28T11:12:01.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.280+0000 7f13acb5c700 1 -- 172.21.15.156:0/4243417315 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1398011990 con 0x7f13a806dee0 2023-11-28T11:12:01.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.280+0000 7f13acb5c700 1 -- 172.21.15.156:0/4243417315 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398011d20 con 0x7f13a806dee0 2023-11-28T11:12:01.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.281+0000 7f13afdc2700 1 -- 172.21.15.156:0/4243417315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13a806dee0 msgr2=0x7f13a806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.281+0000 7f13afdc2700 1 --2- 172.21.15.156:0/4243417315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13a806dee0 0x7f13a806e330 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f13a80c3a10 tx=0x7f1398009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.281+0000 7f13afdc2700 1 -- 172.21.15.156:0/4243417315 shutdown_connections 2023-11-28T11:12:01.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.281+0000 7f13afdc2700 1 --2- 172.21.15.156:0/4243417315 >> v2:172.21.15.186:3300/0 conn(0x7f13a8160540 0x7f13a8160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.281+0000 7f13afdc2700 1 --2- 172.21.15.156:0/4243417315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13a806dee0 0x7f13a806e330 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.281+0000 7f13afdc2700 1 -- 172.21.15.156:0/4243417315 >> 172.21.15.156:0/4243417315 conn(0x7f13a80d0000 msgr2=0x7f13a80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:01.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.281+0000 7f13afdc2700 1 -- 172.21.15.156:0/4243417315 shutdown_connections 2023-11-28T11:12:01.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.281+0000 7f13afdc2700 1 -- 172.21.15.156:0/4243417315 wait complete. 2023-11-28T11:12:01.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.282+0000 7f13afdc2700 1 Processor -- start 2023-11-28T11:12:01.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.282+0000 7f13afdc2700 1 -- start start 2023-11-28T11:12:01.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13afdc2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13a806dee0 0x7f13a814a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13afdc2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13a8071040 0x7f13a80b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13afdc2700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13a8160540 0x7f13a80b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13afdc2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a81f7190 con 0x7f13a806dee0 2023-11-28T11:12:01.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13afdc2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a81f6610 con 0x7f13a8071040 2023-11-28T11:12:01.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13afdc2700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a81f68f0 con 0x7f13a8160540 2023-11-28T11:12:01.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13adb5e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13a806dee0 0x7f13a814a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13adb5e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13a806dee0 0x7f13a814a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36318/0 (socket says 172.21.15.156:36318) 2023-11-28T11:12:01.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13adb5e700 1 -- 172.21.15.156:0/374688598 learned_addr learned my addr 172.21.15.156:0/374688598 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:01.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13ad35d700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13a8071040 0x7f13a80b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13ae35f700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13a8160540 0x7f13a80b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13adb5e700 1 -- 172.21.15.156:0/374688598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13a8160540 msgr2=0x7f13a80b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13adb5e700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13a8160540 0x7f13a80b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13adb5e700 1 -- 172.21.15.156:0/374688598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13a8071040 msgr2=0x7f13a80b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13adb5e700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13a8071040 0x7f13a80b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13adb5e700 1 -- 172.21.15.156:0/374688598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1398009550 con 0x7f13a806dee0 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.283+0000 7f13adb5e700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13a806dee0 0x7f13a814a3b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f13a000cbd0 tx=0x7f13a000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.284+0000 7f139e7fc700 1 -- 172.21.15.156:0/374688598 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0003c00 con 0x7f13a806dee0 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.284+0000 7f139e7fc700 1 -- 172.21.15.156:0/374688598 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a0003d60 con 0x7f13a806dee0 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.284+0000 7f13afdc2700 1 -- 172.21.15.156:0/374688598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a815ccd0 con 0x7f13a806dee0 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.284+0000 7f139e7fc700 1 -- 172.21.15.156:0/374688598 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a000aae0 con 0x7f13a806dee0 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.284+0000 7f13afdc2700 1 -- 172.21.15.156:0/374688598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a806e870 con 0x7f13a806dee0 2023-11-28T11:12:01.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.284+0000 7f13afdc2700 1 -- 172.21.15.156:0/374688598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a806a270 con 0x7f13a806dee0 2023-11-28T11:12:01.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.285+0000 7f139e7fc700 1 -- 172.21.15.156:0/374688598 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f13a0010050 con 0x7f13a806dee0 2023-11-28T11:12:01.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.285+0000 7f139e7fc700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f138c0682f0 0x7f138c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.285+0000 7f139e7fc700 1 -- 172.21.15.156:0/374688598 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f13a0085b60 con 0x7f13a806dee0 2023-11-28T11:12:01.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.285+0000 7f13ad35d700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f138c0682f0 0x7f138c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.286+0000 7f13ad35d700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f138c0682f0 0x7f138c06a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1398014040 tx=0x7f1398008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.289+0000 7f139e7fc700 1 -- 172.21.15.156:0/374688598 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f13a0052340 con 0x7f13a806dee0 2023-11-28T11:12:01.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.468+0000 7f13afdc2700 1 -- 172.21.15.156:0/374688598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13a8072330 con 0x7f13a806dee0 2023-11-28T11:12:01.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.469+0000 7f139e7fc700 1 -- 172.21.15.156:0/374688598 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13a0055960 con 0x7f13a806dee0 2023-11-28T11:12:01.470 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T11:12:01.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.471+0000 7f13afdc2700 1 -- 172.21.15.156:0/374688598 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f138c0682f0 msgr2=0x7f138c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.471+0000 7f13afdc2700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f138c0682f0 0x7f138c06a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1398014040 tx=0x7f1398008030 comp rx=0 tx=0).stop 2023-11-28T11:12:01.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.471+0000 7f13afdc2700 1 -- 172.21.15.156:0/374688598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13a806dee0 msgr2=0x7f13a814a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.471+0000 7f13afdc2700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13a806dee0 0x7f13a814a3b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f13a000cbd0 tx=0x7f13a000cee0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.471+0000 7f13afdc2700 1 -- 172.21.15.156:0/374688598 shutdown_connections 2023-11-28T11:12:01.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.471+0000 7f13afdc2700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f138c0682f0 0x7f138c06a7a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.471+0000 7f13afdc2700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13a8160540 0x7f13a80b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.471+0000 7f13afdc2700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13a806dee0 0x7f13a814a3b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.471+0000 7f13afdc2700 1 --2- 172.21.15.156:0/374688598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13a8071040 0x7f13a80b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.471+0000 7f13afdc2700 1 -- 172.21.15.156:0/374688598 >> 172.21.15.156:0/374688598 conn(0x7f13a80d0000 msgr2=0x7f13a815f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:01.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.471+0000 7f13afdc2700 1 -- 172.21.15.156:0/374688598 shutdown_connections 2023-11-28T11:12:01.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.471+0000 7f13afdc2700 1 -- 172.21.15.156:0/374688598 wait complete. 2023-11-28T11:12:01.484 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:12:01.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.607+0000 7ff643bc6700 1 Processor -- start 2023-11-28T11:12:01.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.607+0000 7ff643bc6700 1 -- start start 2023-11-28T11:12:01.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.608+0000 7ff643bc6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c0cd8c0 0x7ff63c140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.608+0000 7ff643bc6700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff63c13ff00 con 0x7ff63c06f850 2023-11-28T11:12:01.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.608+0000 7ff643bc6700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff63c1404c0 con 0x7ff63c0cdc90 2023-11-28T11:12:01.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.608+0000 7ff643bc6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c15aa80 con 0x7ff63c0cd8c0 2023-11-28T11:12:01.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.608+0000 7ff642163700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff63c06f850 0x7ff63c06fcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51208/0 (socket says 172.21.15.156:51208) 2023-11-28T11:12:01.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.608+0000 7ff642163700 1 -- 172.21.15.156:0/1200706640 learned_addr learned my addr 172.21.15.156:0/1200706640 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:01.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff641962700 1 --2- 172.21.15.156:0/1200706640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c0cd8c0 0x7ff63c140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570737023 0 0) 0x7ff63c13ff00 con 0x7ff63c06f850 2023-11-28T11:12:01.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6280035b0 con 0x7ff63c06f850 2023-11-28T11:12:01.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 995172357 0 0) 0x7ff6280035b0 con 0x7ff63c06f850 2023-11-28T11:12:01.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c0cd8c0 msgr2=0x7ff63c140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:01.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 --2- 172.21.15.156:0/1200706640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c0cd8c0 0x7ff63c140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 >> v1:172.21.15.186:6789/0 conn(0x7ff63c0cdc90 legacy=0x7ff63c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c0b3040 con 0x7ff63c06f850 2023-11-28T11:12:01.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7ff6340019f0 con 0x7ff63c06f850 2023-11-28T11:12:01.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 >> v1:172.21.15.156:6789/0 conn(0x7ff63c06f850 legacy=0x7ff63c06fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 --2- 172.21.15.156:0/1200706640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff628003a30 0x7ff628003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 --2- 172.21.15.156:0/1200706640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6280043d0 0x7ff628004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 --2- 172.21.15.156:0/1200706640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c0cd8c0 0x7ff628005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c15aa80 con 0x7ff6280043d0 2023-11-28T11:12:01.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62800bac0 con 0x7ff628003a30 2023-11-28T11:12:01.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62800bc50 con 0x7ff63c0cd8c0 2023-11-28T11:12:01.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff641161700 1 --2- 172.21.15.156:0/1200706640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6280043d0 0x7ff628004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff641962700 1 --2- 172.21.15.156:0/1200706640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff628003a30 0x7ff628003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff642163700 1 --2- 172.21.15.156:0/1200706640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c0cd8c0 0x7ff628005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff641161700 1 -- 172.21.15.156:0/1200706640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c0cd8c0 msgr2=0x7ff628005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:01.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff641161700 1 --2- 172.21.15.156:0/1200706640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c0cd8c0 0x7ff628005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff641161700 1 -- 172.21.15.156:0/1200706640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff628003a30 msgr2=0x7ff628003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:01.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff641161700 1 --2- 172.21.15.156:0/1200706640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff628003a30 0x7ff628003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.609+0000 7ff641161700 1 -- 172.21.15.156:0/1200706640 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280035b0 con 0x7ff6280043d0 2023-11-28T11:12:01.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.610+0000 7ff641161700 1 --2- 172.21.15.156:0/1200706640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6280043d0 0x7ff628004890 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff638001980 tx=0x7ff628005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3978513130f843 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.610+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638016070 con 0x7ff6280043d0 2023-11-28T11:12:01.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.610+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff638006860 con 0x7ff6280043d0 2023-11-28T11:12:01.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.610+0000 7ff640960700 1 -- 172.21.15.156:0/1200706640 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638001420 con 0x7ff6280043d0 2023-11-28T11:12:01.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.610+0000 7ff643bc6700 1 -- 172.21.15.156:0/1200706640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6280043d0 msgr2=0x7ff628004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.610+0000 7ff643bc6700 1 --2- 172.21.15.156:0/1200706640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6280043d0 0x7ff628004890 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff638001980 tx=0x7ff628005880 comp rx=0 tx=0).stop 2023-11-28T11:12:01.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.610+0000 7ff642163700 1 -- 172.21.15.156:0/1200706640 reap_dead start 2023-11-28T11:12:01.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.610+0000 7ff643bc6700 1 -- 172.21.15.156:0/1200706640 shutdown_connections 2023-11-28T11:12:01.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.610+0000 7ff643bc6700 1 -- 172.21.15.156:0/1200706640 >> 172.21.15.156:0/1200706640 conn(0x7ff63c06aed0 msgr2=0x7ff63c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:01.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.610+0000 7ff643bc6700 1 -- 172.21.15.156:0/1200706640 shutdown_connections 2023-11-28T11:12:01.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.610+0000 7ff643bc6700 1 -- 172.21.15.156:0/1200706640 wait complete. 2023-11-28T11:12:01.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.612+0000 7ff643bc6700 1 Processor -- start 2023-11-28T11:12:01.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.612+0000 7ff643bc6700 1 -- start start 2023-11-28T11:12:01.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.612+0000 7ff643bc6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6280043d0 0x7ff63c0c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.612+0000 7ff643bc6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff63c1e32d0 0x7ff63c1e3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.612+0000 7ff643bc6700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c150f40 0x7ff63c1513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.612+0000 7ff643bc6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c1e4ac0 con 0x7ff6280043d0 2023-11-28T11:12:01.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.612+0000 7ff643bc6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c1518f0 con 0x7ff63c1e32d0 2023-11-28T11:12:01.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.612+0000 7ff643bc6700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c151d90 con 0x7ff63c150f40 2023-11-28T11:12:01.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.612+0000 7ff641962700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6280043d0 0x7ff63c0c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.612+0000 7ff641962700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6280043d0 0x7ff63c0c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36338/0 (socket says 172.21.15.156:36338) 2023-11-28T11:12:01.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.612+0000 7ff641962700 1 -- 172.21.15.156:0/1671747036 learned_addr learned my addr 172.21.15.156:0/1671747036 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:01.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff642163700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c150f40 0x7ff63c1513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff641161700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff63c1e32d0 0x7ff63c1e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff641962700 1 -- 172.21.15.156:0/1671747036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c150f40 msgr2=0x7ff63c1513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff641962700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c150f40 0x7ff63c1513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff641962700 1 -- 172.21.15.156:0/1671747036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff63c1e32d0 msgr2=0x7ff63c1e3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff641962700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff63c1e32d0 0x7ff63c1e3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff641962700 1 -- 172.21.15.156:0/1671747036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c0b35f0 con 0x7ff6280043d0 2023-11-28T11:12:01.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff641962700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6280043d0 0x7ff63c0c61f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009fd0 tx=0x7ff62c011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff6327fc700 1 -- 172.21.15.156:0/1671747036 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c005c30 con 0x7ff6280043d0 2023-11-28T11:12:01.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff6327fc700 1 -- 172.21.15.156:0/1671747036 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62c005d90 con 0x7ff6280043d0 2023-11-28T11:12:01.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff643bc6700 1 -- 172.21.15.156:0/1671747036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63c0b3310 con 0x7ff6280043d0 2023-11-28T11:12:01.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff6327fc700 1 -- 172.21.15.156:0/1671747036 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c006290 con 0x7ff6280043d0 2023-11-28T11:12:01.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.613+0000 7ff643bc6700 1 -- 172.21.15.156:0/1671747036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c140150 con 0x7ff6280043d0 2023-11-28T11:12:01.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.614+0000 7ff643bc6700 1 -- 172.21.15.156:0/1671747036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63c1e4d30 con 0x7ff6280043d0 2023-11-28T11:12:01.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.614+0000 7ff6327fc700 1 -- 172.21.15.156:0/1671747036 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff62c0063f0 con 0x7ff6280043d0 2023-11-28T11:12:01.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.615+0000 7ff6327fc700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6200682f0 0x7ff62006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.615+0000 7ff6327fc700 1 -- 172.21.15.156:0/1671747036 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7ff62c086490 con 0x7ff6280043d0 2023-11-28T11:12:01.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.615+0000 7ff641161700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6200682f0 0x7ff62006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.615+0000 7ff641161700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6200682f0 0x7ff62006a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff63c157bc0 tx=0x7ff638009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.619+0000 7ff6327fc700 1 -- 172.21.15.156:0/1671747036 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff62c052c70 con 0x7ff6280043d0 2023-11-28T11:12:01.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.801+0000 7ff643bc6700 1 -- 172.21.15.156:0/1671747036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff63c068690 con 0x7ff6280043d0 2023-11-28T11:12:01.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.801+0000 7ff6327fc700 1 -- 172.21.15.156:0/1671747036 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff62c056290 con 0x7ff6280043d0 2023-11-28T11:12:01.803 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T11:12:01.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.803+0000 7ff643bc6700 1 -- 172.21.15.156:0/1671747036 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6200682f0 msgr2=0x7ff62006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.803+0000 7ff643bc6700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6200682f0 0x7ff62006a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff63c157bc0 tx=0x7ff638009030 comp rx=0 tx=0).stop 2023-11-28T11:12:01.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.804+0000 7ff643bc6700 1 -- 172.21.15.156:0/1671747036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6280043d0 msgr2=0x7ff63c0c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.804+0000 7ff643bc6700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6280043d0 0x7ff63c0c61f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009fd0 tx=0x7ff62c011c80 comp rx=0 tx=0).stop 2023-11-28T11:12:01.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.804+0000 7ff643bc6700 1 -- 172.21.15.156:0/1671747036 shutdown_connections 2023-11-28T11:12:01.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.804+0000 7ff643bc6700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6200682f0 0x7ff62006a7a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.804+0000 7ff643bc6700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff63c150f40 0x7ff63c1513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.804+0000 7ff643bc6700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6280043d0 0x7ff63c0c61f0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.804+0000 7ff643bc6700 1 --2- 172.21.15.156:0/1671747036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff63c1e32d0 0x7ff63c1e3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.804+0000 7ff643bc6700 1 -- 172.21.15.156:0/1671747036 >> 172.21.15.156:0/1671747036 conn(0x7ff63c06aed0 msgr2=0x7ff63c1407a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:01.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.804+0000 7ff643bc6700 1 -- 172.21.15.156:0/1671747036 shutdown_connections 2023-11-28T11:12:01.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.804+0000 7ff643bc6700 1 -- 172.21.15.156:0/1671747036 wait complete. 2023-11-28T11:12:01.817 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:12:01.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.939+0000 7fe153d30700 1 Processor -- start 2023-11-28T11:12:01.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.940+0000 7fe153d30700 1 -- start start 2023-11-28T11:12:01.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.940+0000 7fe153d30700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe14c071040 0x7fe14c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.940+0000 7fe153d30700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c06dee0 0x7fe14c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.940+0000 7fe153d30700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe14c06cdc0 0x7fe14c06d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.940+0000 7fe153d30700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14c071950 con 0x7fe14c06cdc0 2023-11-28T11:12:01.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.940+0000 7fe153d30700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14c15d060 con 0x7fe14c071040 2023-11-28T11:12:01.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.940+0000 7fe153d30700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c15d340 con 0x7fe14c06dee0 2023-11-28T11:12:01.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe1522cd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe14c06cdc0 0x7fe14c06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe1522cd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe14c06cdc0 0x7fe14c06d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36350/0 (socket says 172.21.15.156:36350) 2023-11-28T11:12:01.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe1522cd700 1 -- 172.21.15.156:0/1222988170 learned_addr learned my addr 172.21.15.156:0/1222988170 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:01.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe151acc700 1 --2- 172.21.15.156:0/1222988170 >> v2:172.21.15.186:3300/0 conn(0x7fe14c071040 0x7fe14c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe1512cb700 1 --2- 172.21.15.156:0/1222988170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c06dee0 0x7fe14c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe1522cd700 1 -- 172.21.15.156:0/1222988170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c06dee0 msgr2=0x7fe14c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:01.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe1522cd700 1 --2- 172.21.15.156:0/1222988170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c06dee0 0x7fe14c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe1522cd700 1 -- 172.21.15.156:0/1222988170 >> v2:172.21.15.186:3300/0 conn(0x7fe14c071040 msgr2=0x7fe14c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:01.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe1522cd700 1 --2- 172.21.15.156:0/1222988170 >> v2:172.21.15.186:3300/0 conn(0x7fe14c071040 0x7fe14c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe1522cd700 1 -- 172.21.15.156:0/1222988170 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c15d620 con 0x7fe14c06cdc0 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe1512cb700 1 --2- 172.21.15.156:0/1222988170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c06dee0 0x7fe14c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe1522cd700 1 --2- 172.21.15.156:0/1222988170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe14c06cdc0 0x7fe14c06d240 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0d0f50 tx=0x7fe14400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e584a233375ab94e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe150aca700 1 -- 172.21.15.156:0/1222988170 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400e070 con 0x7fe14c06cdc0 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe150aca700 1 -- 172.21.15.156:0/1222988170 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14400b910 con 0x7fe14c06cdc0 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.941+0000 7fe150aca700 1 -- 172.21.15.156:0/1222988170 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400bcc0 con 0x7fe14c06cdc0 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.942+0000 7fe153d30700 1 -- 172.21.15.156:0/1222988170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe14c06cdc0 msgr2=0x7fe14c06d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.942+0000 7fe153d30700 1 --2- 172.21.15.156:0/1222988170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe14c06cdc0 0x7fe14c06d240 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0d0f50 tx=0x7fe14400b330 comp rx=0 tx=0).stop 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.942+0000 7fe153d30700 1 -- 172.21.15.156:0/1222988170 shutdown_connections 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.942+0000 7fe153d30700 1 --2- 172.21.15.156:0/1222988170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe14c06cdc0 0x7fe14c06d240 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.942+0000 7fe153d30700 1 --2- 172.21.15.156:0/1222988170 >> v2:172.21.15.186:3300/0 conn(0x7fe14c071040 0x7fe14c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.942+0000 7fe153d30700 1 --2- 172.21.15.156:0/1222988170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c06dee0 0x7fe14c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.942+0000 7fe153d30700 1 -- 172.21.15.156:0/1222988170 >> 172.21.15.156:0/1222988170 conn(0x7fe14c0d0000 msgr2=0x7fe14c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:01.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.942+0000 7fe153d30700 1 -- 172.21.15.156:0/1222988170 shutdown_connections 2023-11-28T11:12:01.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.942+0000 7fe153d30700 1 -- 172.21.15.156:0/1222988170 wait complete. 2023-11-28T11:12:01.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.943+0000 7fe153d30700 1 Processor -- start 2023-11-28T11:12:01.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.943+0000 7fe153d30700 1 -- start start 2023-11-28T11:12:01.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe153d30700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe14c06cdc0 0x7fe14c156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe153d30700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c06dee0 0x7fe14c154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe153d30700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe14c071040 0x7fe14c0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe153d30700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0c6310 con 0x7fe14c06cdc0 2023-11-28T11:12:01.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe153d30700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0b33a0 con 0x7fe14c071040 2023-11-28T11:12:01.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe153d30700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0b3680 con 0x7fe14c06dee0 2023-11-28T11:12:01.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe151acc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe14c06cdc0 0x7fe14c156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe151acc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe14c06cdc0 0x7fe14c156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36352/0 (socket says 172.21.15.156:36352) 2023-11-28T11:12:01.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe151acc700 1 -- 172.21.15.156:0/4079111344 learned_addr learned my addr 172.21.15.156:0/4079111344 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:01.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe1522cd700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe14c071040 0x7fe14c0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe1512cb700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c06dee0 0x7fe14c154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe151acc700 1 -- 172.21.15.156:0/4079111344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c06dee0 msgr2=0x7fe14c154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe151acc700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c06dee0 0x7fe14c154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe151acc700 1 -- 172.21.15.156:0/4079111344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe14c071040 msgr2=0x7fe14c0b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe151acc700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe14c071040 0x7fe14c0b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.944+0000 7fe151acc700 1 -- 172.21.15.156:0/4079111344 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe144009b30 con 0x7fe14c06cdc0 2023-11-28T11:12:01.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.945+0000 7fe151acc700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe14c06cdc0 0x7fe14c156e60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00e870 tx=0x7fe13c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.945+0000 7fe1427fc700 1 -- 172.21.15.156:0/4079111344 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c009590 con 0x7fe14c06cdc0 2023-11-28T11:12:01.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.945+0000 7fe1427fc700 1 -- 172.21.15.156:0/4079111344 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c013680 con 0x7fe14c06cdc0 2023-11-28T11:12:01.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.945+0000 7fe153d30700 1 -- 172.21.15.156:0/4079111344 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14c06e870 con 0x7fe14c06cdc0 2023-11-28T11:12:01.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.945+0000 7fe1427fc700 1 -- 172.21.15.156:0/4079111344 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0139f0 con 0x7fe14c06cdc0 2023-11-28T11:12:01.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.945+0000 7fe153d30700 1 -- 172.21.15.156:0/4079111344 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14c15d900 con 0x7fe14c06cdc0 2023-11-28T11:12:01.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.946+0000 7fe153d30700 1 -- 172.21.15.156:0/4079111344 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14c15ccd0 con 0x7fe14c06cdc0 2023-11-28T11:12:01.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.946+0000 7fe1427fc700 1 -- 172.21.15.156:0/4079111344 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe13c019030 con 0x7fe14c06cdc0 2023-11-28T11:12:01.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.947+0000 7fe1427fc700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe1300682f0 0x7fe13006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.947+0000 7fe1427fc700 1 -- 172.21.15.156:0/4079111344 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fe13c0859d0 con 0x7fe14c06cdc0 2023-11-28T11:12:01.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.947+0000 7fe1512cb700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe1300682f0 0x7fe13006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.947+0000 7fe1512cb700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe1300682f0 0x7fe13006a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe14c1f60d0 tx=0x7fe148008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:01.951+0000 7fe1427fc700 1 -- 172.21.15.156:0/4079111344 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe13c0521b0 con 0x7fe14c06cdc0 2023-11-28T11:12:02.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.131+0000 7fe153d30700 1 -- 172.21.15.156:0/4079111344 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe14c0720a0 con 0x7fe14c06cdc0 2023-11-28T11:12:02.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.132+0000 7fe1427fc700 1 -- 172.21.15.156:0/4079111344 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe13c0557d0 con 0x7fe14c06cdc0 2023-11-28T11:12:02.134 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T11:12:02.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.134+0000 7fe153d30700 1 -- 172.21.15.156:0/4079111344 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe1300682f0 msgr2=0x7fe13006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.134+0000 7fe153d30700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe1300682f0 0x7fe13006a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe14c1f60d0 tx=0x7fe148008040 comp rx=0 tx=0).stop 2023-11-28T11:12:02.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.134+0000 7fe153d30700 1 -- 172.21.15.156:0/4079111344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe14c06cdc0 msgr2=0x7fe14c156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.134+0000 7fe153d30700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe14c06cdc0 0x7fe14c156e60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00e870 tx=0x7fe13c00eb80 comp rx=0 tx=0).stop 2023-11-28T11:12:02.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.134+0000 7fe153d30700 1 -- 172.21.15.156:0/4079111344 shutdown_connections 2023-11-28T11:12:02.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.134+0000 7fe153d30700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe14c071040 0x7fe14c0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.134+0000 7fe153d30700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe14c06cdc0 0x7fe14c156e60 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.134+0000 7fe153d30700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe1300682f0 0x7fe13006a7a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.134+0000 7fe153d30700 1 --2- 172.21.15.156:0/4079111344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c06dee0 0x7fe14c154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.134+0000 7fe153d30700 1 -- 172.21.15.156:0/4079111344 >> 172.21.15.156:0/4079111344 conn(0x7fe14c0d0000 msgr2=0x7fe14c15f760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:02.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.134+0000 7fe153d30700 1 -- 172.21.15.156:0/4079111344 shutdown_connections 2023-11-28T11:12:02.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.134+0000 7fe153d30700 1 -- 172.21.15.156:0/4079111344 wait complete. 2023-11-28T11:12:02.148 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:12:02.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.271+0000 7f299a633700 1 Processor -- start 2023-11-28T11:12:02.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.271+0000 7f299a633700 1 -- start start 2023-11-28T11:12:02.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.272+0000 7f299a633700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2994071040 0x7f2994071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.272+0000 7f299a633700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2994160540 0x7f2994160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.272+0000 7f299a633700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2994071950 con 0x7f299406dee0 2023-11-28T11:12:02.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.272+0000 7f299a633700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299415d060 con 0x7f2994071040 2023-11-28T11:12:02.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.272+0000 7f299a633700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299415d340 con 0x7f2994160540 2023-11-28T11:12:02.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.272+0000 7f2993fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2994071040 0x7f2994071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.272+0000 7f2993fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2994071040 0x7f2994071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36360/0 (socket says 172.21.15.156:36360) 2023-11-28T11:12:02.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.272+0000 7f2993fff700 1 -- 172.21.15.156:0/3912840379 learned_addr learned my addr 172.21.15.156:0/3912840379 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:02.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.272+0000 7f2993fff700 1 -- 172.21.15.156:0/3912840379 >> v2:172.21.15.186:3300/0 conn(0x7f2994160540 msgr2=0x7f2994160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:02.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.272+0000 7f2998bd0700 1 --2- 172.21.15.156:0/3912840379 >> v2:172.21.15.186:3300/0 conn(0x7f2994160540 0x7f2994160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.272+0000 7f2993fff700 1 --2- 172.21.15.156:0/3912840379 >> v2:172.21.15.186:3300/0 conn(0x7f2994160540 0x7f2994160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f2993fff700 1 -- 172.21.15.156:0/3912840379 >> v1:172.21.15.186:6789/0 conn(0x7f299406dee0 legacy=0x7f299406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:02.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f2993fff700 1 -- 172.21.15.156:0/3912840379 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299415d620 con 0x7f2994071040 2023-11-28T11:12:02.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f2993fff700 1 --2- 172.21.15.156:0/3912840379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2994071040 0x7f2994071410 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f29940c3a10 tx=0x7f2988009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ceedb80f763a49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:02.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f2992ffd700 1 -- 172.21.15.156:0/3912840379 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988004230 con 0x7f2994071040 2023-11-28T11:12:02.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f2992ffd700 1 -- 172.21.15.156:0/3912840379 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2988004390 con 0x7f2994071040 2023-11-28T11:12:02.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f2992ffd700 1 -- 172.21.15.156:0/3912840379 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29880036a0 con 0x7f2994071040 2023-11-28T11:12:02.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f299a633700 1 -- 172.21.15.156:0/3912840379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2994071040 msgr2=0x7f2994071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f299a633700 1 --2- 172.21.15.156:0/3912840379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2994071040 0x7f2994071410 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f29940c3a10 tx=0x7f2988009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f299a633700 1 -- 172.21.15.156:0/3912840379 shutdown_connections 2023-11-28T11:12:02.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f299a633700 1 --2- 172.21.15.156:0/3912840379 >> v2:172.21.15.186:3300/0 conn(0x7f2994160540 0x7f2994160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f299a633700 1 --2- 172.21.15.156:0/3912840379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2994071040 0x7f2994071410 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f299a633700 1 -- 172.21.15.156:0/3912840379 >> 172.21.15.156:0/3912840379 conn(0x7f29940d0000 msgr2=0x7f29940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:02.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f299a633700 1 -- 172.21.15.156:0/3912840379 shutdown_connections 2023-11-28T11:12:02.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.273+0000 7f299a633700 1 -- 172.21.15.156:0/3912840379 wait complete. 2023-11-28T11:12:02.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.275+0000 7f299a633700 1 Processor -- start 2023-11-28T11:12:02.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.275+0000 7f299a633700 1 -- start start 2023-11-28T11:12:02.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f299a633700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f299406dee0 0x7f299415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f299a633700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2994160540 0x7f2994155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f299a633700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29940c5100 0x7f29940c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f299a633700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29940c5ae0 con 0x7f299406dee0 2023-11-28T11:12:02.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f299a633700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2994159290 con 0x7f2994160540 2023-11-28T11:12:02.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f299a633700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29940b3200 con 0x7f29940c5100 2023-11-28T11:12:02.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f2993fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f299406dee0 0x7f299415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f2993fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f299406dee0 0x7f299415bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36368/0 (socket says 172.21.15.156:36368) 2023-11-28T11:12:02.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f2993fff700 1 -- 172.21.15.156:0/1353643545 learned_addr learned my addr 172.21.15.156:0/1353643545 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f2993fff700 1 -- 172.21.15.156:0/1353643545 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29940c5100 msgr2=0x7f29940c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f2993fff700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29940c5100 0x7f29940c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f29937fe700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2994160540 0x7f2994155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f2993fff700 1 -- 172.21.15.156:0/1353643545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2994160540 msgr2=0x7f2994155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f2993fff700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2994160540 0x7f2994155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f2993fff700 1 -- 172.21.15.156:0/1353643545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2988009550 con 0x7f299406dee0 2023-11-28T11:12:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f2993fff700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f299406dee0 0x7f299415bf40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2988009130 tx=0x7f2988004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f2990ff9700 1 -- 172.21.15.156:0/1353643545 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988003cc0 con 0x7f299406dee0 2023-11-28T11:12:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f2990ff9700 1 -- 172.21.15.156:0/1353643545 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298801f070 con 0x7f299406dee0 2023-11-28T11:12:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f2990ff9700 1 -- 172.21.15.156:0/1353643545 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988011830 con 0x7f299406dee0 2023-11-28T11:12:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f299a633700 1 -- 172.21.15.156:0/1353643545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299415ccd0 con 0x7f299406dee0 2023-11-28T11:12:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.276+0000 7f299a633700 1 -- 172.21.15.156:0/1353643545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299415d900 con 0x7f299406dee0 2023-11-28T11:12:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.277+0000 7f299a633700 1 -- 172.21.15.156:0/1353643545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299406a270 con 0x7f299406dee0 2023-11-28T11:12:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.278+0000 7f2990ff9700 1 -- 172.21.15.156:0/1353643545 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f298802a020 con 0x7f299406dee0 2023-11-28T11:12:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.278+0000 7f2990ff9700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f29800682f0 0x7f298006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.278+0000 7f2990ff9700 1 -- 172.21.15.156:0/1353643545 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f2988087430 con 0x7f299406dee0 2023-11-28T11:12:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.278+0000 7f29937fe700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f29800682f0 0x7f298006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.278+0000 7f29937fe700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f29800682f0 0x7f298006a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f29940b0eb0 tx=0x7f2984008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.282+0000 7f2990ff9700 1 -- 172.21.15.156:0/1353643545 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2988053bb0 con 0x7f299406dee0 2023-11-28T11:12:02.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.467+0000 7f299a633700 1 -- 172.21.15.156:0/1353643545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f299406a270 con 0x7f299406dee0 2023-11-28T11:12:02.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.467+0000 7f2990ff9700 1 -- 172.21.15.156:0/1353643545 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f29880571d0 con 0x7f299406dee0 2023-11-28T11:12:02.470 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T11:12:02.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.469+0000 7f299a633700 1 -- 172.21.15.156:0/1353643545 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f29800682f0 msgr2=0x7f298006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.469+0000 7f299a633700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f29800682f0 0x7f298006a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f29940b0eb0 tx=0x7f2984008040 comp rx=0 tx=0).stop 2023-11-28T11:12:02.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.469+0000 7f299a633700 1 -- 172.21.15.156:0/1353643545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f299406dee0 msgr2=0x7f299415bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.469+0000 7f299a633700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f299406dee0 0x7f299415bf40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2988009130 tx=0x7f2988004750 comp rx=0 tx=0).stop 2023-11-28T11:12:02.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.469+0000 7f299a633700 1 -- 172.21.15.156:0/1353643545 shutdown_connections 2023-11-28T11:12:02.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.469+0000 7f299a633700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f29800682f0 0x7f298006a7a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.469+0000 7f299a633700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29940c5100 0x7f29940c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.469+0000 7f299a633700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f299406dee0 0x7f299415bf40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.469+0000 7f299a633700 1 --2- 172.21.15.156:0/1353643545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2994160540 0x7f2994155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.469+0000 7f299a633700 1 -- 172.21.15.156:0/1353643545 >> 172.21.15.156:0/1353643545 conn(0x7f29940d0000 msgr2=0x7f29940cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:02.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.469+0000 7f299a633700 1 -- 172.21.15.156:0/1353643545 shutdown_connections 2023-11-28T11:12:02.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.469+0000 7f299a633700 1 -- 172.21.15.156:0/1353643545 wait complete. 2023-11-28T11:12:02.483 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:02.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.601+0000 7fa8dd498700 1 Processor -- start 2023-11-28T11:12:02.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.601+0000 7fa8dd498700 1 -- start start 2023-11-28T11:12:02.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8dd498700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8d8161340 0x7fa8d80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8dd498700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa8d80d2320 0x7fa8d80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8dd498700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d80d1950 con 0x7fa8d815c9b0 2023-11-28T11:12:02.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8dd498700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d80b5800 con 0x7fa8d80d2320 2023-11-28T11:12:02.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8dd498700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d80b5300 con 0x7fa8d8161340 2023-11-28T11:12:02.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8d77fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa8d80d2320 0x7fa8d80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8d77fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa8d80d2320 0x7fa8d80cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36378/0 (socket says 172.21.15.156:36378) 2023-11-28T11:12:02.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8d77fe700 1 -- 172.21.15.156:0/1779939915 learned_addr learned my addr 172.21.15.156:0/1779939915 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:02.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8d67fc700 1 --2- 172.21.15.156:0/1779939915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8d8161340 0x7fa8d80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8d77fe700 1 -- 172.21.15.156:0/1779939915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8d8161340 msgr2=0x7fa8d80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:02.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8d77fe700 1 --2- 172.21.15.156:0/1779939915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8d8161340 0x7fa8d80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8d77fe700 1 -- 172.21.15.156:0/1779939915 >> v1:172.21.15.186:6789/0 conn(0x7fa8d815c9b0 legacy=0x7fa8d80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.602+0000 7fa8d77fe700 1 -- 172.21.15.156:0/1779939915 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d80b5520 con 0x7fa8d80d2320 2023-11-28T11:12:02.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.603+0000 7fa8d77fe700 1 --2- 172.21.15.156:0/1779939915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8d80d2320 0x7fa8d80cd7c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009e80 tx=0x7fa8d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99490444ac4cf3b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:02.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.603+0000 7fa8d5ffb700 1 -- 172.21.15.156:0/1779939915 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d000e070 con 0x7fa8d80d2320 2023-11-28T11:12:02.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.603+0000 7fa8d5ffb700 1 -- 172.21.15.156:0/1779939915 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d000b980 con 0x7fa8d80d2320 2023-11-28T11:12:02.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.603+0000 7fa8d5ffb700 1 -- 172.21.15.156:0/1779939915 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d000bd30 con 0x7fa8d80d2320 2023-11-28T11:12:02.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.603+0000 7fa8dd498700 1 -- 172.21.15.156:0/1779939915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8d80d2320 msgr2=0x7fa8d80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.603+0000 7fa8dd498700 1 --2- 172.21.15.156:0/1779939915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8d80d2320 0x7fa8d80cd7c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009e80 tx=0x7fa8d000b330 comp rx=0 tx=0).stop 2023-11-28T11:12:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.603+0000 7fa8dd498700 1 -- 172.21.15.156:0/1779939915 shutdown_connections 2023-11-28T11:12:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.603+0000 7fa8dd498700 1 --2- 172.21.15.156:0/1779939915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8d80d2320 0x7fa8d80cd7c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.603+0000 7fa8dd498700 1 --2- 172.21.15.156:0/1779939915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8d8161340 0x7fa8d80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.603+0000 7fa8dd498700 1 -- 172.21.15.156:0/1779939915 >> 172.21.15.156:0/1779939915 conn(0x7fa8d8072600 msgr2=0x7fa8d80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.603+0000 7fa8dd498700 1 -- 172.21.15.156:0/1779939915 shutdown_connections 2023-11-28T11:12:02.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.603+0000 7fa8dd498700 1 -- 172.21.15.156:0/1779939915 wait complete. 2023-11-28T11:12:02.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.605+0000 7fa8dd498700 1 Processor -- start 2023-11-28T11:12:02.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.605+0000 7fa8dd498700 1 -- start start 2023-11-28T11:12:02.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.605+0000 7fa8dd498700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d80d2320 0x7fa8d8150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.605+0000 7fa8dd498700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8d815c9b0 0x7fa8d814fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.605+0000 7fa8dd498700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8d8161340 0x7fa8d81e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.605+0000 7fa8dd498700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d81e7e80 con 0x7fa8d8161340 2023-11-28T11:12:02.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.605+0000 7fa8dd498700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d80c85b0 con 0x7fa8d80d2320 2023-11-28T11:12:02.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.605+0000 7fa8dd498700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d80c8890 con 0x7fa8d815c9b0 2023-11-28T11:12:02.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.605+0000 7fa8d77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8d8161340 0x7fa8d81e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8d77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8d8161340 0x7fa8d81e7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36384/0 (socket says 172.21.15.156:36384) 2023-11-28T11:12:02.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8d77fe700 1 -- 172.21.15.156:0/297439091 learned_addr learned my addr 172.21.15.156:0/297439091 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:02.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8d67fc700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8d815c9b0 0x7fa8d814fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8d6ffd700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d80d2320 0x7fa8d8150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8d77fe700 1 -- 172.21.15.156:0/297439091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8d815c9b0 msgr2=0x7fa8d814fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8d77fe700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8d815c9b0 0x7fa8d814fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8d77fe700 1 -- 172.21.15.156:0/297439091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d80d2320 msgr2=0x7fa8d8150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8d77fe700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d80d2320 0x7fa8d8150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8d77fe700 1 -- 172.21.15.156:0/297439091 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d0009b30 con 0x7fa8d8161340 2023-11-28T11:12:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8d77fe700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8d8161340 0x7fa8d81e7630 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80675d0 tx=0x7fa8d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8cf7fe700 1 -- 172.21.15.156:0/297439091 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0004960 con 0x7fa8d8161340 2023-11-28T11:12:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8cf7fe700 1 -- 172.21.15.156:0/297439091 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d00099b0 con 0x7fa8d8161340 2023-11-28T11:12:02.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8dd498700 1 -- 172.21.15.156:0/297439091 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d80d15e0 con 0x7fa8d8161340 2023-11-28T11:12:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8cf7fe700 1 -- 172.21.15.156:0/297439091 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d001b670 con 0x7fa8d8161340 2023-11-28T11:12:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.606+0000 7fa8dd498700 1 -- 172.21.15.156:0/297439091 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d80d1300 con 0x7fa8d8161340 2023-11-28T11:12:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.607+0000 7fa8dd498700 1 -- 172.21.15.156:0/297439091 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d815cd80 con 0x7fa8d8161340 2023-11-28T11:12:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.608+0000 7fa8cf7fe700 1 -- 172.21.15.156:0/297439091 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa8d0021030 con 0x7fa8d8161340 2023-11-28T11:12:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.608+0000 7fa8cf7fe700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa8b80682f0 0x7fa8b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.608+0000 7fa8cf7fe700 1 -- 172.21.15.156:0/297439091 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fa8d00872b0 con 0x7fa8d8161340 2023-11-28T11:12:02.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.608+0000 7fa8d6ffd700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa8b80682f0 0x7fa8b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.608+0000 7fa8d6ffd700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa8b80682f0 0x7fa8b806a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0000f60 tx=0x7fa8c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:02.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.613+0000 7fa8cf7fe700 1 -- 172.21.15.156:0/297439091 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa8d0053a90 con 0x7fa8d8161340 2023-11-28T11:12:02.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.836+0000 7fa8dd498700 1 -- 172.21.15.156:0/297439091 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8d80c8d50 con 0x7fa8d8161340 2023-11-28T11:12:02.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.836+0000 7fa8cf7fe700 1 -- 172.21.15.156:0/297439091 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4655 (secure 0 0 0) 0x7fa8d00570b0 con 0x7fa8d8161340 2023-11-28T11:12:02.838 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:02.839 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6006,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1991065152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1991065152},{"type":"v1","addr":"172.21.15.156:6836","nonce":1991065152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/43543030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":43543030},{"type":"v1","addr":"172.21.15.156:6837","nonce":43543030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/2480189069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2480189069},{"type":"v1","addr":"172.21.15.186:6834","nonce":2480189069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:52.367543+0000","modified":"2023-11-28T11:11:08.808905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6021":{"gid":6021,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6835/3811791267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3811791267},{"type":"v1","addr":"172.21.15.186:6835","nonce":3811791267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:02.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.838+0000 7fa8dd498700 1 -- 172.21.15.156:0/297439091 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa8b80682f0 msgr2=0x7fa8b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.838+0000 7fa8dd498700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa8b80682f0 0x7fa8b806a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0000f60 tx=0x7fa8c0009040 comp rx=0 tx=0).stop 2023-11-28T11:12:02.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.838+0000 7fa8dd498700 1 -- 172.21.15.156:0/297439091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8d8161340 msgr2=0x7fa8d81e7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.838+0000 7fa8dd498700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8d8161340 0x7fa8d81e7630 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80675d0 tx=0x7fa8d0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.838+0000 7fa8dd498700 1 -- 172.21.15.156:0/297439091 shutdown_connections 2023-11-28T11:12:02.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.838+0000 7fa8dd498700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8d8161340 0x7fa8d81e7630 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.838+0000 7fa8dd498700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d80d2320 0x7fa8d8150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.838+0000 7fa8dd498700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa8b80682f0 0x7fa8b806a7a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.838+0000 7fa8dd498700 1 --2- 172.21.15.156:0/297439091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8d815c9b0 0x7fa8d814fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.838+0000 7fa8dd498700 1 -- 172.21.15.156:0/297439091 >> 172.21.15.156:0/297439091 conn(0x7fa8d8072600 msgr2=0x7fa8d80b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:02.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.838+0000 7fa8dd498700 1 -- 172.21.15.156:0/297439091 shutdown_connections 2023-11-28T11:12:02.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.838+0000 7fa8dd498700 1 -- 172.21.15.156:0/297439091 wait complete. 2023-11-28T11:12:02.842 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 32 2023-11-28T11:12:02.852 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:02.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.970+0000 7fc364637700 1 Processor -- start 2023-11-28T11:12:02.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.970+0000 7fc364637700 1 -- start start 2023-11-28T11:12:02.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.971+0000 7fc364637700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc35c0d0a00 0x7fc35c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.971+0000 7fc364637700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc35c06b420 con 0x7fc35c0b2930 2023-11-28T11:12:02.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.971+0000 7fc364637700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc35c06b700 con 0x7fc35c0ce040 2023-11-28T11:12:02.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.971+0000 7fc364637700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35c06ff60 con 0x7fc35c0d0a00 2023-11-28T11:12:02.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.971+0000 7fc361bd2700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc35c0d0a00 0x7fc35c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.971+0000 7fc362bd4700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc35c0b2930 0x7fc35c06c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51270/0 (socket says 172.21.15.156:51270) 2023-11-28T11:12:02.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.971+0000 7fc362bd4700 1 -- 172.21.15.156:0/2517029988 learned_addr learned my addr 172.21.15.156:0/2517029988 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:02.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.971+0000 7fc361bd2700 1 -- 172.21.15.156:0/2517029988 >> v1:172.21.15.186:6789/0 conn(0x7fc35c0ce040 legacy=0x7fc35c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.971+0000 7fc361bd2700 1 -- 172.21.15.156:0/2517029988 >> v1:172.21.15.156:6789/0 conn(0x7fc35c0b2930 legacy=0x7fc35c06c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.971+0000 7fc361bd2700 1 -- 172.21.15.156:0/2517029988 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c06b9e0 con 0x7fc35c0d0a00 2023-11-28T11:12:02.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.971+0000 7fc3613d1700 1 -- 172.21.15.156:0/2517029988 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304088250 0 0) 0x7fc35c06b420 con 0x7fc35c0b2930 2023-11-28T11:12:02.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.972+0000 7fc361bd2700 1 --2- 172.21.15.156:0/2517029988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc35c0d0a00 0x7fc35c15df10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc358009840 tx=0x7fc358009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a822037b819fedda server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:02.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.972+0000 7fc3613d1700 1 -- 172.21.15.156:0/2517029988 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358011830 con 0x7fc35c0d0a00 2023-11-28T11:12:02.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.972+0000 7fc3613d1700 1 -- 172.21.15.156:0/2517029988 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc358011990 con 0x7fc35c0d0a00 2023-11-28T11:12:02.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.972+0000 7fc3613d1700 1 -- 172.21.15.156:0/2517029988 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358011d20 con 0x7fc35c0d0a00 2023-11-28T11:12:02.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.972+0000 7fc364637700 1 -- 172.21.15.156:0/2517029988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc35c0d0a00 msgr2=0x7fc35c15df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.972+0000 7fc364637700 1 --2- 172.21.15.156:0/2517029988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc35c0d0a00 0x7fc35c15df10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc358009840 tx=0x7fc358009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:02.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.972+0000 7fc364637700 1 -- 172.21.15.156:0/2517029988 shutdown_connections 2023-11-28T11:12:02.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.972+0000 7fc364637700 1 --2- 172.21.15.156:0/2517029988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc35c0d0a00 0x7fc35c15df10 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.972+0000 7fc364637700 1 -- 172.21.15.156:0/2517029988 >> 172.21.15.156:0/2517029988 conn(0x7fc35c06aed0 msgr2=0x7fc35c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:02.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.972+0000 7fc364637700 1 -- 172.21.15.156:0/2517029988 shutdown_connections 2023-11-28T11:12:02.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.972+0000 7fc364637700 1 -- 172.21.15.156:0/2517029988 wait complete. 2023-11-28T11:12:02.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.974+0000 7fc364637700 1 Processor -- start 2023-11-28T11:12:02.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.974+0000 7fc364637700 1 -- start start 2023-11-28T11:12:02.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.974+0000 7fc364637700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc35c0b2930 0x7fc35c14b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.974+0000 7fc364637700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35c0ce040 0x7fc35c0c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.974+0000 7fc364637700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc35c0d0a00 0x7fc35c0c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.974+0000 7fc364637700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c14b830 con 0x7fc35c0d0a00 2023-11-28T11:12:02.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.974+0000 7fc364637700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0c52c0 con 0x7fc35c0ce040 2023-11-28T11:12:02.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.974+0000 7fc364637700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0c55a0 con 0x7fc35c0b2930 2023-11-28T11:12:02.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.974+0000 7fc362bd4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc35c0d0a00 0x7fc35c0c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.974+0000 7fc362bd4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc35c0d0a00 0x7fc35c0c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36398/0 (socket says 172.21.15.156:36398) 2023-11-28T11:12:02.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.974+0000 7fc362bd4700 1 -- 172.21.15.156:0/1697552870 learned_addr learned my addr 172.21.15.156:0/1697552870 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:02.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc361bd2700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35c0ce040 0x7fc35c0c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc3623d3700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc35c0b2930 0x7fc35c14b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc362bd4700 1 -- 172.21.15.156:0/1697552870 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc35c0b2930 msgr2=0x7fc35c14b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc362bd4700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc35c0b2930 0x7fc35c14b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc362bd4700 1 -- 172.21.15.156:0/1697552870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35c0ce040 msgr2=0x7fc35c0c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc362bd4700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35c0ce040 0x7fc35c0c49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc362bd4700 1 -- 172.21.15.156:0/1697552870 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3580094a0 con 0x7fc35c0d0a00 2023-11-28T11:12:02.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc362bd4700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc35c0d0a00 0x7fc35c0c6360 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc354002230 tx=0x7fc35400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:02.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc352ffd700 1 -- 172.21.15.156:0/1697552870 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35400ac20 con 0x7fc35c0d0a00 2023-11-28T11:12:02.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc352ffd700 1 -- 172.21.15.156:0/1697552870 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35400ada0 con 0x7fc35c0d0a00 2023-11-28T11:12:02.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc364637700 1 -- 172.21.15.156:0/1697552870 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c06bfa0 con 0x7fc35c0d0a00 2023-11-28T11:12:02.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc352ffd700 1 -- 172.21.15.156:0/1697552870 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354007310 con 0x7fc35c0d0a00 2023-11-28T11:12:02.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.975+0000 7fc364637700 1 -- 172.21.15.156:0/1697552870 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c06bcc0 con 0x7fc35c0d0a00 2023-11-28T11:12:02.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.976+0000 7fc364637700 1 -- 172.21.15.156:0/1697552870 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35c0b2db0 con 0x7fc35c0d0a00 2023-11-28T11:12:02.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.976+0000 7fc352ffd700 1 -- 172.21.15.156:0/1697552870 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc354010070 con 0x7fc35c0d0a00 2023-11-28T11:12:02.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.977+0000 7fc352ffd700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc3400682f0 0x7fc34006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.977+0000 7fc352ffd700 1 -- 172.21.15.156:0/1697552870 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fc354085790 con 0x7fc35c0d0a00 2023-11-28T11:12:02.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.977+0000 7fc3623d3700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc3400682f0 0x7fc34006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.977+0000 7fc3623d3700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc3400682f0 0x7fc34006a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc35c1596b0 tx=0x7fc34c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:02.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:02.982+0000 7fc352ffd700 1 -- 172.21.15.156:0/1697552870 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc354051f10 con 0x7fc35c0d0a00 2023-11-28T11:12:03.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.162+0000 7fc364637700 1 -- 172.21.15.156:0/1697552870 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc35c068690 con 0x7fc35c0d0a00 2023-11-28T11:12:03.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.163+0000 7fc352ffd700 1 -- 172.21.15.156:0/1697552870 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18161 (secure 0 0 0) 0x7fc354055530 con 0x7fc35c0d0a00 2023-11-28T11:12:03.166 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:03.166 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":65,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:11:02.788873+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:50.055098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:51.350508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.33","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:03.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.165+0000 7fc364637700 1 -- 172.21.15.156:0/1697552870 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc3400682f0 msgr2=0x7fc34006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.165+0000 7fc364637700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc3400682f0 0x7fc34006a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc35c1596b0 tx=0x7fc34c009040 comp rx=0 tx=0).stop 2023-11-28T11:12:03.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.165+0000 7fc364637700 1 -- 172.21.15.156:0/1697552870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc35c0d0a00 msgr2=0x7fc35c0c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.165+0000 7fc364637700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc35c0d0a00 0x7fc35c0c6360 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc354002230 tx=0x7fc35400a060 comp rx=0 tx=0).stop 2023-11-28T11:12:03.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.165+0000 7fc364637700 1 -- 172.21.15.156:0/1697552870 shutdown_connections 2023-11-28T11:12:03.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.165+0000 7fc364637700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc35c0d0a00 0x7fc35c0c6360 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.165+0000 7fc364637700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc3400682f0 0x7fc34006a7a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.165+0000 7fc364637700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc35c0b2930 0x7fc35c14b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.165+0000 7fc364637700 1 --2- 172.21.15.156:0/1697552870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35c0ce040 0x7fc35c0c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.166+0000 7fc364637700 1 -- 172.21.15.156:0/1697552870 >> 172.21.15.156:0/1697552870 conn(0x7fc35c06aed0 msgr2=0x7fc35c15c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:03.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.166+0000 7fc364637700 1 -- 172.21.15.156:0/1697552870 shutdown_connections 2023-11-28T11:12:03.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.166+0000 7fc364637700 1 -- 172.21.15.156:0/1697552870 wait complete. 2023-11-28T11:12:03.180 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:12:03.180 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:03.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.299+0000 7f74f8023700 1 Processor -- start 2023-11-28T11:12:03.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.299+0000 7f74f8023700 1 -- start start 2023-11-28T11:12:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.299+0000 7f74f8023700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f015cef0 0x7f74f00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.299+0000 7f74f8023700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f00cc5f0 con 0x7f74f00cbdf0 2023-11-28T11:12:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.299+0000 7f74f8023700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f015d470 con 0x7f74f006ba90 2023-11-28T11:12:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.299+0000 7f74f8023700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f015d750 con 0x7f74f015cef0 2023-11-28T11:12:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f55be700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f74f00cbdf0 0x7f74f006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51280/0 (socket says 172.21.15.156:51280) 2023-11-28T11:12:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f55be700 1 -- 172.21.15.156:0/4248716000 learned_addr learned my addr 172.21.15.156:0/4248716000 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f5dbf700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f015cef0 0x7f74f00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041936147 0 0) 0x7f74f00cc5f0 con 0x7f74f00cbdf0 2023-11-28T11:12:03.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74dc0035b0 con 0x7f74f00cbdf0 2023-11-28T11:12:03.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144260673 0 0) 0x7f74f015d470 con 0x7f74f006ba90 2023-11-28T11:12:03.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74f00cc5f0 con 0x7f74f006ba90 2023-11-28T11:12:03.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3755292854 0 0) 0x7f74dc0035b0 con 0x7f74f00cbdf0 2023-11-28T11:12:03.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f015cef0 msgr2=0x7f74f00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:03.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f015cef0 0x7f74f00af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 >> v1:172.21.15.186:6789/0 conn(0x7f74f006ba90 legacy=0x7f74f00ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f015da30 con 0x7f74f00cbdf0 2023-11-28T11:12:03.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f74ec001670 con 0x7f74f00cbdf0 2023-11-28T11:12:03.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 >> v1:172.21.15.156:6789/0 conn(0x7f74f00cbdf0 legacy=0x7f74f006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc003bc0 0x7f74dc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f015cef0 0x7f74dc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74dc004d90 0x7f74dc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f015d750 con 0x7f74dc003bc0 2023-11-28T11:12:03.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc00bac0 con 0x7f74dc004d90 2023-11-28T11:12:03.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc00bc30 con 0x7f74f015cef0 2023-11-28T11:12:03.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f5dbf700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc003bc0 0x7f74dc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f55be700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f015cef0 0x7f74dc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.300+0000 7f74f65c0700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74dc004d90 0x7f74dc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f5dbf700 1 -- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f015cef0 msgr2=0x7f74dc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:03.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f5dbf700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f015cef0 0x7f74dc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f5dbf700 1 -- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74dc004d90 msgr2=0x7f74dc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:03.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f5dbf700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74dc004d90 0x7f74dc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f5dbf700 1 -- 172.21.15.156:0/4248716000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc0035b0 con 0x7f74dc003bc0 2023-11-28T11:12:03.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f55be700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f015cef0 0x7f74dc004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:03.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f65c0700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74dc004d90 0x7f74dc005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:03.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f5dbf700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc003bc0 0x7f74dc003fb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009b60 tx=0x7f74dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690a41519c654a0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:03.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e000e660 con 0x7f74dc003bc0 2023-11-28T11:12:03.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e000e7c0 con 0x7f74dc003bc0 2023-11-28T11:12:03.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f4dbd700 1 -- 172.21.15.156:0/4248716000 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e000eb30 con 0x7f74dc003bc0 2023-11-28T11:12:03.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f8023700 1 -- 172.21.15.156:0/4248716000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc003bc0 msgr2=0x7f74dc003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f8023700 1 --2- 172.21.15.156:0/4248716000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc003bc0 0x7f74dc003fb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009b60 tx=0x7f74dc005880 comp rx=0 tx=0).stop 2023-11-28T11:12:03.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f65c0700 1 -- 172.21.15.156:0/4248716000 reap_dead start 2023-11-28T11:12:03.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f8023700 1 -- 172.21.15.156:0/4248716000 shutdown_connections 2023-11-28T11:12:03.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f8023700 1 -- 172.21.15.156:0/4248716000 >> 172.21.15.156:0/4248716000 conn(0x7f74f00d0000 msgr2=0x7f74f00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:03.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f8023700 1 -- 172.21.15.156:0/4248716000 shutdown_connections 2023-11-28T11:12:03.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.301+0000 7f74f8023700 1 -- 172.21.15.156:0/4248716000 wait complete. 2023-11-28T11:12:03.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.303+0000 7f74f8023700 1 Processor -- start 2023-11-28T11:12:03.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.303+0000 7f74f8023700 1 -- start start 2023-11-28T11:12:03.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.303+0000 7f74f8023700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc003bc0 0x7f74f01ec870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.303+0000 7f74f8023700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74f00c3b10 0x7f74f00b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.303+0000 7f74f8023700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f01f0520 0x7f74f00cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.303+0000 7f74f8023700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00c67d0 con 0x7f74dc003bc0 2023-11-28T11:12:03.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.303+0000 7f74f8023700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00c4010 con 0x7f74f00c3b10 2023-11-28T11:12:03.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.303+0000 7f74f8023700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00c42f0 con 0x7f74f01f0520 2023-11-28T11:12:03.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.303+0000 7f74f5dbf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc003bc0 0x7f74f01ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.303+0000 7f74f5dbf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc003bc0 0x7f74f01ec870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36414/0 (socket says 172.21.15.156:36414) 2023-11-28T11:12:03.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74f5dbf700 1 -- 172.21.15.156:0/392169851 learned_addr learned my addr 172.21.15.156:0/392169851 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:03.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74f55be700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74f00c3b10 0x7f74f00b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74f65c0700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f01f0520 0x7f74f00cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74f5dbf700 1 -- 172.21.15.156:0/392169851 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f01f0520 msgr2=0x7f74f00cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74f5dbf700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f01f0520 0x7f74f00cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74f5dbf700 1 -- 172.21.15.156:0/392169851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74f00c3b10 msgr2=0x7f74f00b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74f5dbf700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74f00c3b10 0x7f74f00b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74f5dbf700 1 -- 172.21.15.156:0/392169851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f015dff0 con 0x7f74dc003bc0 2023-11-28T11:12:03.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74f5dbf700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc003bc0 0x7f74f01ec870 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f74e000e4b0 tx=0x7f74e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:03.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74e67fc700 1 -- 172.21.15.156:0/392169851 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e000e660 con 0x7f74dc003bc0 2023-11-28T11:12:03.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74e67fc700 1 -- 172.21.15.156:0/392169851 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e0003ad0 con 0x7f74dc003bc0 2023-11-28T11:12:03.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74f8023700 1 -- 172.21.15.156:0/392169851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f015dd10 con 0x7f74dc003bc0 2023-11-28T11:12:03.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74e67fc700 1 -- 172.21.15.156:0/392169851 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0003e80 con 0x7f74dc003bc0 2023-11-28T11:12:03.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.304+0000 7f74f8023700 1 -- 172.21.15.156:0/392169851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f00af400 con 0x7f74dc003bc0 2023-11-28T11:12:03.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.305+0000 7f74f8023700 1 -- 172.21.15.156:0/392169851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f00d0e60 con 0x7f74dc003bc0 2023-11-28T11:12:03.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.305+0000 7f74e67fc700 1 -- 172.21.15.156:0/392169851 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f74e0019030 con 0x7f74dc003bc0 2023-11-28T11:12:03.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.306+0000 7f74e67fc700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74d40682f0 0x7f74d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.306+0000 7f74e67fc700 1 -- 172.21.15.156:0/392169851 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f74e0087490 con 0x7f74dc003bc0 2023-11-28T11:12:03.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.306+0000 7f74f55be700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74d40682f0 0x7f74d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.306+0000 7f74f55be700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74d40682f0 0x7f74d406a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f74f00cd300 tx=0x7f74ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:03.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.310+0000 7f74e67fc700 1 -- 172.21.15.156:0/392169851 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f74e0053c70 con 0x7f74dc003bc0 2023-11-28T11:12:03.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.533+0000 7f74f8023700 1 -- 172.21.15.156:0/392169851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74f01ecdb0 con 0x7f74dc003bc0 2023-11-28T11:12:03.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.534+0000 7f74e67fc700 1 -- 172.21.15.156:0/392169851 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f74e0057290 con 0x7f74dc003bc0 2023-11-28T11:12:03.536 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:03.536 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6006,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1991065152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1991065152},{"type":"v1","addr":"172.21.15.156:6836","nonce":1991065152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/43543030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":43543030},{"type":"v1","addr":"172.21.15.156:6837","nonce":43543030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/2480189069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2480189069},{"type":"v1","addr":"172.21.15.186:6834","nonce":2480189069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:52.367543+0000","modified":"2023-11-28T11:11:08.808905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6021":{"gid":6021,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6835/3811791267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3811791267},{"type":"v1","addr":"172.21.15.186:6835","nonce":3811791267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:03.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.536+0000 7f74f8023700 1 -- 172.21.15.156:0/392169851 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74d40682f0 msgr2=0x7f74d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.536+0000 7f74f8023700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74d40682f0 0x7f74d406a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f74f00cd300 tx=0x7f74ec009040 comp rx=0 tx=0).stop 2023-11-28T11:12:03.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.536+0000 7f74f8023700 1 -- 172.21.15.156:0/392169851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc003bc0 msgr2=0x7f74f01ec870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.536+0000 7f74f8023700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc003bc0 0x7f74f01ec870 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f74e000e4b0 tx=0x7f74e00047c0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.536+0000 7f74f8023700 1 -- 172.21.15.156:0/392169851 shutdown_connections 2023-11-28T11:12:03.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.536+0000 7f74f8023700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74d40682f0 0x7f74d406a7a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.536+0000 7f74f8023700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74f01f0520 0x7f74f00cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.536+0000 7f74f8023700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc003bc0 0x7f74f01ec870 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.536+0000 7f74f8023700 1 --2- 172.21.15.156:0/392169851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74f00c3b10 0x7f74f00b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.536+0000 7f74f8023700 1 -- 172.21.15.156:0/392169851 >> 172.21.15.156:0/392169851 conn(0x7f74f00d0000 msgr2=0x7f74f006d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:03.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.536+0000 7f74f8023700 1 -- 172.21.15.156:0/392169851 shutdown_connections 2023-11-28T11:12:03.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.536+0000 7f74f8023700 1 -- 172.21.15.156:0/392169851 wait complete. 2023-11-28T11:12:03.540 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 32 2023-11-28T11:12:03.551 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:03.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.669+0000 7f214be8f700 1 Processor -- start 2023-11-28T11:12:03.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.669+0000 7f214be8f700 1 -- start start 2023-11-28T11:12:03.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.669+0000 7f214be8f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f21440cd8c0 0x7f2144140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.669+0000 7f214be8f700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f21440cdc90 0x7f21440b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.669+0000 7f214be8f700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214413ff00 con 0x7f21440b2e70 2023-11-28T11:12:03.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.669+0000 7f214be8f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21441404c0 con 0x7f21440cd8c0 2023-11-28T11:12:03.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.669+0000 7f214be8f700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214415aa80 con 0x7f21440cdc90 2023-11-28T11:12:03.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f2149c2b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f21440cd8c0 0x7f2144140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f214a42c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f21440b2e70 0x7f214406f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51304/0 (socket says 172.21.15.156:51304) 2023-11-28T11:12:03.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f214a42c700 1 -- 172.21.15.156:0/2646887598 learned_addr learned my addr 172.21.15.156:0/2646887598 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:03.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f214942a700 1 --2- 172.21.15.156:0/2646887598 >> v2:172.21.15.186:3300/0 conn(0x7f21440cdc90 0x7f21440b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f2149c2b700 1 -- 172.21.15.156:0/2646887598 >> v2:172.21.15.186:3300/0 conn(0x7f21440cdc90 msgr2=0x7f21440b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:03.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f2149c2b700 1 --2- 172.21.15.156:0/2646887598 >> v2:172.21.15.186:3300/0 conn(0x7f21440cdc90 0x7f21440b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f2149c2b700 1 -- 172.21.15.156:0/2646887598 >> v1:172.21.15.156:6789/0 conn(0x7f21440b2e70 legacy=0x7f214406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f2148c29700 1 -- 172.21.15.156:0/2646887598 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426808185 0 0) 0x7f214413ff00 con 0x7f21440b2e70 2023-11-28T11:12:03.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f2149c2b700 1 -- 172.21.15.156:0/2646887598 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21440b3310 con 0x7f21440cd8c0 2023-11-28T11:12:03.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f2149c2b700 1 --2- 172.21.15.156:0/2646887598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21440cd8c0 0x7f2144140950 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2144071c40 tx=0x7f2138009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b16f297b2f16ac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f2148c29700 1 -- 172.21.15.156:0/2646887598 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2138005e30 con 0x7f21440cd8c0 2023-11-28T11:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f2148c29700 1 -- 172.21.15.156:0/2646887598 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2138006030 con 0x7f21440cd8c0 2023-11-28T11:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.670+0000 7f2148c29700 1 -- 172.21.15.156:0/2646887598 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21380063c0 con 0x7f21440cd8c0 2023-11-28T11:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.671+0000 7f214be8f700 1 -- 172.21.15.156:0/2646887598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21440cd8c0 msgr2=0x7f2144140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.671+0000 7f214be8f700 1 --2- 172.21.15.156:0/2646887598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21440cd8c0 0x7f2144140950 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2144071c40 tx=0x7f2138009b00 comp rx=0 tx=0).stop 2023-11-28T11:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.671+0000 7f214be8f700 1 -- 172.21.15.156:0/2646887598 shutdown_connections 2023-11-28T11:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.671+0000 7f214be8f700 1 --2- 172.21.15.156:0/2646887598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21440cd8c0 0x7f2144140950 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.671+0000 7f214be8f700 1 --2- 172.21.15.156:0/2646887598 >> v2:172.21.15.186:3300/0 conn(0x7f21440cdc90 0x7f21440b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.671+0000 7f214be8f700 1 -- 172.21.15.156:0/2646887598 >> 172.21.15.156:0/2646887598 conn(0x7f214406aed0 msgr2=0x7f214406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:03.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.671+0000 7f214be8f700 1 -- 172.21.15.156:0/2646887598 shutdown_connections 2023-11-28T11:12:03.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.671+0000 7f214be8f700 1 -- 172.21.15.156:0/2646887598 wait complete. 2023-11-28T11:12:03.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.672+0000 7f214be8f700 1 Processor -- start 2023-11-28T11:12:03.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.672+0000 7f214be8f700 1 -- start start 2023-11-28T11:12:03.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.673+0000 7f214be8f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21440b2e70 0x7f21440d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.673+0000 7f214be8f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21440cd8c0 0x7f2144156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.673+0000 7f214be8f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21440cdc90 0x7f2144156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.673+0000 7f214be8f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21440cab20 con 0x7f21440cdc90 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.673+0000 7f214be8f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21440c57d0 con 0x7f21440cd8c0 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.673+0000 7f214be8f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2144157300 con 0x7f21440b2e70 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f214a42c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21440cdc90 0x7f2144156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f214a42c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21440cdc90 0x7f2144156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36432/0 (socket says 172.21.15.156:36432) 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f214a42c700 1 -- 172.21.15.156:0/2172373708 learned_addr learned my addr 172.21.15.156:0/2172373708 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f2149c2b700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21440b2e70 0x7f21440d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f214942a700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21440cd8c0 0x7f2144156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f214a42c700 1 -- 172.21.15.156:0/2172373708 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21440b2e70 msgr2=0x7f21440d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f214a42c700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21440b2e70 0x7f21440d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f214a42c700 1 -- 172.21.15.156:0/2172373708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21440cd8c0 msgr2=0x7f2144156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f214a42c700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21440cd8c0 0x7f2144156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f214a42c700 1 -- 172.21.15.156:0/2172373708 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2138009430 con 0x7f21440cdc90 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f214a42c700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21440cdc90 0x7f2144156c60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f213c00cc70 tx=0x7f213c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f21367fc700 1 -- 172.21.15.156:0/2172373708 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c00aac0 con 0x7f21440cdc90 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f21367fc700 1 -- 172.21.15.156:0/2172373708 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c00ac40 con 0x7f21440cdc90 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f214be8f700 1 -- 172.21.15.156:0/2172373708 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2144140150 con 0x7f21440cdc90 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f21367fc700 1 -- 172.21.15.156:0/2172373708 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c0070d0 con 0x7f21440cdc90 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.674+0000 7f214be8f700 1 -- 172.21.15.156:0/2172373708 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21440b35f0 con 0x7f21440cdc90 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.675+0000 7f214be8f700 1 -- 172.21.15.156:0/2172373708 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21440ce100 con 0x7f21440cdc90 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.676+0000 7f21367fc700 1 -- 172.21.15.156:0/2172373708 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f213c01a030 con 0x7f21440cdc90 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.676+0000 7f21367fc700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f21280682f0 0x7f212806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.676+0000 7f21367fc700 1 -- 172.21.15.156:0/2172373708 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f213c085970 con 0x7f21440cdc90 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.676+0000 7f2149c2b700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f21280682f0 0x7f212806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.676+0000 7f2149c2b700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f21280682f0 0x7f212806a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2138009b40 tx=0x7f2138008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:03.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.681+0000 7f21367fc700 1 -- 172.21.15.156:0/2172373708 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f213c052150 con 0x7f21440cdc90 2023-11-28T11:12:03.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.905+0000 7f214be8f700 1 -- 172.21.15.156:0/2172373708 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2144068690 con 0x7f21440cdc90 2023-11-28T11:12:03.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.905+0000 7f21367fc700 1 -- 172.21.15.156:0/2172373708 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f213c018090 con 0x7f21440cdc90 2023-11-28T11:12:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:03.908 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6006,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1991065152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1991065152},{"type":"v1","addr":"172.21.15.156:6836","nonce":1991065152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/43543030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":43543030},{"type":"v1","addr":"172.21.15.156:6837","nonce":43543030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/2480189069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2480189069},{"type":"v1","addr":"172.21.15.186:6834","nonce":2480189069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:52.367543+0000","modified":"2023-11-28T11:11:08.808905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6021":{"gid":6021,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6835/3811791267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":3811791267},{"type":"v1","addr":"172.21.15.186:6835","nonce":3811791267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:03.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.907+0000 7f214be8f700 1 -- 172.21.15.156:0/2172373708 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f21280682f0 msgr2=0x7f212806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.907+0000 7f214be8f700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f21280682f0 0x7f212806a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2138009b40 tx=0x7f2138008070 comp rx=0 tx=0).stop 2023-11-28T11:12:03.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.907+0000 7f214be8f700 1 -- 172.21.15.156:0/2172373708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21440cdc90 msgr2=0x7f2144156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.907+0000 7f214be8f700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21440cdc90 0x7f2144156c60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f213c00cc70 tx=0x7f213c00cf80 comp rx=0 tx=0).stop 2023-11-28T11:12:03.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.907+0000 7f214be8f700 1 -- 172.21.15.156:0/2172373708 shutdown_connections 2023-11-28T11:12:03.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.907+0000 7f214be8f700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21440cdc90 0x7f2144156c60 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.907+0000 7f214be8f700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f21280682f0 0x7f212806a7a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.907+0000 7f214be8f700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21440b2e70 0x7f21440d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.907+0000 7f214be8f700 1 --2- 172.21.15.156:0/2172373708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21440cd8c0 0x7f2144156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.907+0000 7f214be8f700 1 -- 172.21.15.156:0/2172373708 >> 172.21.15.156:0/2172373708 conn(0x7f214406aed0 msgr2=0x7f2144159d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:03.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.907+0000 7f214be8f700 1 -- 172.21.15.156:0/2172373708 shutdown_connections 2023-11-28T11:12:03.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:03.907+0000 7f214be8f700 1 -- 172.21.15.156:0/2172373708 wait complete. 2023-11-28T11:12:03.911 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 32 2023-11-28T11:12:03.921 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:04.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.042+0000 7f43ce705700 1 Processor -- start 2023-11-28T11:12:04.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.042+0000 7f43ce705700 1 -- start start 2023-11-28T11:12:04.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.043+0000 7f43ce705700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c806f850 0x7f43c806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.043+0000 7f43ce705700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c813ff00 con 0x7f43c8140950 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.043+0000 7f43ce705700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c81404c0 con 0x7f43c80cd8c0 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.043+0000 7f43ce705700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c8140dc0 con 0x7f43c806f850 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.043+0000 7f43c77fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f43c8140950 0x7f43c80b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51316/0 (socket says 172.21.15.156:51316) 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.043+0000 7f43c77fe700 1 -- 172.21.15.156:0/1011626520 learned_addr learned my addr 172.21.15.156:0/1011626520 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779182641 0 0) 0x7f43c813ff00 con 0x7f43c8140950 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b40035b0 con 0x7f43c8140950 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43ccca2700 1 --2- 172.21.15.156:0/1011626520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c806f850 0x7f43c806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 665185043 0 0) 0x7f43b40035b0 con 0x7f43c8140950 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c806f850 msgr2=0x7f43c806fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 --2- 172.21.15.156:0/1011626520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c806f850 0x7f43c806fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 >> v1:172.21.15.186:6789/0 conn(0x7f43c80cd8c0 legacy=0x7f43c80cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c815aa80 con 0x7f43c8140950 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f43b0001670 con 0x7f43c8140950 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 >> v1:172.21.15.156:6789/0 conn(0x7f43c8140950 legacy=0x7f43c80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 --2- 172.21.15.156:0/1011626520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c806f850 0x7f43b4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 --2- 172.21.15.156:0/1011626520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43b4003fb0 0x7f43b4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 --2- 172.21.15.156:0/1011626520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b4004d90 0x7f43b4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c8140dc0 con 0x7f43b4004d90 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b400bba0 con 0x7f43b4003fb0 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b400bd60 con 0x7f43c806f850 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43ccca2700 1 --2- 172.21.15.156:0/1011626520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b4004d90 0x7f43b4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c77fe700 1 --2- 172.21.15.156:0/1011626520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43b4003fb0 0x7f43b4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43c7fff700 1 --2- 172.21.15.156:0/1011626520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c806f850 0x7f43b4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43ccca2700 1 -- 172.21.15.156:0/1011626520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c806f850 msgr2=0x7f43b4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:04.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43ccca2700 1 --2- 172.21.15.156:0/1011626520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c806f850 0x7f43b4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43ccca2700 1 -- 172.21.15.156:0/1011626520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43b4003fb0 msgr2=0x7f43b4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43ccca2700 1 --2- 172.21.15.156:0/1011626520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43b4003fb0 0x7f43b4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.044+0000 7f43ccca2700 1 -- 172.21.15.156:0/1011626520 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b40035b0 con 0x7f43b4004d90 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.045+0000 7f43ccca2700 1 --2- 172.21.15.156:0/1011626520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b4004d90 0x7f43b4005200 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009f90 tx=0x7f43b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4bad9ed3baf6c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.045+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c000ec30 con 0x7f43b4004d90 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.045+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c000f040 con 0x7f43b4004d90 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.045+0000 7f43c6ffd700 1 -- 172.21.15.156:0/1011626520 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0014480 con 0x7f43b4004d90 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.045+0000 7f43ce705700 1 -- 172.21.15.156:0/1011626520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b4004d90 msgr2=0x7f43b4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.045+0000 7f43ce705700 1 --2- 172.21.15.156:0/1011626520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b4004d90 0x7f43b4005200 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009f90 tx=0x7f43b4005880 comp rx=0 tx=0).stop 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.045+0000 7f43ccca2700 1 -- 172.21.15.156:0/1011626520 reap_dead start 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.045+0000 7f43ce705700 1 -- 172.21.15.156:0/1011626520 shutdown_connections 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.045+0000 7f43ce705700 1 -- 172.21.15.156:0/1011626520 >> 172.21.15.156:0/1011626520 conn(0x7f43c806aed0 msgr2=0x7f43c806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:04.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.045+0000 7f43ce705700 1 -- 172.21.15.156:0/1011626520 shutdown_connections 2023-11-28T11:12:04.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.045+0000 7f43ce705700 1 -- 172.21.15.156:0/1011626520 wait complete. 2023-11-28T11:12:04.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.047+0000 7f43ce705700 1 Processor -- start 2023-11-28T11:12:04.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.047+0000 7f43ce705700 1 -- start start 2023-11-28T11:12:04.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.047+0000 7f43ce705700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c81565e0 0x7f43c80b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.047+0000 7f43ce705700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c80b69d0 0x7f43c80c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.047+0000 7f43ce705700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c80c9bc0 0x7f43c80c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.047+0000 7f43ce705700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80cb020 con 0x7f43c80c9bc0 2023-11-28T11:12:04.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.047+0000 7f43ce705700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80ca430 con 0x7f43c81565e0 2023-11-28T11:12:04.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.047+0000 7f43ce705700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80ca6f0 con 0x7f43c80b69d0 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43ccca2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c80c9bc0 0x7f43c80c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43ccca2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c80c9bc0 0x7f43c80c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36452/0 (socket says 172.21.15.156:36452) 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43ccca2700 1 -- 172.21.15.156:0/588371188 learned_addr learned my addr 172.21.15.156:0/588371188 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43c7fff700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c81565e0 0x7f43c80b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43c77fe700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c80b69d0 0x7f43c80c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43ccca2700 1 -- 172.21.15.156:0/588371188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c80b69d0 msgr2=0x7f43c80c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43ccca2700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c80b69d0 0x7f43c80c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43ccca2700 1 -- 172.21.15.156:0/588371188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c81565e0 msgr2=0x7f43c80b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43ccca2700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c81565e0 0x7f43c80b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43ccca2700 1 -- 172.21.15.156:0/588371188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c80b3310 con 0x7f43c80c9bc0 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43ccca2700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c80c9bc0 0x7f43c80c5a30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f43b4005810 tx=0x7f43c0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43c4ff9700 1 -- 172.21.15.156:0/588371188 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0009410 con 0x7f43c80c9bc0 2023-11-28T11:12:04.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43c4ff9700 1 -- 172.21.15.156:0/588371188 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c0022070 con 0x7f43c80c9bc0 2023-11-28T11:12:04.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43c4ff9700 1 -- 172.21.15.156:0/588371188 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0014cb0 con 0x7f43c80c9bc0 2023-11-28T11:12:04.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43ce705700 1 -- 172.21.15.156:0/588371188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c80b3040 con 0x7f43c80c9bc0 2023-11-28T11:12:04.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.048+0000 7f43ce705700 1 -- 172.21.15.156:0/588371188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c80b35f0 con 0x7f43c80c9bc0 2023-11-28T11:12:04.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.049+0000 7f43ce705700 1 -- 172.21.15.156:0/588371188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c80b3ce0 con 0x7f43c80c9bc0 2023-11-28T11:12:04.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.050+0000 7f43c4ff9700 1 -- 172.21.15.156:0/588371188 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f43c001e430 con 0x7f43c80c9bc0 2023-11-28T11:12:04.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.050+0000 7f43c4ff9700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f43a80682f0 0x7f43a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.050+0000 7f43c4ff9700 1 -- 172.21.15.156:0/588371188 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f43c0024070 con 0x7f43c80c9bc0 2023-11-28T11:12:04.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.050+0000 7f43c7fff700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f43a80682f0 0x7f43a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.050+0000 7f43c7fff700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f43a80682f0 0x7f43a806a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f43c80cfb40 tx=0x7f43b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:04.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.054+0000 7f43c4ff9700 1 -- 172.21.15.156:0/588371188 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f43c0053ad0 con 0x7f43c80c9bc0 2023-11-28T11:12:04.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.239+0000 7f43ce705700 1 -- 172.21.15.156:0/588371188 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43c8068690 con 0x7f43c80c9bc0 2023-11-28T11:12:04.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.240+0000 7f43c4ff9700 1 -- 172.21.15.156:0/588371188 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18161 (secure 0 0 0) 0x7f43c00570f0 con 0x7f43c80c9bc0 2023-11-28T11:12:04.243 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:04.243 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":65,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:11:02.788873+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:50.055098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:51.350508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.33","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:04.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.242+0000 7f43ce705700 1 -- 172.21.15.156:0/588371188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f43a80682f0 msgr2=0x7f43a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.242+0000 7f43ce705700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f43a80682f0 0x7f43a806a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f43c80cfb40 tx=0x7f43b8009040 comp rx=0 tx=0).stop 2023-11-28T11:12:04.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.242+0000 7f43ce705700 1 -- 172.21.15.156:0/588371188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c80c9bc0 msgr2=0x7f43c80c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.242+0000 7f43ce705700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c80c9bc0 0x7f43c80c5a30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f43b4005810 tx=0x7f43c0009750 comp rx=0 tx=0).stop 2023-11-28T11:12:04.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.242+0000 7f43ce705700 1 -- 172.21.15.156:0/588371188 shutdown_connections 2023-11-28T11:12:04.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.243+0000 7f43ce705700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43c80c9bc0 0x7f43c80c5a30 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.243+0000 7f43ce705700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c81565e0 0x7f43c80b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.243+0000 7f43ce705700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f43a80682f0 0x7f43a806a7a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.243+0000 7f43ce705700 1 --2- 172.21.15.156:0/588371188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43c80b69d0 0x7f43c80c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.243+0000 7f43ce705700 1 -- 172.21.15.156:0/588371188 >> 172.21.15.156:0/588371188 conn(0x7f43c806aed0 msgr2=0x7f43c813f580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:04.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.243+0000 7f43ce705700 1 -- 172.21.15.156:0/588371188 shutdown_connections 2023-11-28T11:12:04.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.243+0000 7f43ce705700 1 -- 172.21.15.156:0/588371188 wait complete. 2023-11-28T11:12:04.258 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:12:04.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.378+0000 7f91e8c95700 1 Processor -- start 2023-11-28T11:12:04.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.378+0000 7f91e8c95700 1 -- start start 2023-11-28T11:12:04.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.379+0000 7f91e8c95700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91e406dee0 0x7f91e406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.379+0000 7f91e8c95700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e415d060 con 0x7f91e4160540 2023-11-28T11:12:04.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.379+0000 7f91e8c95700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e415d340 con 0x7f91e4071040 2023-11-28T11:12:04.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.379+0000 7f91e8c95700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e415d620 con 0x7f91e406dee0 2023-11-28T11:12:04.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.379+0000 7f91e2d9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f91e4160540 0x7f91e41609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51338/0 (socket says 172.21.15.156:51338) 2023-11-28T11:12:04.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.379+0000 7f91e2d9d700 1 -- 172.21.15.156:0/2453670116 learned_addr learned my addr 172.21.15.156:0/2453670116 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:04.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e1d9b700 1 --2- 172.21.15.156:0/2453670116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91e406dee0 0x7f91e406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658987973 0 0) 0x7f91e415d060 con 0x7f91e4160540 2023-11-28T11:12:04.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91cc0035b0 con 0x7f91e4160540 2023-11-28T11:12:04.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3325983871 0 0) 0x7f91cc0035b0 con 0x7f91e4160540 2023-11-28T11:12:04.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91e406dee0 msgr2=0x7f91e406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:04.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 --2- 172.21.15.156:0/2453670116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91e406dee0 0x7f91e406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 >> v1:172.21.15.186:6789/0 conn(0x7f91e4071040 legacy=0x7f91e4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e415d900 con 0x7f91e4160540 2023-11-28T11:12:04.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f91dc0019f0 con 0x7f91e4160540 2023-11-28T11:12:04.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 >> v1:172.21.15.156:6789/0 conn(0x7f91e4160540 legacy=0x7f91e41609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 --2- 172.21.15.156:0/2453670116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91cc003a30 0x7f91cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 --2- 172.21.15.156:0/2453670116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91e406dee0 0x7f91cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 --2- 172.21.15.156:0/2453670116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91cc004d90 0x7f91cc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e415d620 con 0x7f91cc004d90 2023-11-28T11:12:04.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc00bb20 con 0x7f91cc003a30 2023-11-28T11:12:04.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc00bce0 con 0x7f91e406dee0 2023-11-28T11:12:04.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e2d9d700 1 --2- 172.21.15.156:0/2453670116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91cc004d90 0x7f91cc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e1d9b700 1 --2- 172.21.15.156:0/2453670116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91e406dee0 0x7f91cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e259c700 1 --2- 172.21.15.156:0/2453670116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91cc003a30 0x7f91cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e2d9d700 1 -- 172.21.15.156:0/2453670116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91e406dee0 msgr2=0x7f91cc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:04.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e2d9d700 1 --2- 172.21.15.156:0/2453670116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91e406dee0 0x7f91cc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e2d9d700 1 -- 172.21.15.156:0/2453670116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91cc003a30 msgr2=0x7f91cc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:04.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e2d9d700 1 --2- 172.21.15.156:0/2453670116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91cc003a30 0x7f91cc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e2d9d700 1 -- 172.21.15.156:0/2453670116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0035b0 con 0x7f91cc004d90 2023-11-28T11:12:04.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.380+0000 7f91e2d9d700 1 --2- 172.21.15.156:0/2453670116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91cc004d90 0x7f91cc005200 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00a830 tx=0x7f91cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9661db158ba3bc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:04.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.381+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc012600 con 0x7f91cc004d90 2023-11-28T11:12:04.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.381+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc012780 con 0x7f91cc004d90 2023-11-28T11:12:04.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.381+0000 7f91e159a700 1 -- 172.21.15.156:0/2453670116 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc012b30 con 0x7f91cc004d90 2023-11-28T11:12:04.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.381+0000 7f91e8c95700 1 -- 172.21.15.156:0/2453670116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91cc004d90 msgr2=0x7f91cc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.381+0000 7f91e8c95700 1 --2- 172.21.15.156:0/2453670116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91cc004d90 0x7f91cc005200 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00a830 tx=0x7f91cc005880 comp rx=0 tx=0).stop 2023-11-28T11:12:04.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.381+0000 7f91e2d9d700 1 -- 172.21.15.156:0/2453670116 reap_dead start 2023-11-28T11:12:04.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.381+0000 7f91e8c95700 1 -- 172.21.15.156:0/2453670116 shutdown_connections 2023-11-28T11:12:04.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.381+0000 7f91e8c95700 1 -- 172.21.15.156:0/2453670116 >> 172.21.15.156:0/2453670116 conn(0x7f91e40d0000 msgr2=0x7f91e40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:04.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.381+0000 7f91e8c95700 1 -- 172.21.15.156:0/2453670116 shutdown_connections 2023-11-28T11:12:04.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.381+0000 7f91e8c95700 1 -- 172.21.15.156:0/2453670116 wait complete. 2023-11-28T11:12:04.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.383+0000 7f91e8c95700 1 Processor -- start 2023-11-28T11:12:04.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.383+0000 7f91e8c95700 1 -- start start 2023-11-28T11:12:04.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e8c95700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91cc004d90 0x7f91e40c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e8c95700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91e415a8d0 0x7f91e40b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e8c95700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91e40b43c0 0x7f91e40b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e8c95700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e415b140 con 0x7f91e415a8d0 2023-11-28T11:12:04.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e8c95700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e415b420 con 0x7f91e40b43c0 2023-11-28T11:12:04.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e8c95700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e415b700 con 0x7f91cc004d90 2023-11-28T11:12:04.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e1d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91e415a8d0 0x7f91e40b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e1d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91e415a8d0 0x7f91e40b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36470/0 (socket says 172.21.15.156:36470) 2023-11-28T11:12:04.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e1d9b700 1 -- 172.21.15.156:0/4256377112 learned_addr learned my addr 172.21.15.156:0/4256377112 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:04.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e259c700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91cc004d90 0x7f91e40c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e2d9d700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91e40b43c0 0x7f91e40b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e1d9b700 1 -- 172.21.15.156:0/4256377112 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91cc004d90 msgr2=0x7f91e40c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e1d9b700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91cc004d90 0x7f91e40c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e1d9b700 1 -- 172.21.15.156:0/4256377112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91e40b43c0 msgr2=0x7f91e40b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e1d9b700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91e40b43c0 0x7f91e40b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e1d9b700 1 -- 172.21.15.156:0/4256377112 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e406a270 con 0x7f91e415a8d0 2023-11-28T11:12:04.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e1d9b700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91e415a8d0 0x7f91e40b3e80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f91d4011aa0 tx=0x7f91d4011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:04.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91d2ffd700 1 -- 172.21.15.156:0/4256377112 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4011ee0 con 0x7f91e415a8d0 2023-11-28T11:12:04.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91d2ffd700 1 -- 172.21.15.156:0/4256377112 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d400f940 con 0x7f91e415a8d0 2023-11-28T11:12:04.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91d2ffd700 1 -- 172.21.15.156:0/4256377112 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d400fcf0 con 0x7f91e415a8d0 2023-11-28T11:12:04.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e8c95700 1 -- 172.21.15.156:0/4256377112 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e415ccd0 con 0x7f91e415a8d0 2023-11-28T11:12:04.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.384+0000 7f91e8c95700 1 -- 172.21.15.156:0/4256377112 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e415bcc0 con 0x7f91e415a8d0 2023-11-28T11:12:04.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.385+0000 7f91e8c95700 1 -- 172.21.15.156:0/4256377112 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e415c030 con 0x7f91e415a8d0 2023-11-28T11:12:04.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.386+0000 7f91d2ffd700 1 -- 172.21.15.156:0/4256377112 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f91d4014440 con 0x7f91e415a8d0 2023-11-28T11:12:04.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.386+0000 7f91d2ffd700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f91c40682f0 0x7f91c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.386+0000 7f91d2ffd700 1 -- 172.21.15.156:0/4256377112 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f91d40864b0 con 0x7f91e415a8d0 2023-11-28T11:12:04.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.386+0000 7f91e259c700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f91c40682f0 0x7f91c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.387+0000 7f91e259c700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f91c40682f0 0x7f91c406a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f91d8000c00 tx=0x7f91d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:04.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.391+0000 7f91d2ffd700 1 -- 172.21.15.156:0/4256377112 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f91d4052da0 con 0x7f91e415a8d0 2023-11-28T11:12:04.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.617+0000 7f91e8c95700 1 -- 172.21.15.156:0/4256377112 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f91e406a400 con 0x7f91e415a8d0 2023-11-28T11:12:04.958 INFO:tasks.ceph.mon.a.smithi156.stderr:2023-11-28T11:12:04.957+0000 7f6a4eb7c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:12:04.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.963+0000 7f91d2ffd700 1 -- 172.21.15.156:0/4256377112 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f91e406a400 con 0x7f91e415a8d0 2023-11-28T11:12:04.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.966+0000 7f91e8c95700 1 -- 172.21.15.156:0/4256377112 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f91c40682f0 msgr2=0x7f91c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.966+0000 7f91e8c95700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f91c40682f0 0x7f91c406a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f91d8000c00 tx=0x7f91d8009040 comp rx=0 tx=0).stop 2023-11-28T11:12:04.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.966+0000 7f91e8c95700 1 -- 172.21.15.156:0/4256377112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91e415a8d0 msgr2=0x7f91e40b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.966+0000 7f91e8c95700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91e415a8d0 0x7f91e40b3e80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f91d4011aa0 tx=0x7f91d4011db0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.966+0000 7f91e8c95700 1 -- 172.21.15.156:0/4256377112 shutdown_connections 2023-11-28T11:12:04.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.966+0000 7f91e8c95700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91e40b43c0 0x7f91e40b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.966+0000 7f91e8c95700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f91c40682f0 0x7f91c406a7a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.966+0000 7f91e8c95700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91cc004d90 0x7f91e40c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.966+0000 7f91e8c95700 1 --2- 172.21.15.156:0/4256377112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91e415a8d0 0x7f91e40b3e80 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.966+0000 7f91e8c95700 1 -- 172.21.15.156:0/4256377112 >> 172.21.15.156:0/4256377112 conn(0x7f91e40d0000 msgr2=0x7f91e40aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:04.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.966+0000 7f91e8c95700 1 -- 172.21.15.156:0/4256377112 shutdown_connections 2023-11-28T11:12:04.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:04.966+0000 7f91e8c95700 1 -- 172.21.15.156:0/4256377112 wait complete. 2023-11-28T11:12:04.969 INFO:teuthology.orchestra.run.smithi156.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:12:04.969 INFO:tasks.ceph.mds.d.smithi186.stderr: -7302> 2023-11-28T11:10:57.865+0000 7f634dda2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:12:04.973 INFO:tasks.ceph.mds.d.smithi186.stderr: -3695> 2023-11-28T11:11:05.200+0000 7f634a59b700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6186=0-4194304@1} caps={6186=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6186 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55dbdd779600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:12:04.979 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:12:05.003 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-11-28T11:12:05.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.101+0000 7fea3b442700 1 Processor -- start 2023-11-28T11:12:05.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.101+0000 7fea3b442700 1 -- start start 2023-11-28T11:12:05.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea3b442700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fea3406dee0 0x7fea3406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea3b442700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3415d060 con 0x7fea34071040 2023-11-28T11:12:05.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea3b442700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3415d340 con 0x7fea34160540 2023-11-28T11:12:05.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea3b442700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3415d620 con 0x7fea3406dee0 2023-11-28T11:12:05.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea391de700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fea34071040 0x7fea34071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51354/0 (socket says 172.21.15.156:51354) 2023-11-28T11:12:05.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea391de700 1 -- 172.21.15.156:0/202333495 learned_addr learned my addr 172.21.15.156:0/202333495 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:05.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea389dd700 1 --2- 172.21.15.156:0/202333495 >> v2:172.21.15.186:3300/0 conn(0x7fea3406dee0 0x7fea3406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675401933 0 0) 0x7fea3415d060 con 0x7fea34071040 2023-11-28T11:12:05.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea200035b0 con 0x7fea34071040 2023-11-28T11:12:05.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 283974675 0 0) 0x7fea200035b0 con 0x7fea34071040 2023-11-28T11:12:05.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 >> v2:172.21.15.186:3300/0 conn(0x7fea3406dee0 msgr2=0x7fea3406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:05.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea2bfff700 1 --2- 172.21.15.156:0/202333495 >> v2:172.21.15.186:3300/0 conn(0x7fea3406dee0 0x7fea3406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 >> v1:172.21.15.186:6789/0 conn(0x7fea34160540 legacy=0x7fea34160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3415d900 con 0x7fea34071040 2023-11-28T11:12:05.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7fea30001670 con 0x7fea34071040 2023-11-28T11:12:05.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 >> v1:172.21.15.156:6789/0 conn(0x7fea34071040 legacy=0x7fea34071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea2bfff700 1 --2- 172.21.15.156:0/202333495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea20003a00 0x7fea20003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea2bfff700 1 --2- 172.21.15.156:0/202333495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea20004850 0x7fea20004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.102+0000 7fea2bfff700 1 --2- 172.21.15.156:0/202333495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea20005320 0x7fea2000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3415d620 con 0x7fea20004850 2023-11-28T11:12:05.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea20014a60 con 0x7fea20005320 2023-11-28T11:12:05.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea20014c20 con 0x7fea20003a00 2023-11-28T11:12:05.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea389dd700 1 --2- 172.21.15.156:0/202333495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea20004850 0x7fea20004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea399df700 1 --2- 172.21.15.156:0/202333495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea20005320 0x7fea2000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea391de700 1 --2- 172.21.15.156:0/202333495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea20003a00 0x7fea20003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea389dd700 1 -- 172.21.15.156:0/202333495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea20003a00 msgr2=0x7fea20003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:05.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea389dd700 1 --2- 172.21.15.156:0/202333495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea20003a00 0x7fea20003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea389dd700 1 -- 172.21.15.156:0/202333495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea20005320 msgr2=0x7fea2000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:05.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea389dd700 1 --2- 172.21.15.156:0/202333495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea20005320 0x7fea2000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea399df700 1 -- 172.21.15.156:0/202333495 reap_dead start 2023-11-28T11:12:05.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea389dd700 1 -- 172.21.15.156:0/202333495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea200035b0 con 0x7fea20004850 2023-11-28T11:12:05.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea389dd700 1 --2- 172.21.15.156:0/202333495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea20004850 0x7fea20004cc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fea24009b60 tx=0x7fea20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0122618cef366e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:05.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2400e6c0 con 0x7fea20004850 2023-11-28T11:12:05.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2400e820 con 0x7fea20004850 2023-11-28T11:12:05.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.103+0000 7fea2bfff700 1 -- 172.21.15.156:0/202333495 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2400eb90 con 0x7fea20004850 2023-11-28T11:12:05.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.104+0000 7fea3b442700 1 -- 172.21.15.156:0/202333495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea20004850 msgr2=0x7fea20004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.104+0000 7fea3b442700 1 --2- 172.21.15.156:0/202333495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea20004850 0x7fea20004cc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fea24009b60 tx=0x7fea20005880 comp rx=0 tx=0).stop 2023-11-28T11:12:05.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.104+0000 7fea3b442700 1 -- 172.21.15.156:0/202333495 shutdown_connections 2023-11-28T11:12:05.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.104+0000 7fea3b442700 1 --2- 172.21.15.156:0/202333495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea20004850 0x7fea20004cc0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.104+0000 7fea3b442700 1 -- 172.21.15.156:0/202333495 >> 172.21.15.156:0/202333495 conn(0x7fea340d0000 msgr2=0x7fea340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:05.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.104+0000 7fea3b442700 1 -- 172.21.15.156:0/202333495 shutdown_connections 2023-11-28T11:12:05.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.104+0000 7fea3b442700 1 -- 172.21.15.156:0/202333495 wait complete. 2023-11-28T11:12:05.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.105+0000 7fea3b442700 1 Processor -- start 2023-11-28T11:12:05.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.105+0000 7fea3b442700 1 -- start start 2023-11-28T11:12:05.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea3b442700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea20004850 0x7fea34150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea3b442700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea34150050 0x7fea340c5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea3b442700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea340c5970 0x7fea340c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea3b442700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea341504c0 con 0x7fea340c5970 2023-11-28T11:12:05.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea3b442700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea340c64e0 con 0x7fea20004850 2023-11-28T11:12:05.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea3b442700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea340b4f80 con 0x7fea34150050 2023-11-28T11:12:05.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea399df700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea340c5970 0x7fea340c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea399df700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea340c5970 0x7fea340c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36492/0 (socket says 172.21.15.156:36492) 2023-11-28T11:12:05.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea399df700 1 -- 172.21.15.156:0/2414349128 learned_addr learned my addr 172.21.15.156:0/2414349128 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:05.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea389dd700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea34150050 0x7fea340c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea391de700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea20004850 0x7fea34150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea399df700 1 -- 172.21.15.156:0/2414349128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea34150050 msgr2=0x7fea340c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea399df700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea34150050 0x7fea340c5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea399df700 1 -- 172.21.15.156:0/2414349128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea20004850 msgr2=0x7fea34150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea399df700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea20004850 0x7fea34150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea399df700 1 -- 172.21.15.156:0/2414349128 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3406a270 con 0x7fea340c5970 2023-11-28T11:12:05.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea399df700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea340c5970 0x7fea340c5df0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fea340d0c90 tx=0x7fea2c00ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:05.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.106+0000 7fea29ffb700 1 -- 172.21.15.156:0/2414349128 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c003cd0 con 0x7fea340c5970 2023-11-28T11:12:05.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.107+0000 7fea29ffb700 1 -- 172.21.15.156:0/2414349128 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c003e50 con 0x7fea340c5970 2023-11-28T11:12:05.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.107+0000 7fea29ffb700 1 -- 172.21.15.156:0/2414349128 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c005310 con 0x7fea340c5970 2023-11-28T11:12:05.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.107+0000 7fea3b442700 1 -- 172.21.15.156:0/2414349128 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3415ccd0 con 0x7fea340c5970 2023-11-28T11:12:05.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.107+0000 7fea3b442700 1 -- 172.21.15.156:0/2414349128 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea340b5540 con 0x7fea340c5970 2023-11-28T11:12:05.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.108+0000 7fea3b442700 1 -- 172.21.15.156:0/2414349128 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3406a400 con 0x7fea340c5970 2023-11-28T11:12:05.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.108+0000 7fea29ffb700 1 -- 172.21.15.156:0/2414349128 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea2c005470 con 0x7fea340c5970 2023-11-28T11:12:05.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.108+0000 7fea29ffb700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea180682f0 0x7fea1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.109+0000 7fea391de700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea180682f0 0x7fea1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.109+0000 7fea29ffb700 1 -- 172.21.15.156:0/2414349128 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fea2c085f70 con 0x7fea340c5970 2023-11-28T11:12:05.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.109+0000 7fea391de700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea180682f0 0x7fea1806a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fea340af530 tx=0x7fea30001040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:05.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.113+0000 7fea29ffb700 1 -- 172.21.15.156:0/2414349128 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fea3406a400 con 0x7fea340c5970 2023-11-28T11:12:05.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.340+0000 7fea3b442700 1 -- 172.21.15.156:0/2414349128 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fea3406a400 con 0x7fea340c5970 2023-11-28T11:12:05.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.965+0000 7fea29ffb700 1 -- 172.21.15.156:0/2414349128 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7fea3406a400 con 0x7fea340c5970 2023-11-28T11:12:05.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.968+0000 7fea3b442700 1 -- 172.21.15.156:0/2414349128 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea180682f0 msgr2=0x7fea1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.968+0000 7fea3b442700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea180682f0 0x7fea1806a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fea340af530 tx=0x7fea30001040 comp rx=0 tx=0).stop 2023-11-28T11:12:05.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.968+0000 7fea3b442700 1 -- 172.21.15.156:0/2414349128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea340c5970 msgr2=0x7fea340c5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.968+0000 7fea3b442700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea340c5970 0x7fea340c5df0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fea340d0c90 tx=0x7fea2c00ff00 comp rx=0 tx=0).stop 2023-11-28T11:12:05.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.968+0000 7fea3b442700 1 -- 172.21.15.156:0/2414349128 shutdown_connections 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.968+0000 7fea3b442700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea340c5970 0x7fea340c5df0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.968+0000 7fea3b442700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea20004850 0x7fea34150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.968+0000 7fea3b442700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fea180682f0 0x7fea1806a7a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.968+0000 7fea3b442700 1 --2- 172.21.15.156:0/2414349128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea34150050 0x7fea340c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.968+0000 7fea3b442700 1 -- 172.21.15.156:0/2414349128 >> 172.21.15.156:0/2414349128 conn(0x7fea340d0000 msgr2=0x7fea3415e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.968+0000 7fea3b442700 1 -- 172.21.15.156:0/2414349128 shutdown_connections 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:05.968+0000 7fea3b442700 1 -- 172.21.15.156:0/2414349128 wait complete. 2023-11-28T11:12:05.984 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:12:06.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.113+0000 7f55ff4ef700 1 Processor -- start 2023-11-28T11:12:06.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.114+0000 7f55ff4ef700 1 -- start start 2023-11-28T11:12:06.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.114+0000 7f55ff4ef700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55f80d0140 0x7f55f80d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.114+0000 7f55ff4ef700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f815af70 con 0x7f55f815aba0 2023-11-28T11:12:06.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.114+0000 7f55ff4ef700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f80d0ae0 con 0x7f55f80b6ae0 2023-11-28T11:12:06.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.114+0000 7f55ff4ef700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f80cfec0 con 0x7f55f80d0140 2023-11-28T11:12:06.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.114+0000 7f55fda8c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55f80d0140 0x7f55f80d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:06.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.114+0000 7f55fd28b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f55f815aba0 0x7f55f80b3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51374/0 (socket says 172.21.15.156:51374) 2023-11-28T11:12:06.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.114+0000 7f55fda8c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55f80d0140 0x7f55f80d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36498/0 (socket says 172.21.15.156:36498) 2023-11-28T11:12:06.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.114+0000 7f55fd28b700 1 -- 172.21.15.156:0/1270568282 learned_addr learned my addr 172.21.15.156:0/1270568282 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:06.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.115+0000 7f55fda8c700 1 -- 172.21.15.156:0/1270568282 >> v1:172.21.15.186:6789/0 conn(0x7f55f80b6ae0 legacy=0x7f55f80cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:06.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.115+0000 7f55fda8c700 1 -- 172.21.15.156:0/1270568282 >> v1:172.21.15.156:6789/0 conn(0x7f55f815aba0 legacy=0x7f55f80b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.115+0000 7f55fda8c700 1 -- 172.21.15.156:0/1270568282 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f80cc870 con 0x7f55f80d0140 2023-11-28T11:12:06.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.115+0000 7f55f7fff700 1 -- 172.21.15.156:0/1270568282 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574462816 0 0) 0x7f55f815af70 con 0x7f55f815aba0 2023-11-28T11:12:06.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.115+0000 7f55fda8c700 1 --2- 172.21.15.156:0/1270568282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f80d0140 0x7f55f80d05a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009840 tx=0x7f55f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d733847491af5468 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:06.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.115+0000 7f55f7fff700 1 -- 172.21.15.156:0/1270568282 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0004230 con 0x7f55f80d0140 2023-11-28T11:12:06.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.115+0000 7f55f7fff700 1 -- 172.21.15.156:0/1270568282 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f0004390 con 0x7f55f80d0140 2023-11-28T11:12:06.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.115+0000 7f55f7fff700 1 -- 172.21.15.156:0/1270568282 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f00036a0 con 0x7f55f80d0140 2023-11-28T11:12:06.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.116+0000 7f55ff4ef700 1 -- 172.21.15.156:0/1270568282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f80d0140 msgr2=0x7f55f80d05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.116+0000 7f55ff4ef700 1 --2- 172.21.15.156:0/1270568282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f80d0140 0x7f55f80d05a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009840 tx=0x7f55f0009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:06.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.116+0000 7f55ff4ef700 1 -- 172.21.15.156:0/1270568282 shutdown_connections 2023-11-28T11:12:06.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.116+0000 7f55ff4ef700 1 --2- 172.21.15.156:0/1270568282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f80d0140 0x7f55f80d05a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.116+0000 7f55ff4ef700 1 -- 172.21.15.156:0/1270568282 >> 172.21.15.156:0/1270568282 conn(0x7f55f806df70 msgr2=0x7f55f806e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:06.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.116+0000 7f55ff4ef700 1 -- 172.21.15.156:0/1270568282 shutdown_connections 2023-11-28T11:12:06.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.116+0000 7f55ff4ef700 1 -- 172.21.15.156:0/1270568282 wait complete. 2023-11-28T11:12:06.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.117+0000 7f55ff4ef700 1 Processor -- start 2023-11-28T11:12:06.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.117+0000 7f55ff4ef700 1 -- start start 2023-11-28T11:12:06.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55ff4ef700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55f80d0140 0x7f55f80ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55ff4ef700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55f815aba0 0x7f55f806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55ff4ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f80b4d80 0x7f55f80b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55ff4ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f806c2e0 con 0x7f55f80b4d80 2023-11-28T11:12:06.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55ff4ef700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80ca820 con 0x7f55f80d0140 2023-11-28T11:12:06.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55ff4ef700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80b5740 con 0x7f55f815aba0 2023-11-28T11:12:06.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55fda8c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f80b4d80 0x7f55f80b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:06.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55fda8c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f80b4d80 0x7f55f80b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36504/0 (socket says 172.21.15.156:36504) 2023-11-28T11:12:06.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55fda8c700 1 -- 172.21.15.156:0/3297131762 learned_addr learned my addr 172.21.15.156:0/3297131762 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:06.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55fd28b700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55f80d0140 0x7f55f80ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:06.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55fca8a700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55f815aba0 0x7f55f806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:06.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55fda8c700 1 -- 172.21.15.156:0/3297131762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55f815aba0 msgr2=0x7f55f806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55fda8c700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55f815aba0 0x7f55f806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55fda8c700 1 -- 172.21.15.156:0/3297131762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55f80d0140 msgr2=0x7f55f80ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55fda8c700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55f80d0140 0x7f55f80ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55fda8c700 1 -- 172.21.15.156:0/3297131762 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f00094a0 con 0x7f55f80b4d80 2023-11-28T11:12:06.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55fda8c700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f80b4d80 0x7f55f80b5200 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f55f8067360 tx=0x7f55f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:06.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.118+0000 7f55f5ffb700 1 -- 172.21.15.156:0/3297131762 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0003d60 con 0x7f55f80b4d80 2023-11-28T11:12:06.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.119+0000 7f55f5ffb700 1 -- 172.21.15.156:0/3297131762 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f001e070 con 0x7f55f80b4d80 2023-11-28T11:12:06.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.119+0000 7f55ff4ef700 1 -- 172.21.15.156:0/3297131762 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f80ccd80 con 0x7f55f80b4d80 2023-11-28T11:12:06.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.119+0000 7f55f5ffb700 1 -- 172.21.15.156:0/3297131762 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0011870 con 0x7f55f80b4d80 2023-11-28T11:12:06.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.119+0000 7f55ff4ef700 1 -- 172.21.15.156:0/3297131762 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f80ccaa0 con 0x7f55f80b4d80 2023-11-28T11:12:06.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.120+0000 7f55ff4ef700 1 -- 172.21.15.156:0/3297131762 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f80cd060 con 0x7f55f80b4d80 2023-11-28T11:12:06.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.120+0000 7f55f5ffb700 1 -- 172.21.15.156:0/3297131762 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f55f0017020 con 0x7f55f80b4d80 2023-11-28T11:12:06.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.120+0000 7f55f5ffb700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f55e40682f0 0x7f55e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.120+0000 7f55f5ffb700 1 -- 172.21.15.156:0/3297131762 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f55f0087010 con 0x7f55f80b4d80 2023-11-28T11:12:06.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.121+0000 7f55fd28b700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f55e40682f0 0x7f55e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:06.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.121+0000 7f55fd28b700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f55e40682f0 0x7f55e406a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f55e8007c40 tx=0x7f55e8007b90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:06.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.125+0000 7f55f5ffb700 1 -- 172.21.15.156:0/3297131762 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f55f0053720 con 0x7f55f80b4d80 2023-11-28T11:12:06.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.309+0000 7f55ff4ef700 1 -- 172.21.15.156:0/3297131762 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f55f8068690 con 0x7f55f80b4d80 2023-11-28T11:12:06.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.966+0000 7f55f5ffb700 1 -- 172.21.15.156:0/3297131762 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 181+0+0 (secure 0 0 0) 0x7f55f0020070 con 0x7f55f80b4d80 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.971+0000 7f55ff4ef700 1 -- 172.21.15.156:0/3297131762 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f55e40682f0 msgr2=0x7f55e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.971+0000 7f55ff4ef700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f55e40682f0 0x7f55e406a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f55e8007c40 tx=0x7f55e8007b90 comp rx=0 tx=0).stop 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.971+0000 7f55ff4ef700 1 -- 172.21.15.156:0/3297131762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f80b4d80 msgr2=0x7f55f80b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.971+0000 7f55ff4ef700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f80b4d80 0x7f55f80b5200 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f55f8067360 tx=0x7f55f0004750 comp rx=0 tx=0).stop 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.972+0000 7f55ff4ef700 1 -- 172.21.15.156:0/3297131762 shutdown_connections 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.972+0000 7f55ff4ef700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f80b4d80 0x7f55f80b5200 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.972+0000 7f55ff4ef700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55f80d0140 0x7f55f80ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.972+0000 7f55ff4ef700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f55e40682f0 0x7f55e406a7a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.972+0000 7f55ff4ef700 1 --2- 172.21.15.156:0/3297131762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55f815aba0 0x7f55f806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.972+0000 7f55ff4ef700 1 -- 172.21.15.156:0/3297131762 >> 172.21.15.156:0/3297131762 conn(0x7f55f806df70 msgr2=0x7f55f8159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:06.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.972+0000 7f55ff4ef700 1 -- 172.21.15.156:0/3297131762 shutdown_connections 2023-11-28T11:12:06.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:06.972+0000 7f55ff4ef700 1 -- 172.21.15.156:0/3297131762 wait complete. 2023-11-28T11:12:06.974 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:12:06.985 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:12:07.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.110+0000 7f087f8e4700 1 Processor -- start 2023-11-28T11:12:07.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.110+0000 7f087f8e4700 1 -- start start 2023-11-28T11:12:07.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f087f8e4700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f087815aba0 0x7f08780b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f087f8e4700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087815af70 con 0x7f08780d0140 2023-11-28T11:12:07.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f087f8e4700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08780cc870 con 0x7f08780b6ae0 2023-11-28T11:12:07.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f087f8e4700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08780cfec0 con 0x7f087815aba0 2023-11-28T11:12:07.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f087de81700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f08780d0140 0x7f08780cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51388/0 (socket says 172.21.15.156:51388) 2023-11-28T11:12:07.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f087de81700 1 -- 172.21.15.156:0/1453408161 learned_addr learned my addr 172.21.15.156:0/1453408161 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:07.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f087d680700 1 --2- 172.21.15.156:0/1453408161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f087815aba0 0x7f08780b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540374278 0 0) 0x7f087815af70 con 0x7f08780d0140 2023-11-28T11:12:07.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08640035b0 con 0x7f08780d0140 2023-11-28T11:12:07.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3107761966 0 0) 0x7f08640035b0 con 0x7f08780d0140 2023-11-28T11:12:07.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f087815aba0 msgr2=0x7f08780b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f086ffff700 1 --2- 172.21.15.156:0/1453408161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f087815aba0 0x7f08780b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 >> v1:172.21.15.186:6789/0 conn(0x7f08780b6ae0 legacy=0x7f08780cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780b4290 con 0x7f08780d0140 2023-11-28T11:12:07.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f08700019f0 con 0x7f08780d0140 2023-11-28T11:12:07.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 >> v1:172.21.15.156:6789/0 conn(0x7f08780d0140 legacy=0x7f08780cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f086ffff700 1 --2- 172.21.15.156:0/1453408161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0864003a30 0x7f0864003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.111+0000 7f086ffff700 1 --2- 172.21.15.156:0/1453408161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08640043d0 0x7f0864004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f086ffff700 1 --2- 172.21.15.156:0/1453408161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f087815aba0 0x7f08640050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08780cfec0 con 0x7f08640043d0 2023-11-28T11:12:07.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086400ba10 con 0x7f0864003a30 2023-11-28T11:12:07.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086400bba0 con 0x7f087815aba0 2023-11-28T11:12:07.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f087ce7f700 1 --2- 172.21.15.156:0/1453408161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08640043d0 0x7f0864004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f087d680700 1 --2- 172.21.15.156:0/1453408161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0864003a30 0x7f0864003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f087de81700 1 --2- 172.21.15.156:0/1453408161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f087815aba0 0x7f08640050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f087ce7f700 1 -- 172.21.15.156:0/1453408161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f087815aba0 msgr2=0x7f08640050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f087ce7f700 1 --2- 172.21.15.156:0/1453408161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f087815aba0 0x7f08640050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f087ce7f700 1 -- 172.21.15.156:0/1453408161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0864003a30 msgr2=0x7f0864003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f087ce7f700 1 --2- 172.21.15.156:0/1453408161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0864003a30 0x7f0864003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f087ce7f700 1 -- 172.21.15.156:0/1453408161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08640035b0 con 0x7f08640043d0 2023-11-28T11:12:07.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f087ce7f700 1 --2- 172.21.15.156:0/1453408161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08640043d0 0x7f0864004890 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0874001980 tx=0x7f0864005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bd8acc0ed444dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874016070 con 0x7f08640043d0 2023-11-28T11:12:07.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0874006860 con 0x7f08640043d0 2023-11-28T11:12:07.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.112+0000 7f086ffff700 1 -- 172.21.15.156:0/1453408161 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874001420 con 0x7f08640043d0 2023-11-28T11:12:07.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.113+0000 7f087f8e4700 1 -- 172.21.15.156:0/1453408161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08640043d0 msgr2=0x7f0864004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.113+0000 7f087f8e4700 1 --2- 172.21.15.156:0/1453408161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08640043d0 0x7f0864004890 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0874001980 tx=0x7f0864005880 comp rx=0 tx=0).stop 2023-11-28T11:12:07.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.113+0000 7f087de81700 1 -- 172.21.15.156:0/1453408161 reap_dead start 2023-11-28T11:12:07.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.113+0000 7f087f8e4700 1 -- 172.21.15.156:0/1453408161 shutdown_connections 2023-11-28T11:12:07.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.113+0000 7f087f8e4700 1 -- 172.21.15.156:0/1453408161 >> 172.21.15.156:0/1453408161 conn(0x7f087806df70 msgr2=0x7f087806e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.113+0000 7f087f8e4700 1 -- 172.21.15.156:0/1453408161 shutdown_connections 2023-11-28T11:12:07.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.113+0000 7f087f8e4700 1 -- 172.21.15.156:0/1453408161 wait complete. 2023-11-28T11:12:07.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.114+0000 7f087f8e4700 1 Processor -- start 2023-11-28T11:12:07.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.114+0000 7f087f8e4700 1 -- start start 2023-11-28T11:12:07.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087f8e4700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08640043d0 0x7f087806bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087f8e4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0878154360 0x7f08780ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087f8e4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08780cecf0 0x7f08781f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087f8e4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087815c340 con 0x7f08780cecf0 2023-11-28T11:12:07.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087f8e4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08780cf160 con 0x7f0878154360 2023-11-28T11:12:07.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087f8e4700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087815beb0 con 0x7f08640043d0 2023-11-28T11:12:07.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087de81700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08780cecf0 0x7f08781f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087de81700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08780cecf0 0x7f08781f5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36524/0 (socket says 172.21.15.156:36524) 2023-11-28T11:12:07.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087de81700 1 -- 172.21.15.156:0/466524734 learned_addr learned my addr 172.21.15.156:0/466524734 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:07.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087d680700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08640043d0 0x7f087806bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087ce7f700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0878154360 0x7f08780ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087de81700 1 -- 172.21.15.156:0/466524734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08640043d0 msgr2=0x7f087806bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087de81700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08640043d0 0x7f087806bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087de81700 1 -- 172.21.15.156:0/466524734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0878154360 msgr2=0x7f08780ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087de81700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0878154360 0x7f08780ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087de81700 1 -- 172.21.15.156:0/466524734 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780ccaa0 con 0x7f08780cecf0 2023-11-28T11:12:07.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.115+0000 7f087de81700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08780cecf0 0x7f08781f5950 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f08780cc300 tx=0x7f087000a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.116+0000 7f086dffb700 1 -- 172.21.15.156:0/466524734 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000acf0 con 0x7f08780cecf0 2023-11-28T11:12:07.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.116+0000 7f086dffb700 1 -- 172.21.15.156:0/466524734 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000ae50 con 0x7f08780cecf0 2023-11-28T11:12:07.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.116+0000 7f087f8e4700 1 -- 172.21.15.156:0/466524734 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08780cfbf0 con 0x7f08780cecf0 2023-11-28T11:12:07.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.116+0000 7f086dffb700 1 -- 172.21.15.156:0/466524734 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870013630 con 0x7f08780cecf0 2023-11-28T11:12:07.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.116+0000 7f087f8e4700 1 -- 172.21.15.156:0/466524734 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08780ccd80 con 0x7f08780cecf0 2023-11-28T11:12:07.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.117+0000 7f087f8e4700 1 -- 172.21.15.156:0/466524734 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0878068690 con 0x7f08780cecf0 2023-11-28T11:12:07.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.117+0000 7f086dffb700 1 -- 172.21.15.156:0/466524734 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0870013790 con 0x7f08780cecf0 2023-11-28T11:12:07.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.117+0000 7f086dffb700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f085c068320 0x7f085c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.118+0000 7f086dffb700 1 -- 172.21.15.156:0/466524734 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5797+0+0 (secure 0 0 0) 0x7f08700863e0 con 0x7f08780cecf0 2023-11-28T11:12:07.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.118+0000 7f087d680700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f085c068320 0x7f085c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.118+0000 7f087d680700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f085c068320 0x7f085c06a7d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0868009be0 tx=0x7f0868009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.122+0000 7f086dffb700 1 -- 172.21.15.156:0/466524734 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0870052ce0 con 0x7f08780cecf0 2023-11-28T11:12:07.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.305+0000 7f087f8e4700 1 -- 172.21.15.156:0/466524734 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f087815bff0 con 0x7f08780cecf0 2023-11-28T11:12:07.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.975+0000 7f086dffb700 1 -- 172.21.15.156:0/466524734 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0870056300 con 0x7f08780cecf0 2023-11-28T11:12:07.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.980+0000 7f087f8e4700 1 -- 172.21.15.156:0/466524734 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f085c068320 msgr2=0x7f085c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.980+0000 7f087f8e4700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f085c068320 0x7f085c06a7d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0868009be0 tx=0x7f0868009210 comp rx=0 tx=0).stop 2023-11-28T11:12:07.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.980+0000 7f087f8e4700 1 -- 172.21.15.156:0/466524734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08780cecf0 msgr2=0x7f08781f5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.980+0000 7f087f8e4700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08780cecf0 0x7f08781f5950 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f08780cc300 tx=0x7f087000a5b0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.980+0000 7f087f8e4700 1 -- 172.21.15.156:0/466524734 shutdown_connections 2023-11-28T11:12:07.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.980+0000 7f087f8e4700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08780cecf0 0x7f08781f5950 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.980+0000 7f087f8e4700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f085c068320 0x7f085c06a7d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.980+0000 7f087f8e4700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08640043d0 0x7f087806bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.980+0000 7f087f8e4700 1 --2- 172.21.15.156:0/466524734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0878154360 0x7f08780ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.980+0000 7f087f8e4700 1 -- 172.21.15.156:0/466524734 >> 172.21.15.156:0/466524734 conn(0x7f087806df70 msgr2=0x7f08781585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.980+0000 7f087f8e4700 1 -- 172.21.15.156:0/466524734 shutdown_connections 2023-11-28T11:12:07.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:07.980+0000 7f087f8e4700 1 -- 172.21.15.156:0/466524734 wait complete. 2023-11-28T11:12:07.983 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' removed 2023-11-28T11:12:07.996 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:12:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.132+0000 7f3fe0b2c700 1 Processor -- start 2023-11-28T11:12:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.132+0000 7f3fe0b2c700 1 -- start start 2023-11-28T11:12:08.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.133+0000 7f3fe0b2c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3fdc071750 0x7f3fdc15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.133+0000 7f3fe0b2c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fdc155240 con 0x7f3fdc0cdbd0 2023-11-28T11:12:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.133+0000 7f3fe0b2c700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fdc15b100 con 0x7f3fdc154e70 2023-11-28T11:12:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.133+0000 7f3fe0b2c700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fdc0ce050 con 0x7f3fdc071750 2023-11-28T11:12:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.133+0000 7f3fd9d9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3fdc0cdbd0 0x7f3fdc071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51406/0 (socket says 172.21.15.156:51406) 2023-11-28T11:12:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.133+0000 7f3fd9d9b700 1 -- 172.21.15.156:0/3361269760 learned_addr learned my addr 172.21.15.156:0/3361269760 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.133+0000 7f3fdad9d700 1 --2- 172.21.15.156:0/3361269760 >> v2:172.21.15.186:3300/0 conn(0x7f3fdc071750 0x7f3fdc15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.133+0000 7f3fd959a700 1 -- 172.21.15.156:0/3361269760 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587722118 0 0) 0x7f3fdc155240 con 0x7f3fdc0cdbd0 2023-11-28T11:12:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.133+0000 7f3fd959a700 1 -- 172.21.15.156:0/3361269760 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc40035b0 con 0x7f3fdc0cdbd0 2023-11-28T11:12:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.134+0000 7f3fdad9d700 1 -- 172.21.15.156:0/3361269760 >> v1:172.21.15.186:6789/0 conn(0x7f3fdc154e70 legacy=0x7f3fdc0cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.134+0000 7f3fd959a700 1 -- 172.21.15.156:0/3361269760 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1840874542 0 0) 0x7f3fc40035b0 con 0x7f3fdc0cdbd0 2023-11-28T11:12:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.134+0000 7f3fdad9d700 1 -- 172.21.15.156:0/3361269760 >> v1:172.21.15.156:6789/0 conn(0x7f3fdc0cdbd0 legacy=0x7f3fdc071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.134+0000 7f3fdad9d700 1 -- 172.21.15.156:0/3361269760 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc15b3e0 con 0x7f3fdc071750 2023-11-28T11:12:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.134+0000 7f3fdad9d700 1 --2- 172.21.15.156:0/3361269760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fdc071750 0x7f3fdc15e680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009d90 tx=0x7f3fd400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=756004e3904394e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.134+0000 7f3fd959a700 1 -- 172.21.15.156:0/3361269760 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400e070 con 0x7f3fdc071750 2023-11-28T11:12:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.134+0000 7f3fd959a700 1 -- 172.21.15.156:0/3361269760 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd4003a20 con 0x7f3fdc071750 2023-11-28T11:12:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.134+0000 7f3fd959a700 1 -- 172.21.15.156:0/3361269760 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc001670 con 0x7f3fdc071750 2023-11-28T11:12:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.134+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/3361269760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fdc071750 msgr2=0x7f3fdc15e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.134+0000 7f3fe0b2c700 1 --2- 172.21.15.156:0/3361269760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fdc071750 0x7f3fdc15e680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009d90 tx=0x7f3fd400b330 comp rx=0 tx=0).stop 2023-11-28T11:12:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.135+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/3361269760 shutdown_connections 2023-11-28T11:12:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.135+0000 7f3fe0b2c700 1 --2- 172.21.15.156:0/3361269760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fdc071750 0x7f3fdc15e680 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.135+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/3361269760 >> 172.21.15.156:0/3361269760 conn(0x7f3fdc0d0000 msgr2=0x7f3fdc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.135+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/3361269760 shutdown_connections 2023-11-28T11:12:08.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.135+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/3361269760 wait complete. 2023-11-28T11:12:08.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.136+0000 7f3fe0b2c700 1 Processor -- start 2023-11-28T11:12:08.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.136+0000 7f3fe0b2c700 1 -- start start 2023-11-28T11:12:08.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fe0b2c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3fdc071750 0x7f3fdc0b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fe0b2c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fdc0cdbd0 0x7f3fdc15a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fe0b2c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc154e70 0x7f3fdc15a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fe0b2c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc158980 con 0x7f3fdc154e70 2023-11-28T11:12:08.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fe0b2c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc15ab80 con 0x7f3fdc0cdbd0 2023-11-28T11:12:08.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fe0b2c700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc158bd0 con 0x7f3fdc071750 2023-11-28T11:12:08.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fdad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc154e70 0x7f3fdc15a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fdad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc154e70 0x7f3fdc15a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36538/0 (socket says 172.21.15.156:36538) 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fdad9d700 1 -- 172.21.15.156:0/1406907124 learned_addr learned my addr 172.21.15.156:0/1406907124 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fd9d9b700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fdc0cdbd0 0x7f3fdc15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fdad9d700 1 -- 172.21.15.156:0/1406907124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3fdc071750 msgr2=0x7f3fdc0b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fdad9d700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3fdc071750 0x7f3fdc0b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fdad9d700 1 -- 172.21.15.156:0/1406907124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fdc0cdbd0 msgr2=0x7f3fdc15a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fdad9d700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fdc0cdbd0 0x7f3fdc15a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fdad9d700 1 -- 172.21.15.156:0/1406907124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd4009a70 con 0x7f3fdc154e70 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fdad9d700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc154e70 0x7f3fdc15a640 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc073bb0 tx=0x7f3fd4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fcaffd700 1 -- 172.21.15.156:0/1406907124 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4004640 con 0x7f3fdc154e70 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fcaffd700 1 -- 172.21.15.156:0/1406907124 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd40047a0 con 0x7f3fdc154e70 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/1406907124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc15b9a0 con 0x7f3fdc154e70 2023-11-28T11:12:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fcaffd700 1 -- 172.21.15.156:0/1406907124 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd401d670 con 0x7f3fdc154e70 2023-11-28T11:12:08.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.137+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/1406907124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc15b6c0 con 0x7f3fdc154e70 2023-11-28T11:12:08.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.138+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/1406907124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc1596b0 con 0x7f3fdc154e70 2023-11-28T11:12:08.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.139+0000 7f3fcaffd700 1 -- 172.21.15.156:0/1406907124 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3fd4018070 con 0x7f3fdc154e70 2023-11-28T11:12:08.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.139+0000 7f3fcaffd700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3fbc0682f0 0x7f3fbc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.139+0000 7f3fda59c700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3fbc0682f0 0x7f3fbc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.139+0000 7f3fcaffd700 1 -- 172.21.15.156:0/1406907124 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3fd4087670 con 0x7f3fdc154e70 2023-11-28T11:12:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.140+0000 7f3fda59c700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3fbc0682f0 0x7f3fbc06a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0000f60 tx=0x7f3fd0007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.143+0000 7f3fcaffd700 1 -- 172.21.15.156:0/1406907124 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3fdc1596b0 con 0x7f3fdc154e70 2023-11-28T11:12:08.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.326+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/1406907124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3fdc1596b0 con 0x7f3fdc154e70 2023-11-28T11:12:08.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.330+0000 7f3fcaffd700 1 -- 172.21.15.156:0/1406907124 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3fdc1596b0 con 0x7f3fdc154e70 2023-11-28T11:12:08.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.332+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/1406907124 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3fbc0682f0 msgr2=0x7f3fbc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.332+0000 7f3fe0b2c700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3fbc0682f0 0x7f3fbc06a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0000f60 tx=0x7f3fd0007a40 comp rx=0 tx=0).stop 2023-11-28T11:12:08.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.332+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/1406907124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc154e70 msgr2=0x7f3fdc15a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.332+0000 7f3fe0b2c700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc154e70 0x7f3fdc15a640 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc073bb0 tx=0x7f3fd4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.332+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/1406907124 shutdown_connections 2023-11-28T11:12:08.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.332+0000 7f3fe0b2c700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc154e70 0x7f3fdc15a640 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.332+0000 7f3fe0b2c700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3fbc0682f0 0x7f3fbc06a7a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.332+0000 7f3fe0b2c700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3fdc071750 0x7f3fdc0b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.332+0000 7f3fe0b2c700 1 --2- 172.21.15.156:0/1406907124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fdc0cdbd0 0x7f3fdc15a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.332+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/1406907124 >> 172.21.15.156:0/1406907124 conn(0x7f3fdc0d0000 msgr2=0x7f3fdc0acdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.332+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/1406907124 shutdown_connections 2023-11-28T11:12:08.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.332+0000 7f3fe0b2c700 1 -- 172.21.15.156:0/1406907124 wait complete. 2023-11-28T11:12:08.345 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T11:12:08.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.464+0000 7fceb2468700 1 Processor -- start 2023-11-28T11:12:08.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.464+0000 7fceb2468700 1 -- start start 2023-11-28T11:12:08.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb2468700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fceac0cd8c0 0x7fceac140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb2468700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fceac0cdc90 0x7fceac0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb2468700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fceac0b2e70 0x7fceac06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb2468700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceac13ff00 con 0x7fceac0b2e70 2023-11-28T11:12:08.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb2468700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceac1404c0 con 0x7fceac0cdc90 2023-11-28T11:12:08.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb2468700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceac15aa80 con 0x7fceac0cd8c0 2023-11-28T11:12:08.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb0a05700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fceac0b2e70 0x7fceac06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb0a05700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fceac0b2e70 0x7fceac06f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36546/0 (socket says 172.21.15.156:36546) 2023-11-28T11:12:08.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb0a05700 1 -- 172.21.15.156:0/2404995213 learned_addr learned my addr 172.21.15.156:0/2404995213 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:08.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceabfff700 1 --2- 172.21.15.156:0/2404995213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fceac0cd8c0 0x7fceac140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceab7fe700 1 --2- 172.21.15.156:0/2404995213 >> v2:172.21.15.186:3300/0 conn(0x7fceac0cdc90 0x7fceac0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb0a05700 1 -- 172.21.15.156:0/2404995213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fceac0cd8c0 msgr2=0x7fceac140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb0a05700 1 --2- 172.21.15.156:0/2404995213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fceac0cd8c0 0x7fceac140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb0a05700 1 -- 172.21.15.156:0/2404995213 >> v2:172.21.15.186:3300/0 conn(0x7fceac0cdc90 msgr2=0x7fceac0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb0a05700 1 --2- 172.21.15.156:0/2404995213 >> v2:172.21.15.186:3300/0 conn(0x7fceac0cdc90 0x7fceac0b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb0a05700 1 -- 172.21.15.156:0/2404995213 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac0b3310 con 0x7fceac0b2e70 2023-11-28T11:12:08.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceabfff700 1 --2- 172.21.15.156:0/2404995213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fceac0cd8c0 0x7fceac140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:08.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.465+0000 7fceb0a05700 1 --2- 172.21.15.156:0/2404995213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceac0b2e70 0x7fceac06f850 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fceac067c60 tx=0x7fcea400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f5758f681ec0fc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.466+0000 7fceaaffd700 1 -- 172.21.15.156:0/2404995213 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea400e070 con 0x7fceac0b2e70 2023-11-28T11:12:08.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.466+0000 7fceaaffd700 1 -- 172.21.15.156:0/2404995213 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea400b9c0 con 0x7fceac0b2e70 2023-11-28T11:12:08.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.466+0000 7fceaaffd700 1 -- 172.21.15.156:0/2404995213 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea400bd70 con 0x7fceac0b2e70 2023-11-28T11:12:08.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.466+0000 7fceb2468700 1 -- 172.21.15.156:0/2404995213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceac0b2e70 msgr2=0x7fceac06f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.466+0000 7fceb2468700 1 --2- 172.21.15.156:0/2404995213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceac0b2e70 0x7fceac06f850 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fceac067c60 tx=0x7fcea400b330 comp rx=0 tx=0).stop 2023-11-28T11:12:08.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.466+0000 7fceb2468700 1 -- 172.21.15.156:0/2404995213 shutdown_connections 2023-11-28T11:12:08.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.466+0000 7fceb2468700 1 --2- 172.21.15.156:0/2404995213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceac0b2e70 0x7fceac06f850 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.466+0000 7fceb2468700 1 --2- 172.21.15.156:0/2404995213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fceac0cd8c0 0x7fceac140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.466+0000 7fceb2468700 1 --2- 172.21.15.156:0/2404995213 >> v2:172.21.15.186:3300/0 conn(0x7fceac0cdc90 0x7fceac0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.466+0000 7fceb2468700 1 -- 172.21.15.156:0/2404995213 >> 172.21.15.156:0/2404995213 conn(0x7fceac06aed0 msgr2=0x7fceac06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.466+0000 7fceb2468700 1 -- 172.21.15.156:0/2404995213 shutdown_connections 2023-11-28T11:12:08.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.466+0000 7fceb2468700 1 -- 172.21.15.156:0/2404995213 wait complete. 2023-11-28T11:12:08.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.467+0000 7fceb2468700 1 Processor -- start 2023-11-28T11:12:08.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.467+0000 7fceb2468700 1 -- start start 2023-11-28T11:12:08.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.468+0000 7fceb2468700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fceac0b2e70 0x7fceac0b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.468+0000 7fceb2468700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fceac0cd8c0 0x7fceac156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.468+0000 7fceb2468700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceac0cdc90 0x7fceac0c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.468+0000 7fceb2468700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac140ff0 con 0x7fceac0cdc90 2023-11-28T11:12:08.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.468+0000 7fceb2468700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac141890 con 0x7fceac0b2e70 2023-11-28T11:12:08.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.468+0000 7fceb2468700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceac0b5f90 con 0x7fceac0cd8c0 2023-11-28T11:12:08.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.468+0000 7fceb0a05700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceac0cdc90 0x7fceac0c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.468+0000 7fceb0a05700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceac0cdc90 0x7fceac0c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36552/0 (socket says 172.21.15.156:36552) 2023-11-28T11:12:08.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.468+0000 7fceb0a05700 1 -- 172.21.15.156:0/516092599 learned_addr learned my addr 172.21.15.156:0/516092599 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:08.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.468+0000 7fceab7fe700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fceac0cd8c0 0x7fceac156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.468+0000 7fceabfff700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fceac0b2e70 0x7fceac0b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.469+0000 7fceb0a05700 1 -- 172.21.15.156:0/516092599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fceac0cd8c0 msgr2=0x7fceac156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.469+0000 7fceb0a05700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fceac0cd8c0 0x7fceac156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.469+0000 7fceb0a05700 1 -- 172.21.15.156:0/516092599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fceac0b2e70 msgr2=0x7fceac0b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.469+0000 7fceb0a05700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fceac0b2e70 0x7fceac0b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.469+0000 7fceb0a05700 1 -- 172.21.15.156:0/516092599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea4009b30 con 0x7fceac0cdc90 2023-11-28T11:12:08.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.469+0000 7fceabfff700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fceac0b2e70 0x7fceac0b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:08.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.469+0000 7fceb0a05700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceac0cdc90 0x7fceac0c69f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fcea400b370 tx=0x7fcea4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.469+0000 7fcea8ff9700 1 -- 172.21.15.156:0/516092599 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea4004310 con 0x7fceac0cdc90 2023-11-28T11:12:08.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.469+0000 7fcea8ff9700 1 -- 172.21.15.156:0/516092599 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea4004470 con 0x7fceac0cdc90 2023-11-28T11:12:08.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.469+0000 7fceb2468700 1 -- 172.21.15.156:0/516092599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceac140150 con 0x7fceac0cdc90 2023-11-28T11:12:08.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.469+0000 7fcea8ff9700 1 -- 172.21.15.156:0/516092599 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea4004820 con 0x7fceac0cdc90 2023-11-28T11:12:08.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.469+0000 7fceb2468700 1 -- 172.21.15.156:0/516092599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceac0b35f0 con 0x7fceac0cdc90 2023-11-28T11:12:08.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.470+0000 7fceb2468700 1 -- 172.21.15.156:0/516092599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceac0ce100 con 0x7fceac0cdc90 2023-11-28T11:12:08.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.470+0000 7fcea8ff9700 1 -- 172.21.15.156:0/516092599 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcea401b430 con 0x7fceac0cdc90 2023-11-28T11:12:08.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.471+0000 7fcea8ff9700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fce980682f0 0x7fce9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.471+0000 7fcea8ff9700 1 -- 172.21.15.156:0/516092599 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcea4086cd0 con 0x7fceac0cdc90 2023-11-28T11:12:08.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.471+0000 7fceabfff700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fce980682f0 0x7fce9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.471+0000 7fceabfff700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fce980682f0 0x7fce9806a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcea000aaf0 tx=0x7fcea0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.475+0000 7fcea8ff9700 1 -- 172.21.15.156:0/516092599 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcea40537d0 con 0x7fceac0cdc90 2023-11-28T11:12:08.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.731+0000 7fceb2468700 1 -- 172.21.15.156:0/516092599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fceac068690 con 0x7fceac0cdc90 2023-11-28T11:12:08.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.976+0000 7fcea8ff9700 1 -- 172.21.15.156:0/516092599 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v199) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcea401f0f0 con 0x7fceac0cdc90 2023-11-28T11:12:08.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.979+0000 7fceb2468700 1 -- 172.21.15.156:0/516092599 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fce980682f0 msgr2=0x7fce9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.979+0000 7fceb2468700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fce980682f0 0x7fce9806a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcea000aaf0 tx=0x7fcea0008040 comp rx=0 tx=0).stop 2023-11-28T11:12:08.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.979+0000 7fceb2468700 1 -- 172.21.15.156:0/516092599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceac0cdc90 msgr2=0x7fceac0c69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.979+0000 7fceb2468700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceac0cdc90 0x7fceac0c69f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fcea400b370 tx=0x7fcea4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.979+0000 7fceb2468700 1 -- 172.21.15.156:0/516092599 shutdown_connections 2023-11-28T11:12:08.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.979+0000 7fceb2468700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceac0cdc90 0x7fceac0c69f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.979+0000 7fceb2468700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fceac0b2e70 0x7fceac0b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.979+0000 7fceb2468700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fce980682f0 0x7fce9806a7a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.979+0000 7fceb2468700 1 --2- 172.21.15.156:0/516092599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fceac0cd8c0 0x7fceac156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.979+0000 7fceb2468700 1 -- 172.21.15.156:0/516092599 >> 172.21.15.156:0/516092599 conn(0x7fceac06aed0 msgr2=0x7fceac13e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.979+0000 7fceb2468700 1 -- 172.21.15.156:0/516092599 shutdown_connections 2023-11-28T11:12:08.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:08.979+0000 7fceb2468700 1 -- 172.21.15.156:0/516092599 wait complete. 2023-11-28T11:12:08.993 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ERROR 2023-11-28T11:12:08.993 INFO:tasks.cephfs_test_runner: 2023-11-28T11:12:08.993 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T11:12:08.994 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:12:08.994 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T11:12:08.994 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:12:08.994 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T11:12:08.994 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T11:12:08.994 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=20) 2023-11-28T11:12:08.994 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T11:12:08.994 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T11:12:08.994 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 20s and 0 retries 2023-11-28T11:12:08.994 INFO:tasks.cephfs_test_runner: 2023-11-28T11:12:08.994 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:12:08.994 INFO:tasks.cephfs_test_runner:Ran 3 tests in 199.485s 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner: 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner: 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=20) 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T11:12:08.995 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 20s and 0 retries 2023-11-28T11:12:08.996 INFO:tasks.cephfs_test_runner: 2023-11-28T11:12:08.996 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:12:09.086 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=66a04596572b4ca59ca9f6957153d158 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:12:09.087 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-28T11:12:09.097 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-28T11:12:09.142 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-28T11:12:09.142 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:12:09.142 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:09.172 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2550 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:12:09.173 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T11:12:09.173 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:09.230 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2550 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:12:09.231 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:12:09.231 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:09.251 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:12:09.252 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T11:12:09.252 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:12:09.311 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:12:09.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:09.312 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T11:12:09.330 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:12:09.330 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:12:09.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:09.333 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:12:09.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:09.409 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:12:09.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:09.493 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-11-28T11:12:09.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:09.654 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T11:12:09.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:09.671 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-11-28T11:12:09.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:12:09.742 INFO:teuthology.orchestra.run.smithi156.stderr:Cannot find device "ceph-brx" 2023-11-28T11:12:09.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:09.743 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-11-28T11:12:09.763 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-28T11:12:09.764 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T11:12:09.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:09.764 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T11:12:09.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:09.850 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:12:09.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:09.931 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2023-11-28T11:12:10.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:10.026 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-11-28T11:12:10.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:10.044 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2023-11-28T11:12:10.114 INFO:teuthology.orchestra.run.smithi186.stderr:Cannot find device "ceph-brx" 2023-11-28T11:12:10.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:12:10.116 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T11:12:10.128 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T11:12:10.128 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:10.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.250+0000 7f92c1fc2700 1 Processor -- start 2023-11-28T11:12:10.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.251+0000 7f92c1fc2700 1 -- start start 2023-11-28T11:12:10.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.251+0000 7f92c1fc2700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92bc159490 0x7f92bc0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.251+0000 7f92c1fc2700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f92bc159860 0x7f92bc0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.251+0000 7f92c1fc2700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92bc0ccc00 con 0x7f92bc0cdc80 2023-11-28T11:12:10.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.251+0000 7f92c1fc2700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92bc0ccec0 con 0x7f92bc159860 2023-11-28T11:12:10.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.251+0000 7f92c1fc2700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0cd1a0 con 0x7f92bc159490 2023-11-28T11:12:10.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92bb7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92bc159490 0x7f92bc0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92baffd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f92bc159860 0x7f92bc0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92baffd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f92bc159860 0x7f92bc0b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:47572/0 (socket says 172.21.15.156:47572) 2023-11-28T11:12:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92bb7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92bc159490 0x7f92bc0b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:40420/0 (socket says 172.21.15.156:40420) 2023-11-28T11:12:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92baffd700 1 -- 172.21.15.156:0/2819668248 learned_addr learned my addr 172.21.15.156:0/2819668248 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92baffd700 1 -- 172.21.15.156:0/2819668248 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92bc159490 msgr2=0x7f92bc0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92baffd700 1 --2- 172.21.15.156:0/2819668248 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92bc159490 0x7f92bc0b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92baffd700 1 -- 172.21.15.156:0/2819668248 >> v1:172.21.15.186:6789/0 conn(0x7f92bc0cdc80 legacy=0x7f92bc0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92baffd700 1 -- 172.21.15.156:0/2819668248 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0cd480 con 0x7f92bc159860 2023-11-28T11:12:10.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92bb7fe700 1 --2- 172.21.15.156:0/2819668248 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92bc159490 0x7f92bc0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:10.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92baffd700 1 --2- 172.21.15.156:0/2819668248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92bc159860 0x7f92bc0b0bb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f92a40098f0 tx=0x7f92a4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccbb920529f426f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:10.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92ba7fc700 1 -- 172.21.15.156:0/2819668248 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4011830 con 0x7f92bc159860 2023-11-28T11:12:10.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92ba7fc700 1 -- 172.21.15.156:0/2819668248 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a4011990 con 0x7f92bc159860 2023-11-28T11:12:10.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.252+0000 7f92ba7fc700 1 -- 172.21.15.156:0/2819668248 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4011d20 con 0x7f92bc159860 2023-11-28T11:12:10.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.253+0000 7f92c1fc2700 1 -- 172.21.15.156:0/2819668248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92bc159860 msgr2=0x7f92bc0b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.253+0000 7f92c1fc2700 1 --2- 172.21.15.156:0/2819668248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92bc159860 0x7f92bc0b0bb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f92a40098f0 tx=0x7f92a4009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:10.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.253+0000 7f92c1fc2700 1 -- 172.21.15.156:0/2819668248 shutdown_connections 2023-11-28T11:12:10.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.253+0000 7f92c1fc2700 1 --2- 172.21.15.156:0/2819668248 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92bc159490 0x7f92bc0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.253+0000 7f92c1fc2700 1 --2- 172.21.15.156:0/2819668248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92bc159860 0x7f92bc0b0bb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.253+0000 7f92c1fc2700 1 -- 172.21.15.156:0/2819668248 >> 172.21.15.156:0/2819668248 conn(0x7f92bc0cb670 msgr2=0x7f92bc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:10.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.253+0000 7f92c1fc2700 1 -- 172.21.15.156:0/2819668248 shutdown_connections 2023-11-28T11:12:10.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.253+0000 7f92c1fc2700 1 -- 172.21.15.156:0/2819668248 wait complete. 2023-11-28T11:12:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.254+0000 7f92c1fc2700 1 Processor -- start 2023-11-28T11:12:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.255+0000 7f92c1fc2700 1 -- start start 2023-11-28T11:12:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92c1fc2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92bc0cdc80 0x7f92bc15c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92c1fc2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92bc159490 0x7f92bc0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92c1fc2700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92bc0b1a20 0x7f92bc070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92c1fc2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0b1f20 con 0x7f92bc159490 2023-11-28T11:12:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92c1fc2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0b2200 con 0x7f92bc0cdc80 2023-11-28T11:12:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92c1fc2700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0b24e0 con 0x7f92bc0b1a20 2023-11-28T11:12:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92baffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92bc159490 0x7f92bc0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92baffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92bc159490 0x7f92bc0b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36564/0 (socket says 172.21.15.156:36564) 2023-11-28T11:12:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92baffd700 1 -- 172.21.15.156:0/3653914037 learned_addr learned my addr 172.21.15.156:0/3653914037 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:10.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92bbfff700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92bc0b1a20 0x7f92bc070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92bb7fe700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92bc0cdc80 0x7f92bc15c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92baffd700 1 -- 172.21.15.156:0/3653914037 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92bc0b1a20 msgr2=0x7f92bc070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92baffd700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92bc0b1a20 0x7f92bc070b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92baffd700 1 -- 172.21.15.156:0/3653914037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92bc0cdc80 msgr2=0x7f92bc15c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92baffd700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92bc0cdc80 0x7f92bc15c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92baffd700 1 -- 172.21.15.156:0/3653914037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac00b740 con 0x7f92bc159490 2023-11-28T11:12:10.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92baffd700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92bc159490 0x7f92bc0b0080 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009c40 tx=0x7f92a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:10.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92b3fff700 1 -- 172.21.15.156:0/3653914037 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4004510 con 0x7f92bc159490 2023-11-28T11:12:10.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92b3fff700 1 -- 172.21.15.156:0/3653914037 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a401e070 con 0x7f92bc159490 2023-11-28T11:12:10.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92c1fc2700 1 -- 172.21.15.156:0/3653914037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a4009550 con 0x7f92bc159490 2023-11-28T11:12:10.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92b3fff700 1 -- 172.21.15.156:0/3653914037 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a40038e0 con 0x7f92bc159490 2023-11-28T11:12:10.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.256+0000 7f92c1fc2700 1 -- 172.21.15.156:0/3653914037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc0c9bd0 con 0x7f92bc159490 2023-11-28T11:12:10.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.257+0000 7f92c1fc2700 1 -- 172.21.15.156:0/3653914037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92bc0cd760 con 0x7f92bc159490 2023-11-28T11:12:10.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.258+0000 7f92b3fff700 1 -- 172.21.15.156:0/3653914037 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f92a400f050 con 0x7f92bc159490 2023-11-28T11:12:10.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.258+0000 7f92b3fff700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f929c0682f0 0x7f929c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.258+0000 7f92b3fff700 1 -- 172.21.15.156:0/3653914037 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f92a4086860 con 0x7f92bc159490 2023-11-28T11:12:10.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.258+0000 7f92bb7fe700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f929c0682f0 0x7f929c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.258+0000 7f92bb7fe700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f929c0682f0 0x7f929c06a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0114a0 tx=0x7f92ac00b420 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:10.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.262+0000 7f92b3fff700 1 -- 172.21.15.156:0/3653914037 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f92a4053340 con 0x7f92bc159490 2023-11-28T11:12:10.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.438+0000 7f92c1fc2700 1 -- 172.21.15.156:0/3653914037 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92bc0c9d60 con 0x7f929c0682f0 2023-11-28T11:12:10.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.441+0000 7f92b3fff700 1 -- 172.21.15.156:0/3653914037 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72794 (secure 0 0 0) 0x7f92bc0c9d60 con 0x7f929c0682f0 2023-11-28T11:12:10.443 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:10.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.443+0000 7f92c1fc2700 1 -- 172.21.15.156:0/3653914037 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f929c0682f0 msgr2=0x7f929c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.443+0000 7f92c1fc2700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f929c0682f0 0x7f929c06a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0114a0 tx=0x7f92ac00b420 comp rx=0 tx=0).stop 2023-11-28T11:12:10.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.443+0000 7f92c1fc2700 1 -- 172.21.15.156:0/3653914037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92bc159490 msgr2=0x7f92bc0b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.443+0000 7f92c1fc2700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92bc159490 0x7f92bc0b0080 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009c40 tx=0x7f92a4004750 comp rx=0 tx=0).stop 2023-11-28T11:12:10.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.443+0000 7f92c1fc2700 1 -- 172.21.15.156:0/3653914037 shutdown_connections 2023-11-28T11:12:10.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.443+0000 7f92c1fc2700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f929c0682f0 0x7f929c06a7a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.443+0000 7f92c1fc2700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92bc0b1a20 0x7f92bc070b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.443+0000 7f92c1fc2700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92bc0cdc80 0x7f92bc15c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.443+0000 7f92c1fc2700 1 --2- 172.21.15.156:0/3653914037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92bc159490 0x7f92bc0b0080 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.443+0000 7f92c1fc2700 1 -- 172.21.15.156:0/3653914037 >> 172.21.15.156:0/3653914037 conn(0x7f92bc0cb670 msgr2=0x7f92bc0c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:10.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.443+0000 7f92c1fc2700 1 -- 172.21.15.156:0/3653914037 shutdown_connections 2023-11-28T11:12:10.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.443+0000 7f92c1fc2700 1 -- 172.21.15.156:0/3653914037 wait complete. 2023-11-28T11:12:10.447 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:12:10.457 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2023-11-28T11:12:08.578106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":88280,"kb_used_data":7448,"kb_used_omap":0,"kb_used_meta":80768,"kb_avail":754886440,"statfs":{"total":773094113280,"available":773003714560,"internally_reserved":0,"allocated":7626752,"data_stored":4034660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":82706432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12570,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-75,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-70,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001217"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.799800+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:11:02.799800+0000","last_peered":"2023-11-28T11:11:02.799800+0000","last_clean":"2023-11-28T11:11:02.799800+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:11:02.799800+0000","last_undegraded":"2023-11-28T11:11:02.799800+0000","last_fullsized":"2023-11-28T11:11:02.799800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.795953+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:11:02.795953+0000","last_peered":"2023-11-28T11:11:02.795953+0000","last_clean":"2023-11-28T11:11:02.795953+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:11:02.795953+0000","last_undegraded":"2023-11-28T11:11:02.795953+0000","last_fullsized":"2023-11-28T11:11:02.795953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.797281+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:11:02.797281+0000","last_peered":"2023-11-28T11:11:02.797281+0000","last_clean":"2023-11-28T11:11:02.797281+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:11:02.797281+0000","last_undegraded":"2023-11-28T11:11:02.797281+0000","last_fullsized":"2023-11-28T11:11:02.797281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.796416+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:11:02.796416+0000","last_peered":"2023-11-28T11:11:02.796416+0000","last_clean":"2023-11-28T11:11:02.796416+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:11:02.796416+0000","last_undegraded":"2023-11-28T11:11:02.796416+0000","last_fullsized":"2023-11-28T11:11:02.796416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":207,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.800135+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:11:02.800135+0000","last_peered":"2023-11-28T11:11:02.800135+0000","last_clean":"2023-11-28T11:11:02.800135+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:11:02.800135+0000","last_undegraded":"2023-11-28T11:11:02.800135+0000","last_fullsized":"2023-11-28T11:11:02.800135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.800320+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:11:02.800320+0000","last_peered":"2023-11-28T11:11:02.800320+0000","last_clean":"2023-11-28T11:11:02.800320+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:11:02.800320+0000","last_undegraded":"2023-11-28T11:11:02.800320+0000","last_fullsized":"2023-11-28T11:11:02.800320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.800729+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:11:02.800729+0000","last_peered":"2023-11-28T11:11:02.800729+0000","last_clean":"2023-11-28T11:11:02.800729+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:11:02.800729+0000","last_undegraded":"2023-11-28T11:11:02.800729+0000","last_fullsized":"2023-11-28T11:11:02.800729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.800313+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:11:02.800313+0000","last_peered":"2023-11-28T11:11:02.800313+0000","last_clean":"2023-11-28T11:11:02.800313+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:11:02.800313+0000","last_undegraded":"2023-11-28T11:11:02.800313+0000","last_fullsized":"2023-11-28T11:11:02.800313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.881685+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:11:02.881685+0000","last_peered":"2023-11-28T11:11:02.881685+0000","last_clean":"2023-11-28T11:11:02.881685+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:11:02.881685+0000","last_undegraded":"2023-11-28T11:11:02.881685+0000","last_fullsized":"2023-11-28T11:11:02.881685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11040,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360800,"statfs":{"total":96636764160,"available":96625459200,"internally_reserved":0,"allocated":1400832,"data_stored":944656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53600000000000003,"15min":0.54800000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.76100000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.621},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55500000000000005,"15min":0.56499999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":1.006,"15min":1.006},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73299999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.746,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.63,"15min":0.68500000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66500000000000004,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56299999999999994}]},{"osd":3,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.36199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82499999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.66800000000000004,"15min":0.68100000000000005},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83699999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66800000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.501},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68300000000000005,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.60799999999999998},"min":{"1min":0.37,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.747,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59699999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.625,"15min":0.625},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79800000000000004,"5min":1.018,"15min":1.018},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72999999999999998,"5min":1.123,"15min":1.123},"last":0.63600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9492,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362348,"statfs":{"total":96636764160,"available":96627044352,"internally_reserved":0,"allocated":798720,"data_stored":353632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.66700000000000004,"15min":0.73499999999999999},"min":{"1min":0.372,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71699999999999997,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.66600000000000004,"15min":0.72199999999999998},"min":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72199999999999998,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.51400000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.753,"15min":0.83899999999999997},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72799999999999998,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.71299999999999997,"15min":0.78600000000000003},"min":{"1min":0.40000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.67400000000000004,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.59599999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.72399999999999998,"15min":0.81399999999999995},"min":{"1min":0.32700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68200000000000005,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.379},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.66100000000000003,"15min":0.70699999999999996},"min":{"1min":0.40600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.76800000000000002,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.34699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.44700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78400000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.63300000000000001,"15min":0.66900000000000004},"min":{"1min":0.39800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.441}]},{"osd":4,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81699999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.501},{"interface":"front","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.622}]},{"osd":7,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.61299999999999999},"min":{"1min":0.45400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9780,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362060,"statfs":{"total":96636764160,"available":96626749440,"internally_reserved":0,"allocated":831488,"data_stored":366134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.53800000000000003,"15min":0.626},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.59499999999999997,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.371,"15min":0.38},"min":{"1min":0.247,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56100000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.26600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.57499999999999996,"15min":0.66700000000000004},"min":{"1min":0.26300000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.48599999999999999,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.372},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40600000000000003,"15min":0.41599999999999998},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.53000000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28299999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56399999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.377,"15min":0.375},"min":{"1min":0.313,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51500000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.307}]},{"osd":4,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":1.139,"15min":1.373},"min":{"1min":0.40600000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79800000000000004,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86799999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45000000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80200000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":1.1679999999999999,"15min":1.403},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85299999999999998,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.5}]},{"osd":6,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":1.1719999999999999,"15min":1.4079999999999999},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.433,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.62},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88700000000000001,"5min":1.018,"15min":1.018},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.84699999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.47499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14868,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":13504,"kb_avail":94356972,"statfs":{"total":96636764160,"available":96621539328,"internally_reserved":0,"allocated":1388544,"data_stored":943966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13828096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.35099999999999998},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34999999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.45300000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.24199999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38900000000000001,"15min":0.40200000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48299999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.375,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.55200000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.28399999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.26200000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56000000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.30599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.54400000000000004,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.26500000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":0.878,"15min":0.878},"last":0.51800000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.621},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.68100000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.629,"15min":0.629},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83199999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60699999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.62},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":1.026,"15min":1.026},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9620,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362220,"statfs":{"total":96636764160,"available":96626913280,"internally_reserved":0,"allocated":798720,"data_stored":353668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.31900000000000001},"min":{"1min":0.23200000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54500000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.38600000000000001,"15min":0.37},"min":{"1min":0.312,"5min":0.245,"15min":0.245},"max":{"1min":0.77400000000000002,"5min":1.131,"15min":1.131},"last":0.27400000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.33900000000000002},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58099999999999996,"5min":1.151,"15min":1.151},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.35999999999999999,"15min":0.34100000000000003},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.74099999999999999,"5min":1.177,"15min":1.177},"last":0.24199999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34399999999999997,"15min":0.33500000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.50900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39300000000000002,"15min":0.373},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59399999999999997,"5min":1.109,"15min":1.109},"last":0.28799999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71199999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.47499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":1.502,"15min":1.502},"last":0.51100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79000000000000004,"5min":1.323,"15min":1.323},"last":0.498},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.503,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86199999999999999,"5min":1.464,"15min":1.464},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":1.355,"15min":1.355},"last":0.54100000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.50900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82499999999999996,"5min":1.421,"15min":1.421},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.443,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81200000000000006,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.47399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13968,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":13184,"kb_avail":94357872,"statfs":{"total":96636764160,"available":96622460928,"internally_reserved":0,"allocated":794624,"data_stored":353598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13500416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34100000000000003,"15min":0.34399999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.31900000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44600000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.36499999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.253,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.44500000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59299999999999997,"5min":1.365,"15min":1.365},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.374,"15min":0.371},"min":{"1min":0.30599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.27200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55800000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.441}]},{"osd":4,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.44900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70099999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.75600000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71499999999999997,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.745,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.67500000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75700000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79900000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58099999999999996,"15min":0.56999999999999995},"min":{"1min":0.499,"5min":0.436,"15min":0.436},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.59199999999999997},"min":{"1min":0.48999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.69499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9828,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362012,"statfs":{"total":96636764160,"available":96626700288,"internally_reserved":0,"allocated":815104,"data_stored":365386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.629,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.55600000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":4.9740000000000002,"15min":4.9740000000000002},"last":0.54200000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.81200000000000006,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.439},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.97699999999999998,"15min":1.161},"min":{"1min":0.44700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":12.263,"15min":12.263},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80300000000000005,"5min":1.524,"15min":1.524},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76100000000000001,"5min":1.236,"15min":1.236},"last":0.52000000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56200000000000006,"15min":0.54300000000000004},"min":{"1min":0.44500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74299999999999999,"5min":1.54,"15min":1.54},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.69599999999999995,"15min":0.72199999999999998},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.498}]},{"osd":5,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.85799999999999998,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59799999999999998,"15min":0.55100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95499999999999996,"5min":5.0220000000000002,"15min":5.0220000000000002},"last":0.63}]},{"osd":6,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.93600000000000005,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":1.923,"15min":1.923},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82399999999999995,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.996,"5min":1.49,"15min":1.49},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9684,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362156,"statfs":{"total":96636764160,"available":96626847744,"internally_reserved":0,"allocated":798720,"data_stored":353620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.501},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40799999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69599999999999995,"5min":0.75,"15min":0.75},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.437}]},{"osd":2,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.495,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.54800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46300000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.502,"15min":0.499},"min":{"1min":0.36899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60299999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.42999999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:10.500 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:10.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.620+0000 7f9c61605700 1 Processor -- start 2023-11-28T11:12:10.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.620+0000 7f9c61605700 1 -- start start 2023-11-28T11:12:10.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.621+0000 7f9c61605700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9c5c15ca50 0x7f9c5c06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.621+0000 7f9c61605700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c5c0cce10 0x7f9c5c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.621+0000 7f9c61605700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c0cf5e0 con 0x7f9c5c06b8c0 2023-11-28T11:12:10.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.621+0000 7f9c61605700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5c0cd290 con 0x7f9c5c15ca50 2023-11-28T11:12:10.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.621+0000 7f9c61605700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c06bd60 con 0x7f9c5c0cce10 2023-11-28T11:12:10.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.621+0000 7f9c5b7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c5c0cce10 0x7f9c5c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.621+0000 7f9c5affd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9c5c15ca50 0x7f9c5c06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.621+0000 7f9c5b7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c5c0cce10 0x7f9c5c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:40434/0 (socket says 172.21.15.156:40434) 2023-11-28T11:12:10.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.621+0000 7f9c5b7fe700 1 -- 172.21.15.156:0/3740363377 learned_addr learned my addr 172.21.15.156:0/3740363377 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:10.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c5b7fe700 1 -- 172.21.15.156:0/3740363377 >> v2:172.21.15.186:3300/0 conn(0x7f9c5c15ca50 msgr2=0x7f9c5c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:10.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c5b7fe700 1 --2- 172.21.15.156:0/3740363377 >> v2:172.21.15.186:3300/0 conn(0x7f9c5c15ca50 0x7f9c5c06b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c5b7fe700 1 -- 172.21.15.156:0/3740363377 >> v1:172.21.15.186:6789/0 conn(0x7f9c5c06b8c0 legacy=0x7f9c5c0cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c5b7fe700 1 -- 172.21.15.156:0/3740363377 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c0cecb0 con 0x7f9c5c0cce10 2023-11-28T11:12:10.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c5b7fe700 1 --2- 172.21.15.156:0/3740363377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c5c0cce10 0x7f9c5c0cfa70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9c54009e50 tx=0x7f9c5400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e998a30ba191c592 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:10.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c59ffb700 1 -- 172.21.15.156:0/3740363377 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400e070 con 0x7f9c5c0cce10 2023-11-28T11:12:10.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c59ffb700 1 -- 172.21.15.156:0/3740363377 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5400b910 con 0x7f9c5c0cce10 2023-11-28T11:12:10.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c59ffb700 1 -- 172.21.15.156:0/3740363377 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400bcc0 con 0x7f9c5c0cce10 2023-11-28T11:12:10.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c61605700 1 -- 172.21.15.156:0/3740363377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c5c0cce10 msgr2=0x7f9c5c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c61605700 1 --2- 172.21.15.156:0/3740363377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c5c0cce10 0x7f9c5c0cfa70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9c54009e50 tx=0x7f9c5400b330 comp rx=0 tx=0).stop 2023-11-28T11:12:10.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c61605700 1 -- 172.21.15.156:0/3740363377 shutdown_connections 2023-11-28T11:12:10.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c61605700 1 --2- 172.21.15.156:0/3740363377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c5c0cce10 0x7f9c5c0cfa70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c61605700 1 --2- 172.21.15.156:0/3740363377 >> v2:172.21.15.186:3300/0 conn(0x7f9c5c15ca50 0x7f9c5c06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c61605700 1 -- 172.21.15.156:0/3740363377 >> 172.21.15.156:0/3740363377 conn(0x7f9c5c09a230 msgr2=0x7f9c5c08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:10.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c61605700 1 -- 172.21.15.156:0/3740363377 shutdown_connections 2023-11-28T11:12:10.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.622+0000 7f9c61605700 1 -- 172.21.15.156:0/3740363377 wait complete. 2023-11-28T11:12:10.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.624+0000 7f9c61605700 1 Processor -- start 2023-11-28T11:12:10.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.624+0000 7f9c61605700 1 -- start start 2023-11-28T11:12:10.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.624+0000 7f9c61605700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c5c06b8c0 0x7f9c5c1f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.624+0000 7f9c61605700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c5c0cce10 0x7f9c5c0c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.624+0000 7f9c61605700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c5c15ca50 0x7f9c5c0b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.624+0000 7f9c61605700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c14fd20 con 0x7f9c5c0cce10 2023-11-28T11:12:10.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.624+0000 7f9c61605700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0c4f00 con 0x7f9c5c06b8c0 2023-11-28T11:12:10.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.624+0000 7f9c61605700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0c5160 con 0x7f9c5c15ca50 2023-11-28T11:12:10.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c5a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c5c0cce10 0x7f9c5c0c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c5a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c5c0cce10 0x7f9c5c0c49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36576/0 (socket says 172.21.15.156:36576) 2023-11-28T11:12:10.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c5a7fc700 1 -- 172.21.15.156:0/725358919 learned_addr learned my addr 172.21.15.156:0/725358919 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:10.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c5b7fe700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c5c15ca50 0x7f9c5c0b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c5affd700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c5c06b8c0 0x7f9c5c1f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c5a7fc700 1 -- 172.21.15.156:0/725358919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c5c15ca50 msgr2=0x7f9c5c0b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c5a7fc700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c5c15ca50 0x7f9c5c0b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c5a7fc700 1 -- 172.21.15.156:0/725358919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c5c06b8c0 msgr2=0x7f9c5c1f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c5a7fc700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c5c06b8c0 0x7f9c5c1f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c5a7fc700 1 -- 172.21.15.156:0/725358919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c54009b30 con 0x7f9c5c0cce10 2023-11-28T11:12:10.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c5a7fc700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c5c0cce10 0x7f9c5c0c49c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00c4e0 tx=0x7f9c4c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:10.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c437fe700 1 -- 172.21.15.156:0/725358919 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0052d0 con 0x7f9c5c0cce10 2023-11-28T11:12:10.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c437fe700 1 -- 172.21.15.156:0/725358919 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c005430 con 0x7f9c5c0cce10 2023-11-28T11:12:10.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c61605700 1 -- 172.21.15.156:0/725358919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c0cf270 con 0x7f9c5c0cce10 2023-11-28T11:12:10.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c437fe700 1 -- 172.21.15.156:0/725358919 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0038c0 con 0x7f9c5c0cce10 2023-11-28T11:12:10.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.625+0000 7f9c61605700 1 -- 172.21.15.156:0/725358919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c0cef90 con 0x7f9c5c0cce10 2023-11-28T11:12:10.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.626+0000 7f9c61605700 1 -- 172.21.15.156:0/725358919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c15ce20 con 0x7f9c5c0cce10 2023-11-28T11:12:10.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.627+0000 7f9c437fe700 1 -- 172.21.15.156:0/725358919 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9c4c019030 con 0x7f9c5c0cce10 2023-11-28T11:12:10.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.627+0000 7f9c437fe700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9c480682f0 0x7f9c4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.627+0000 7f9c437fe700 1 -- 172.21.15.156:0/725358919 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9c4c084bf0 con 0x7f9c5c0cce10 2023-11-28T11:12:10.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.627+0000 7f9c5affd700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9c480682f0 0x7f9c4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.627+0000 7f9c5affd700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9c480682f0 0x7f9c4806a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009b40 tx=0x7f9c50009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:10.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.631+0000 7f9c437fe700 1 -- 172.21.15.156:0/725358919 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9c4c08b080 con 0x7f9c5c0cce10 2023-11-28T11:12:10.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.807+0000 7f9c61605700 1 -- 172.21.15.156:0/725358919 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c5c0745c0 con 0x7f9c480682f0 2023-11-28T11:12:10.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.809+0000 7f9c437fe700 1 -- 172.21.15.156:0/725358919 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72794 (secure 0 0 0) 0x7f9c5c0745c0 con 0x7f9c480682f0 2023-11-28T11:12:10.812 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:10.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.812+0000 7f9c61605700 1 -- 172.21.15.156:0/725358919 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9c480682f0 msgr2=0x7f9c4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.812+0000 7f9c61605700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9c480682f0 0x7f9c4806a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009b40 tx=0x7f9c50009210 comp rx=0 tx=0).stop 2023-11-28T11:12:10.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.812+0000 7f9c61605700 1 -- 172.21.15.156:0/725358919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c5c0cce10 msgr2=0x7f9c5c0c49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.812+0000 7f9c61605700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c5c0cce10 0x7f9c5c0c49c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00c4e0 tx=0x7f9c4c00c7f0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.812+0000 7f9c61605700 1 -- 172.21.15.156:0/725358919 shutdown_connections 2023-11-28T11:12:10.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.812+0000 7f9c61605700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f9c480682f0 0x7f9c4806a7a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.812+0000 7f9c61605700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c5c15ca50 0x7f9c5c0b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.812+0000 7f9c61605700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c5c06b8c0 0x7f9c5c1f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.812+0000 7f9c61605700 1 --2- 172.21.15.156:0/725358919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c5c0cce10 0x7f9c5c0c49c0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.812+0000 7f9c61605700 1 -- 172.21.15.156:0/725358919 >> 172.21.15.156:0/725358919 conn(0x7f9c5c09a230 msgr2=0x7f9c5c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:10.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.812+0000 7f9c61605700 1 -- 172.21.15.156:0/725358919 shutdown_connections 2023-11-28T11:12:10.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.812+0000 7f9c61605700 1 -- 172.21.15.156:0/725358919 wait complete. 2023-11-28T11:12:10.815 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:12:10.825 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2023-11-28T11:12:10.578385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":88280,"kb_used_data":7448,"kb_used_omap":0,"kb_used_meta":80768,"kb_avail":754886440,"statfs":{"total":773094113280,"available":773003714560,"internally_reserved":0,"allocated":7626752,"data_stored":4034660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":82706432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12570,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-75,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-70,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000977"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.799800+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:11:02.799800+0000","last_peered":"2023-11-28T11:11:02.799800+0000","last_clean":"2023-11-28T11:11:02.799800+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:11:02.799800+0000","last_undegraded":"2023-11-28T11:11:02.799800+0000","last_fullsized":"2023-11-28T11:11:02.799800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.795953+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:11:02.795953+0000","last_peered":"2023-11-28T11:11:02.795953+0000","last_clean":"2023-11-28T11:11:02.795953+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:11:02.795953+0000","last_undegraded":"2023-11-28T11:11:02.795953+0000","last_fullsized":"2023-11-28T11:11:02.795953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.797281+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:11:02.797281+0000","last_peered":"2023-11-28T11:11:02.797281+0000","last_clean":"2023-11-28T11:11:02.797281+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:11:02.797281+0000","last_undegraded":"2023-11-28T11:11:02.797281+0000","last_fullsized":"2023-11-28T11:11:02.797281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.796416+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:11:02.796416+0000","last_peered":"2023-11-28T11:11:02.796416+0000","last_clean":"2023-11-28T11:11:02.796416+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:11:02.796416+0000","last_undegraded":"2023-11-28T11:11:02.796416+0000","last_fullsized":"2023-11-28T11:11:02.796416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":207,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.800135+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:11:02.800135+0000","last_peered":"2023-11-28T11:11:02.800135+0000","last_clean":"2023-11-28T11:11:02.800135+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:11:02.800135+0000","last_undegraded":"2023-11-28T11:11:02.800135+0000","last_fullsized":"2023-11-28T11:11:02.800135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.800320+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:11:02.800320+0000","last_peered":"2023-11-28T11:11:02.800320+0000","last_clean":"2023-11-28T11:11:02.800320+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:11:02.800320+0000","last_undegraded":"2023-11-28T11:11:02.800320+0000","last_fullsized":"2023-11-28T11:11:02.800320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.800729+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:11:02.800729+0000","last_peered":"2023-11-28T11:11:02.800729+0000","last_clean":"2023-11-28T11:11:02.800729+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:11:02.800729+0000","last_undegraded":"2023-11-28T11:11:02.800729+0000","last_fullsized":"2023-11-28T11:11:02.800729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.800313+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:11:02.800313+0000","last_peered":"2023-11-28T11:11:02.800313+0000","last_clean":"2023-11-28T11:11:02.800313+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:11:02.800313+0000","last_undegraded":"2023-11-28T11:11:02.800313+0000","last_fullsized":"2023-11-28T11:11:02.800313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.881685+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:11:02.881685+0000","last_peered":"2023-11-28T11:11:02.881685+0000","last_clean":"2023-11-28T11:11:02.881685+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:11:02.881685+0000","last_undegraded":"2023-11-28T11:11:02.881685+0000","last_fullsized":"2023-11-28T11:11:02.881685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11040,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360800,"statfs":{"total":96636764160,"available":96625459200,"internally_reserved":0,"allocated":1400832,"data_stored":944656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53600000000000003,"15min":0.54800000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.76100000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.621},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55500000000000005,"15min":0.56499999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":1.006,"15min":1.006},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73299999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.746,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.63,"15min":0.68500000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66500000000000004,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56299999999999994}]},{"osd":3,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.36199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82499999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.66800000000000004,"15min":0.68100000000000005},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83699999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66800000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.501},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68300000000000005,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.60799999999999998},"min":{"1min":0.37,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.747,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59699999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.625,"15min":0.625},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79800000000000004,"5min":1.018,"15min":1.018},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72999999999999998,"5min":1.123,"15min":1.123},"last":0.63600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9492,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362348,"statfs":{"total":96636764160,"available":96627044352,"internally_reserved":0,"allocated":798720,"data_stored":353632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.66700000000000004,"15min":0.73499999999999999},"min":{"1min":0.372,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71699999999999997,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.66600000000000004,"15min":0.72199999999999998},"min":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72199999999999998,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.51400000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.753,"15min":0.83899999999999997},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72799999999999998,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.71299999999999997,"15min":0.78600000000000003},"min":{"1min":0.40000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.67400000000000004,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.59599999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.72399999999999998,"15min":0.81399999999999995},"min":{"1min":0.32700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68200000000000005,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.379},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.66100000000000003,"15min":0.70699999999999996},"min":{"1min":0.40600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.76800000000000002,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.34699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.44700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78400000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.63300000000000001,"15min":0.66900000000000004},"min":{"1min":0.39800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.441}]},{"osd":4,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81699999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.501},{"interface":"front","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.622}]},{"osd":7,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.61299999999999999},"min":{"1min":0.45400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9780,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362060,"statfs":{"total":96636764160,"available":96626749440,"internally_reserved":0,"allocated":831488,"data_stored":366134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.53800000000000003,"15min":0.626},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.59499999999999997,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.371,"15min":0.38},"min":{"1min":0.247,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56100000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.26600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.57499999999999996,"15min":0.66700000000000004},"min":{"1min":0.26300000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.48599999999999999,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.372},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40600000000000003,"15min":0.41599999999999998},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.53000000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28299999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56399999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.377,"15min":0.375},"min":{"1min":0.313,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51500000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.307}]},{"osd":4,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":1.139,"15min":1.373},"min":{"1min":0.40600000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79800000000000004,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86799999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45000000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80200000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":1.1679999999999999,"15min":1.403},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85299999999999998,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.5}]},{"osd":6,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":1.1719999999999999,"15min":1.4079999999999999},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.433,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.62},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88700000000000001,"5min":1.018,"15min":1.018},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.84699999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.47499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14868,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":13504,"kb_avail":94356972,"statfs":{"total":96636764160,"available":96621539328,"internally_reserved":0,"allocated":1388544,"data_stored":943966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13828096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.35099999999999998},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34999999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.45300000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.24199999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38900000000000001,"15min":0.40200000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48299999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.375,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.55200000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.28399999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.26200000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56000000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.30599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.54400000000000004,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.26500000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":0.878,"15min":0.878},"last":0.51800000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.621},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.68100000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.629,"15min":0.629},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83199999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60699999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.62},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":1.026,"15min":1.026},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9620,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362220,"statfs":{"total":96636764160,"available":96626913280,"internally_reserved":0,"allocated":798720,"data_stored":353668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.31900000000000001},"min":{"1min":0.23200000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54500000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.38600000000000001,"15min":0.37},"min":{"1min":0.312,"5min":0.245,"15min":0.245},"max":{"1min":0.77400000000000002,"5min":1.131,"15min":1.131},"last":0.27400000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.33900000000000002},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58099999999999996,"5min":1.151,"15min":1.151},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.35999999999999999,"15min":0.34100000000000003},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.74099999999999999,"5min":1.177,"15min":1.177},"last":0.24199999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34399999999999997,"15min":0.33500000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.50900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39300000000000002,"15min":0.373},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59399999999999997,"5min":1.109,"15min":1.109},"last":0.28799999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71199999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.47499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":1.502,"15min":1.502},"last":0.51100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79000000000000004,"5min":1.323,"15min":1.323},"last":0.498},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.503,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86199999999999999,"5min":1.464,"15min":1.464},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":1.355,"15min":1.355},"last":0.54100000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.50900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82499999999999996,"5min":1.421,"15min":1.421},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.443,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81200000000000006,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.47399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13968,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":13184,"kb_avail":94357872,"statfs":{"total":96636764160,"available":96622460928,"internally_reserved":0,"allocated":794624,"data_stored":353598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13500416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34100000000000003,"15min":0.34399999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.31900000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44600000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.36499999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.253,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.44500000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59299999999999997,"5min":1.365,"15min":1.365},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.374,"15min":0.371},"min":{"1min":0.30599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.27200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55800000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.441}]},{"osd":4,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.44900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70099999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.75600000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71499999999999997,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.745,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.67500000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75700000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79900000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58099999999999996,"15min":0.56999999999999995},"min":{"1min":0.499,"5min":0.436,"15min":0.436},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.59199999999999997},"min":{"1min":0.48999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.69499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9828,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362012,"statfs":{"total":96636764160,"available":96626700288,"internally_reserved":0,"allocated":815104,"data_stored":365386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.629,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.55600000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":4.9740000000000002,"15min":4.9740000000000002},"last":0.54200000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.81200000000000006,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.439},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.97699999999999998,"15min":1.161},"min":{"1min":0.44700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":12.263,"15min":12.263},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80300000000000005,"5min":1.524,"15min":1.524},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76100000000000001,"5min":1.236,"15min":1.236},"last":0.52000000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56200000000000006,"15min":0.54300000000000004},"min":{"1min":0.44500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74299999999999999,"5min":1.54,"15min":1.54},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.69599999999999995,"15min":0.72199999999999998},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.498}]},{"osd":5,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.85799999999999998,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59799999999999998,"15min":0.55100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95499999999999996,"5min":5.0220000000000002,"15min":5.0220000000000002},"last":0.63}]},{"osd":6,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.93600000000000005,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":1.923,"15min":1.923},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82399999999999995,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.996,"5min":1.49,"15min":1.49},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9684,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362156,"statfs":{"total":96636764160,"available":96626847744,"internally_reserved":0,"allocated":798720,"data_stored":353620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.501},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40799999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69599999999999995,"5min":0.75,"15min":0.75},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.437}]},{"osd":2,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.495,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.54800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46300000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.502,"15min":0.499},"min":{"1min":0.36899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60299999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.42999999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:10.828 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T11:12:10.828 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:10.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.947+0000 7f7e58e00700 1 Processor -- start 2023-11-28T11:12:10.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.948+0000 7f7e58e00700 1 -- start start 2023-11-28T11:12:10.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e58e00700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7e5415c9b0 0x7f7e540b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e58e00700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e54161340 0x7f7e540d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e58e00700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e540d1950 con 0x7f7e540d2320 2023-11-28T11:12:10.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e58e00700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e540b5800 con 0x7f7e5415c9b0 2023-11-28T11:12:10.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e58e00700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540b5520 con 0x7f7e54161340 2023-11-28T11:12:10.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e52d9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7e540d2320 0x7f7e540cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51462/0 (socket says 172.21.15.156:51462) 2023-11-28T11:12:10.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e52d9d700 1 -- 172.21.15.156:0/265958020 learned_addr learned my addr 172.21.15.156:0/265958020 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:10.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829304450 0 0) 0x7f7e540d1950 con 0x7f7e540d2320 2023-11-28T11:12:10.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e400035b0 con 0x7f7e540d2320 2023-11-28T11:12:10.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e51d9b700 1 --2- 172.21.15.156:0/265958020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e54161340 0x7f7e540d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5259c700 1 --2- 172.21.15.156:0/265958020 >> v2:172.21.15.186:3300/0 conn(0x7f7e5415c9b0 0x7f7e540b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 309824877 0 0) 0x7f7e400035b0 con 0x7f7e540d2320 2023-11-28T11:12:10.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e54161340 msgr2=0x7f7e540d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:10.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 --2- 172.21.15.156:0/265958020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e54161340 0x7f7e540d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 >> v2:172.21.15.186:3300/0 conn(0x7f7e5415c9b0 msgr2=0x7f7e540b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:10.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 --2- 172.21.15.156:0/265958020 >> v2:172.21.15.186:3300/0 conn(0x7f7e5415c9b0 0x7f7e540b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e540d1300 con 0x7f7e540d2320 2023-11-28T11:12:10.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f7e4c0019f0 con 0x7f7e540d2320 2023-11-28T11:12:10.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 >> v1:172.21.15.156:6789/0 conn(0x7f7e540d2320 legacy=0x7f7e540cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 --2- 172.21.15.156:0/265958020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e40003a00 0x7f7e40003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 --2- 172.21.15.156:0/265958020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e54161340 0x7f7e40004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 --2- 172.21.15.156:0/265958020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e40004d90 0x7f7e40005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540b5800 con 0x7f7e40004d90 2023-11-28T11:12:10.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540b5520 con 0x7f7e40003a00 2023-11-28T11:12:10.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4000ba10 con 0x7f7e54161340 2023-11-28T11:12:10.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.949+0000 7f7e52d9d700 1 --2- 172.21.15.156:0/265958020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e40004d90 0x7f7e40005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.950+0000 7f7e5259c700 1 --2- 172.21.15.156:0/265958020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e40003a00 0x7f7e40003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.950+0000 7f7e51d9b700 1 --2- 172.21.15.156:0/265958020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e54161340 0x7f7e40004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.950+0000 7f7e52d9d700 1 -- 172.21.15.156:0/265958020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e54161340 msgr2=0x7f7e40004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:10.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.950+0000 7f7e52d9d700 1 --2- 172.21.15.156:0/265958020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e54161340 0x7f7e40004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.950+0000 7f7e52d9d700 1 -- 172.21.15.156:0/265958020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e40003a00 msgr2=0x7f7e40003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:10.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.950+0000 7f7e52d9d700 1 --2- 172.21.15.156:0/265958020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e40003a00 0x7f7e40003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.950+0000 7f7e52d9d700 1 -- 172.21.15.156:0/265958020 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e400035b0 con 0x7f7e40004d90 2023-11-28T11:12:10.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.950+0000 7f7e52d9d700 1 --2- 172.21.15.156:0/265958020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e40004d90 0x7f7e40005200 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00a8d0 tx=0x7f7e40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4516a124900092 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:10.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.950+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c00a340 con 0x7f7e40004d90 2023-11-28T11:12:10.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.950+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c012600 con 0x7f7e40004d90 2023-11-28T11:12:10.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.950+0000 7f7e5159a700 1 -- 172.21.15.156:0/265958020 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0129b0 con 0x7f7e40004d90 2023-11-28T11:12:10.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.951+0000 7f7e58e00700 1 -- 172.21.15.156:0/265958020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e40004d90 msgr2=0x7f7e40005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.951+0000 7f7e58e00700 1 --2- 172.21.15.156:0/265958020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e40004d90 0x7f7e40005200 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00a8d0 tx=0x7f7e40005880 comp rx=0 tx=0).stop 2023-11-28T11:12:10.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.951+0000 7f7e52d9d700 1 -- 172.21.15.156:0/265958020 reap_dead start 2023-11-28T11:12:10.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.951+0000 7f7e58e00700 1 -- 172.21.15.156:0/265958020 shutdown_connections 2023-11-28T11:12:10.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.951+0000 7f7e58e00700 1 -- 172.21.15.156:0/265958020 >> 172.21.15.156:0/265958020 conn(0x7f7e54072600 msgr2=0x7f7e540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:10.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.951+0000 7f7e58e00700 1 -- 172.21.15.156:0/265958020 shutdown_connections 2023-11-28T11:12:10.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.951+0000 7f7e58e00700 1 -- 172.21.15.156:0/265958020 wait complete. 2023-11-28T11:12:10.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.952+0000 7f7e58e00700 1 Processor -- start 2023-11-28T11:12:10.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.952+0000 7f7e58e00700 1 -- start start 2023-11-28T11:12:10.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e58e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e40004d90 0x7f7e54154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e58e00700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e540c5d80 0x7f7e54153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e58e00700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e540c61f0 0x7f7e5415f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e58e00700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e54151630 con 0x7f7e540c61f0 2023-11-28T11:12:10.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e58e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e54151910 con 0x7f7e40004d90 2023-11-28T11:12:10.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e58e00700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e541550f0 con 0x7f7e540c5d80 2023-11-28T11:12:10.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e52d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e540c61f0 0x7f7e5415f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e52d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e540c61f0 0x7f7e5415f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36596/0 (socket says 172.21.15.156:36596) 2023-11-28T11:12:10.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e52d9d700 1 -- 172.21.15.156:0/1271321470 learned_addr learned my addr 172.21.15.156:0/1271321470 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:10.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e51d9b700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e540c5d80 0x7f7e54153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e5259c700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e40004d90 0x7f7e54154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e52d9d700 1 -- 172.21.15.156:0/1271321470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e540c5d80 msgr2=0x7f7e54153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e52d9d700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e540c5d80 0x7f7e54153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e52d9d700 1 -- 172.21.15.156:0/1271321470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e40004d90 msgr2=0x7f7e54154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:10.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e52d9d700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e40004d90 0x7f7e54154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:10.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e52d9d700 1 -- 172.21.15.156:0/1271321470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e540b5130 con 0x7f7e540c61f0 2023-11-28T11:12:10.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.953+0000 7f7e52d9d700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e540c61f0 0x7f7e5415f070 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c012000 tx=0x7f7e4c001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:10.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.954+0000 7f7e4affd700 1 -- 172.21.15.156:0/1271321470 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c00a340 con 0x7f7e540c61f0 2023-11-28T11:12:10.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.954+0000 7f7e4affd700 1 -- 172.21.15.156:0/1271321470 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c0019c0 con 0x7f7e540c61f0 2023-11-28T11:12:10.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.954+0000 7f7e58e00700 1 -- 172.21.15.156:0/1271321470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e540d15e0 con 0x7f7e540c61f0 2023-11-28T11:12:10.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.954+0000 7f7e4affd700 1 -- 172.21.15.156:0/1271321470 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c026640 con 0x7f7e540c61f0 2023-11-28T11:12:10.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.954+0000 7f7e58e00700 1 -- 172.21.15.156:0/1271321470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e540b52c0 con 0x7f7e540c61f0 2023-11-28T11:12:10.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.955+0000 7f7e58e00700 1 -- 172.21.15.156:0/1271321470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e54151a80 con 0x7f7e540c61f0 2023-11-28T11:12:10.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.955+0000 7f7e4affd700 1 -- 172.21.15.156:0/1271321470 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7e4c010070 con 0x7f7e540c61f0 2023-11-28T11:12:10.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.955+0000 7f7e4affd700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7e340682f0 0x7f7e3406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:10.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.955+0000 7f7e4affd700 1 -- 172.21.15.156:0/1271321470 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7e4c0877f0 con 0x7f7e540c61f0 2023-11-28T11:12:10.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.956+0000 7f7e5259c700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7e340682f0 0x7f7e3406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:10.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.956+0000 7f7e5259c700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7e340682f0 0x7f7e3406a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009ba0 tx=0x7f7e3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:10.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:10.960+0000 7f7e4affd700 1 -- 172.21.15.156:0/1271321470 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7e4c054400 con 0x7f7e540c61f0 2023-11-28T11:12:11.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.138+0000 7f7e58e00700 1 -- 172.21.15.156:0/1271321470 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e540745c0 con 0x7f7e340682f0 2023-11-28T11:12:11.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.141+0000 7f7e4affd700 1 -- 172.21.15.156:0/1271321470 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72794 (secure 0 0 0) 0x7f7e540745c0 con 0x7f7e340682f0 2023-11-28T11:12:11.143 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:11.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.143+0000 7f7e58e00700 1 -- 172.21.15.156:0/1271321470 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7e340682f0 msgr2=0x7f7e3406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:11.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.143+0000 7f7e58e00700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7e340682f0 0x7f7e3406a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009ba0 tx=0x7f7e3c008040 comp rx=0 tx=0).stop 2023-11-28T11:12:11.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.143+0000 7f7e58e00700 1 -- 172.21.15.156:0/1271321470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e540c61f0 msgr2=0x7f7e5415f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:11.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.143+0000 7f7e58e00700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e540c61f0 0x7f7e5415f070 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c012000 tx=0x7f7e4c001360 comp rx=0 tx=0).stop 2023-11-28T11:12:11.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.143+0000 7f7e58e00700 1 -- 172.21.15.156:0/1271321470 shutdown_connections 2023-11-28T11:12:11.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.143+0000 7f7e58e00700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e540c61f0 0x7f7e5415f070 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:11.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.143+0000 7f7e58e00700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e40004d90 0x7f7e54154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:11.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.143+0000 7f7e58e00700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7e340682f0 0x7f7e3406a7a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:11.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.143+0000 7f7e58e00700 1 --2- 172.21.15.156:0/1271321470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e540c5d80 0x7f7e54153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:11.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.143+0000 7f7e58e00700 1 -- 172.21.15.156:0/1271321470 >> 172.21.15.156:0/1271321470 conn(0x7f7e54072600 msgr2=0x7f7e5406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:11.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.143+0000 7f7e58e00700 1 -- 172.21.15.156:0/1271321470 shutdown_connections 2023-11-28T11:12:11.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.143+0000 7f7e58e00700 1 -- 172.21.15.156:0/1271321470 wait complete. 2023-11-28T11:12:11.147 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:12:11.156 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2023-11-28T11:12:10.578385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":88280,"kb_used_data":7448,"kb_used_omap":0,"kb_used_meta":80768,"kb_avail":754886440,"statfs":{"total":773094113280,"available":773003714560,"internally_reserved":0,"allocated":7626752,"data_stored":4034660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":82706432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12570,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-75,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-70,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000977"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.799800+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:11:02.799800+0000","last_peered":"2023-11-28T11:11:02.799800+0000","last_clean":"2023-11-28T11:11:02.799800+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:11:02.799800+0000","last_undegraded":"2023-11-28T11:11:02.799800+0000","last_fullsized":"2023-11-28T11:11:02.799800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.795953+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:11:02.795953+0000","last_peered":"2023-11-28T11:11:02.795953+0000","last_clean":"2023-11-28T11:11:02.795953+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:11:02.795953+0000","last_undegraded":"2023-11-28T11:11:02.795953+0000","last_fullsized":"2023-11-28T11:11:02.795953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.797281+0000","last_change":"2023-11-28T11:08:11.952454+0000","last_active":"2023-11-28T11:11:02.797281+0000","last_peered":"2023-11-28T11:11:02.797281+0000","last_clean":"2023-11-28T11:11:02.797281+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:11:02.797281+0000","last_undegraded":"2023-11-28T11:11:02.797281+0000","last_fullsized":"2023-11-28T11:11:02.797281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:08.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.796416+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:11:02.796416+0000","last_peered":"2023-11-28T11:11:02.796416+0000","last_clean":"2023-11-28T11:11:02.796416+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:11:02.796416+0000","last_undegraded":"2023-11-28T11:11:02.796416+0000","last_fullsized":"2023-11-28T11:11:02.796416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":207,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.800135+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:11:02.800135+0000","last_peered":"2023-11-28T11:11:02.800135+0000","last_clean":"2023-11-28T11:11:02.800135+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:11:02.800135+0000","last_undegraded":"2023-11-28T11:11:02.800135+0000","last_fullsized":"2023-11-28T11:11:02.800135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.459301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.800320+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:11:02.800320+0000","last_peered":"2023-11-28T11:11:02.800320+0000","last_clean":"2023-11-28T11:11:02.800320+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:11:02.800320+0000","last_undegraded":"2023-11-28T11:11:02.800320+0000","last_fullsized":"2023-11-28T11:11:02.800320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.800729+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:11:02.800729+0000","last_peered":"2023-11-28T11:11:02.800729+0000","last_clean":"2023-11-28T11:11:02.800729+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:11:02.800729+0000","last_undegraded":"2023-11-28T11:11:02.800729+0000","last_fullsized":"2023-11-28T11:11:02.800729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.800313+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:11:02.800313+0000","last_peered":"2023-11-28T11:11:02.800313+0000","last_clean":"2023-11-28T11:11:02.800313+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:11:02.800313+0000","last_undegraded":"2023-11-28T11:11:02.800313+0000","last_fullsized":"2023-11-28T11:11:02.800313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.881685+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:11:02.881685+0000","last_peered":"2023-11-28T11:11:02.881685+0000","last_clean":"2023-11-28T11:11:02.881685+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:11:02.881685+0000","last_undegraded":"2023-11-28T11:11:02.881685+0000","last_fullsized":"2023-11-28T11:11:02.881685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:23:00.501336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11040,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360800,"statfs":{"total":96636764160,"available":96625459200,"internally_reserved":0,"allocated":1400832,"data_stored":944656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53600000000000003,"15min":0.54800000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.76100000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.621},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55500000000000005,"15min":0.56499999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":1.006,"15min":1.006},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73299999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.746,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.63,"15min":0.68500000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66500000000000004,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56299999999999994}]},{"osd":3,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.36199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82499999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.66800000000000004,"15min":0.68100000000000005},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83699999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.32100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66800000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.501},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68300000000000005,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.60799999999999998},"min":{"1min":0.37,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.747,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59699999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.625,"15min":0.625},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79800000000000004,"5min":1.018,"15min":1.018},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72999999999999998,"5min":1.123,"15min":1.123},"last":0.63600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9492,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362348,"statfs":{"total":96636764160,"available":96627044352,"internally_reserved":0,"allocated":798720,"data_stored":353632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.66700000000000004,"15min":0.73499999999999999},"min":{"1min":0.372,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71699999999999997,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.66600000000000004,"15min":0.72199999999999998},"min":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72199999999999998,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.51400000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.753,"15min":0.83899999999999997},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72799999999999998,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.71299999999999997,"15min":0.78600000000000003},"min":{"1min":0.40000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.67400000000000004,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.59599999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.72399999999999998,"15min":0.81399999999999995},"min":{"1min":0.32700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68200000000000005,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.379},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.66100000000000003,"15min":0.70699999999999996},"min":{"1min":0.40600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.76800000000000002,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.34699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.44700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78400000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.63300000000000001,"15min":0.66900000000000004},"min":{"1min":0.39800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.441}]},{"osd":4,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81699999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.501},{"interface":"front","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.622}]},{"osd":7,"last update":"Tue Nov 28 11:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.61299999999999999},"min":{"1min":0.45400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9780,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362060,"statfs":{"total":96636764160,"available":96626749440,"internally_reserved":0,"allocated":831488,"data_stored":366134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.53800000000000003,"15min":0.626},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.59499999999999997,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.371,"15min":0.38},"min":{"1min":0.247,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56100000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.26600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.57499999999999996,"15min":0.66700000000000004},"min":{"1min":0.26300000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.48599999999999999,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.372},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40600000000000003,"15min":0.41599999999999998},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.53000000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28299999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56399999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.377,"15min":0.375},"min":{"1min":0.313,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51500000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.307}]},{"osd":4,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":1.139,"15min":1.373},"min":{"1min":0.40600000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79800000000000004,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86799999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45000000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80200000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":1.1679999999999999,"15min":1.403},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85299999999999998,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.5}]},{"osd":6,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":1.1719999999999999,"15min":1.4079999999999999},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.433,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.62},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88700000000000001,"5min":1.018,"15min":1.018},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.84699999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.47499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14868,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":13504,"kb_avail":94356972,"statfs":{"total":96636764160,"available":96621539328,"internally_reserved":0,"allocated":1388544,"data_stored":943966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13828096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.35099999999999998},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34999999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.45300000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.24199999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38900000000000001,"15min":0.40200000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48299999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.375,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.55200000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.28399999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.26200000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56000000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.30599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.54400000000000004,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.26500000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":0.878,"15min":0.878},"last":0.51800000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.621},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.68100000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.629,"15min":0.629},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83199999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60699999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.62},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":1.026,"15min":1.026},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9620,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362220,"statfs":{"total":96636764160,"available":96626913280,"internally_reserved":0,"allocated":798720,"data_stored":353668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.31900000000000001},"min":{"1min":0.23200000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54500000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.38600000000000001,"15min":0.37},"min":{"1min":0.312,"5min":0.245,"15min":0.245},"max":{"1min":0.77400000000000002,"5min":1.131,"15min":1.131},"last":0.27400000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.33900000000000002},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58099999999999996,"5min":1.151,"15min":1.151},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.35999999999999999,"15min":0.34100000000000003},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.74099999999999999,"5min":1.177,"15min":1.177},"last":0.24199999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34399999999999997,"15min":0.33500000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.50900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39300000000000002,"15min":0.373},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59399999999999997,"5min":1.109,"15min":1.109},"last":0.28799999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71199999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.47499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":1.502,"15min":1.502},"last":0.51100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79000000000000004,"5min":1.323,"15min":1.323},"last":0.498},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.503,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86199999999999999,"5min":1.464,"15min":1.464},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":1.355,"15min":1.355},"last":0.54100000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.50900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82499999999999996,"5min":1.421,"15min":1.421},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.443,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81200000000000006,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.47399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13968,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":13184,"kb_avail":94357872,"statfs":{"total":96636764160,"available":96622460928,"internally_reserved":0,"allocated":794624,"data_stored":353598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13500416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34100000000000003,"15min":0.34399999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.31900000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44600000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.36499999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.253,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.44500000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59299999999999997,"5min":1.365,"15min":1.365},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.374,"15min":0.371},"min":{"1min":0.30599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.27200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55800000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.441}]},{"osd":4,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.44900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70099999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.75600000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71499999999999997,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.745,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.67500000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75700000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79900000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58099999999999996,"15min":0.56999999999999995},"min":{"1min":0.499,"5min":0.436,"15min":0.436},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.59199999999999997},"min":{"1min":0.48999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.69499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9828,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362012,"statfs":{"total":96636764160,"available":96626700288,"internally_reserved":0,"allocated":815104,"data_stored":365386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.629,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.55600000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":4.9740000000000002,"15min":4.9740000000000002},"last":0.54200000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.81200000000000006,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.439},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.97699999999999998,"15min":1.161},"min":{"1min":0.44700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":12.263,"15min":12.263},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80300000000000005,"5min":1.524,"15min":1.524},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76100000000000001,"5min":1.236,"15min":1.236},"last":0.52000000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56200000000000006,"15min":0.54300000000000004},"min":{"1min":0.44500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74299999999999999,"5min":1.54,"15min":1.54},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.69599999999999995,"15min":0.72199999999999998},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.498}]},{"osd":5,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.85799999999999998,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59799999999999998,"15min":0.55100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95499999999999996,"5min":5.0220000000000002,"15min":5.0220000000000002},"last":0.63}]},{"osd":6,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.93600000000000005,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":1.923,"15min":1.923},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82399999999999995,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.996,"5min":1.49,"15min":1.49},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9684,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362156,"statfs":{"total":96636764160,"available":96626847744,"internally_reserved":0,"allocated":798720,"data_stored":353620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.501},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40799999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69599999999999995,"5min":0.75,"15min":0.75},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.437}]},{"osd":2,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.495,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.54800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46300000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.502,"15min":0.499},"min":{"1min":0.36899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60299999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.42999999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:11.158 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:11.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.278+0000 7fb33bca3700 1 Processor -- start 2023-11-28T11:12:11.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.278+0000 7fb33bca3700 1 -- start start 2023-11-28T11:12:11.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.279+0000 7fb33bca3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3340cb0e0 0x7fb334065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:11.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.279+0000 7fb33bca3700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb334066580 con 0x7fb3340cec70 2023-11-28T11:12:11.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.279+0000 7fb33bca3700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3340cded0 con 0x7fb33406ec90 2023-11-28T11:12:11.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.279+0000 7fb33bca3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3340ce1b0 con 0x7fb3340cb0e0 2023-11-28T11:12:11.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.279+0000 7fb33a240700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb3340cec70 0x7fb33406d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51480/0 (socket says 172.21.15.156:51480) 2023-11-28T11:12:11.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.279+0000 7fb33a240700 1 -- 172.21.15.156:0/1758590491 learned_addr learned my addr 172.21.15.156:0/1758590491 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:11.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.279+0000 7fb33923e700 1 --2- 172.21.15.156:0/1758590491 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3340cb0e0 0x7fb334065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:11.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.279+0000 7fb338a3d700 1 -- 172.21.15.156:0/1758590491 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315048323 0 0) 0x7fb334066580 con 0x7fb3340cec70 2023-11-28T11:12:11.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.279+0000 7fb338a3d700 1 -- 172.21.15.156:0/1758590491 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3200035b0 con 0x7fb3340cec70 2023-11-28T11:12:11.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.280+0000 7fb338a3d700 1 -- 172.21.15.156:0/1758590491 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494564299 0 0) 0x7fb3340cded0 con 0x7fb33406ec90 2023-11-28T11:12:11.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.280+0000 7fb338a3d700 1 -- 172.21.15.156:0/1758590491 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb334066580 con 0x7fb33406ec90 2023-11-28T11:12:11.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.280+0000 7fb338a3d700 1 -- 172.21.15.156:0/1758590491 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 23328644 0 0) 0x7fb3200035b0 con 0x7fb3340cec70 2023-11-28T11:12:11.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.280+0000 7fb33923e700 1 -- 172.21.15.156:0/1758590491 >> v1:172.21.15.186:6789/0 conn(0x7fb33406ec90 legacy=0x7fb3340ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:11.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.280+0000 7fb33923e700 1 -- 172.21.15.156:0/1758590491 >> v1:172.21.15.156:6789/0 conn(0x7fb3340cec70 legacy=0x7fb33406d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:11.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.280+0000 7fb33923e700 1 -- 172.21.15.156:0/1758590491 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3340ce490 con 0x7fb3340cb0e0 2023-11-28T11:12:11.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.280+0000 7fb33923e700 1 --2- 172.21.15.156:0/1758590491 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3340cb0e0 0x7fb334065ef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb330009870 tx=0x7fb330009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=501466d69686d63e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:11.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.280+0000 7fb338a3d700 1 -- 172.21.15.156:0/1758590491 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c0019f0 con 0x7fb3340cb0e0 2023-11-28T11:12:11.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.280+0000 7fb338a3d700 1 -- 172.21.15.156:0/1758590491 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb330004030 con 0x7fb3340cb0e0 2023-11-28T11:12:11.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.280+0000 7fb338a3d700 1 -- 172.21.15.156:0/1758590491 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3300043c0 con 0x7fb3340cb0e0 2023-11-28T11:12:11.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.280+0000 7fb33bca3700 1 -- 172.21.15.156:0/1758590491 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3340cb0e0 msgr2=0x7fb334065ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:11.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.280+0000 7fb33bca3700 1 --2- 172.21.15.156:0/1758590491 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3340cb0e0 0x7fb334065ef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb330009870 tx=0x7fb330009b80 comp rx=0 tx=0).stop 2023-11-28T11:12:11.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.281+0000 7fb33bca3700 1 -- 172.21.15.156:0/1758590491 shutdown_connections 2023-11-28T11:12:11.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.281+0000 7fb33bca3700 1 --2- 172.21.15.156:0/1758590491 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3340cb0e0 0x7fb334065ef0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:11.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.281+0000 7fb33bca3700 1 -- 172.21.15.156:0/1758590491 >> 172.21.15.156:0/1758590491 conn(0x7fb3340b12b0 msgr2=0x7fb3340b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:11.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.281+0000 7fb33bca3700 1 -- 172.21.15.156:0/1758590491 shutdown_connections 2023-11-28T11:12:11.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.281+0000 7fb33bca3700 1 -- 172.21.15.156:0/1758590491 wait complete. 2023-11-28T11:12:11.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.282+0000 7fb33bca3700 1 Processor -- start 2023-11-28T11:12:11.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.282+0000 7fb33bca3700 1 -- start start 2023-11-28T11:12:11.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb33bca3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33406ec90 0x7fb3340c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:11.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb33bca3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3340cb0e0 0x7fb33414baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:11.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb33bca3700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3340cec70 0x7fb3340c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:11.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb33bca3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3340c60b0 con 0x7fb33406ec90 2023-11-28T11:12:11.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb33bca3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3340c5850 con 0x7fb3340cb0e0 2023-11-28T11:12:11.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb33bca3700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3340c5af0 con 0x7fb3340cec70 2023-11-28T11:12:11.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb339a3f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33406ec90 0x7fb3340c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:11.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb339a3f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33406ec90 0x7fb3340c3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36606/0 (socket says 172.21.15.156:36606) 2023-11-28T11:12:11.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb339a3f700 1 -- 172.21.15.156:0/2635170430 learned_addr learned my addr 172.21.15.156:0/2635170430 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:11.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb33a240700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3340cec70 0x7fb3340c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:11.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb33923e700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3340cb0e0 0x7fb33414baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:11.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb339a3f700 1 -- 172.21.15.156:0/2635170430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3340cec70 msgr2=0x7fb3340c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:11.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb339a3f700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3340cec70 0x7fb3340c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:11.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb339a3f700 1 -- 172.21.15.156:0/2635170430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3340cb0e0 msgr2=0x7fb33414baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:11.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb339a3f700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3340cb0e0 0x7fb33414baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:11.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb339a3f700 1 -- 172.21.15.156:0/2635170430 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3300094a0 con 0x7fb33406ec90 2023-11-28T11:12:11.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb339a3f700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33406ec90 0x7fb3340c3c30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb32400c520 tx=0x7fb32400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:11.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb32a7fc700 1 -- 172.21.15.156:0/2635170430 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3240052d0 con 0x7fb33406ec90 2023-11-28T11:12:11.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb32a7fc700 1 -- 172.21.15.156:0/2635170430 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb324005430 con 0x7fb33406ec90 2023-11-28T11:12:11.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb33bca3700 1 -- 172.21.15.156:0/2635170430 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3340cb6c0 con 0x7fb33406ec90 2023-11-28T11:12:11.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb32a7fc700 1 -- 172.21.15.156:0/2635170430 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3240108e0 con 0x7fb33406ec90 2023-11-28T11:12:11.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.283+0000 7fb33bca3700 1 -- 172.21.15.156:0/2635170430 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3340ce770 con 0x7fb33406ec90 2023-11-28T11:12:11.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.284+0000 7fb33bca3700 1 -- 172.21.15.156:0/2635170430 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33406f060 con 0x7fb33406ec90 2023-11-28T11:12:11.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.285+0000 7fb32a7fc700 1 -- 172.21.15.156:0/2635170430 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb324018020 con 0x7fb33406ec90 2023-11-28T11:12:11.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.285+0000 7fb32a7fc700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb3180682f0 0x7fb31806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:11.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.285+0000 7fb33923e700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb3180682f0 0x7fb31806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:11.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.285+0000 7fb32a7fc700 1 -- 172.21.15.156:0/2635170430 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb3240876d0 con 0x7fb33406ec90 2023-11-28T11:12:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.286+0000 7fb33923e700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb3180682f0 0x7fb31806a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb334070670 tx=0x7fb330008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.289+0000 7fb32a7fc700 1 -- 172.21.15.156:0/2635170430 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb324054230 con 0x7fb33406ec90 2023-11-28T11:12:11.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.471+0000 7fb33bca3700 1 -- 172.21.15.156:0/2635170430 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3340cf110 con 0x7fb33406ec90 2023-11-28T11:12:11.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.472+0000 7fb32a7fc700 1 -- 172.21.15.156:0/2635170430 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+14447 (secure 0 0 0) 0x7fb324057850 con 0x7fb33406ec90 2023-11-28T11:12:11.474 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:11.474 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":68,"fsid":"351da896-8dbf-473b-9472-c3454d9368bf","created":"2023-11-28T11:07:54.980566+0000","modified":"2023-11-28T11:12:07.970475+0000","last_up_change":"2023-11-28T11:08:05.845694+0000","last_in_change":"2023-11-28T11:07:57.479373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:08:06.566571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:08:10.283177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0bd69671-96c1-4bb2-a04c-644fa7e2ba5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":103563},{"type":"v1","addr":"172.21.15.156:6804","nonce":103563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":103563},{"type":"v1","addr":"172.21.15.156:6808","nonce":103563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":103563},{"type":"v1","addr":"172.21.15.156:6815","nonce":103563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":103563},{"type":"v1","addr":"172.21.15.156:6812","nonce":103563}]},"public_addr":"172.21.15.156:6804/103563","cluster_addr":"172.21.15.156:6808/103563","heartbeat_back_addr":"172.21.15.156:6815/103563","heartbeat_front_addr":"172.21.15.156:6812/103563","state":["exists","up"]},{"osd":1,"uuid":"fa3e6ec6-caad-43ac-b667-9fe984bbf201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":103566},{"type":"v1","addr":"172.21.15.156:6822","nonce":103566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":103566},{"type":"v1","addr":"172.21.15.156:6826","nonce":103566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":103566},{"type":"v1","addr":"172.21.15.156:6831","nonce":103566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":103566},{"type":"v1","addr":"172.21.15.156:6829","nonce":103566}]},"public_addr":"172.21.15.156:6822/103566","cluster_addr":"172.21.15.156:6826/103566","heartbeat_back_addr":"172.21.15.156:6831/103566","heartbeat_front_addr":"172.21.15.156:6829/103566","state":["exists","up"]},{"osd":2,"uuid":"c6cb5b07-8272-46e0-8551-afb3c654c34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":103564},{"type":"v1","addr":"172.21.15.156:6801","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6803","nonce":103564},{"type":"v1","addr":"172.21.15.156:6805","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6811","nonce":103564},{"type":"v1","addr":"172.21.15.156:6813","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6807","nonce":103564},{"type":"v1","addr":"172.21.15.156:6809","nonce":103564}]},"public_addr":"172.21.15.156:6801/103564","cluster_addr":"172.21.15.156:6805/103564","heartbeat_back_addr":"172.21.15.156:6813/103564","heartbeat_front_addr":"172.21.15.156:6809/103564","state":["exists","up"]},{"osd":3,"uuid":"769bf956-8498-49a7-8945-8cc91bcee69e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":103567},{"type":"v1","addr":"172.21.15.156:6817","nonce":103567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":103567},{"type":"v1","addr":"172.21.15.156:6819","nonce":103567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6825","nonce":103567},{"type":"v1","addr":"172.21.15.156:6827","nonce":103567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":103567},{"type":"v1","addr":"172.21.15.156:6823","nonce":103567}]},"public_addr":"172.21.15.156:6817/103567","cluster_addr":"172.21.15.156:6819/103567","heartbeat_back_addr":"172.21.15.156:6827/103567","heartbeat_front_addr":"172.21.15.156:6823/103567","state":["exists","up"]},{"osd":4,"uuid":"5e44c417-5342-4241-9950-331ce1ceb42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":103781},{"type":"v1","addr":"172.21.15.186:6825","nonce":103781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":103781},{"type":"v1","addr":"172.21.15.186:6827","nonce":103781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":103781},{"type":"v1","addr":"172.21.15.186:6831","nonce":103781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":103781},{"type":"v1","addr":"172.21.15.186:6829","nonce":103781}]},"public_addr":"172.21.15.186:6825/103781","cluster_addr":"172.21.15.186:6827/103781","heartbeat_back_addr":"172.21.15.186:6831/103781","heartbeat_front_addr":"172.21.15.186:6829/103781","state":["exists","up"]},{"osd":5,"uuid":"9927448f-0e01-4a87-9525-5052812549fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":103778},{"type":"v1","addr":"172.21.15.186:6809","nonce":103778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":103778},{"type":"v1","addr":"172.21.15.186:6811","nonce":103778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":103778},{"type":"v1","addr":"172.21.15.186:6815","nonce":103778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":103778},{"type":"v1","addr":"172.21.15.186:6813","nonce":103778}]},"public_addr":"172.21.15.186:6809/103778","cluster_addr":"172.21.15.186:6811/103778","heartbeat_back_addr":"172.21.15.186:6815/103778","heartbeat_front_addr":"172.21.15.186:6813/103778","state":["exists","up"]},{"osd":6,"uuid":"0890154e-fd59-440c-8e5c-58495a4cf47d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":103779},{"type":"v1","addr":"172.21.15.186:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":103779},{"type":"v1","addr":"172.21.15.186:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":103779},{"type":"v1","addr":"172.21.15.186:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":103779},{"type":"v1","addr":"172.21.15.186:6805","nonce":103779}]},"public_addr":"172.21.15.186:6801/103779","cluster_addr":"172.21.15.186:6803/103779","heartbeat_back_addr":"172.21.15.186:6807/103779","heartbeat_front_addr":"172.21.15.186:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"f639cc58-c503-44a9-aa39-4b000df009cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":103780},{"type":"v1","addr":"172.21.15.186:6817","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":103780},{"type":"v1","addr":"172.21.15.186:6819","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":103780},{"type":"v1","addr":"172.21.15.186:6823","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":103780},{"type":"v1","addr":"172.21.15.186:6821","nonce":103780}]},"public_addr":"172.21.15.186:6817/103780","cluster_addr":"172.21.15.186:6819/103780","heartbeat_back_addr":"172.21.15.186:6823/103780","heartbeat_front_addr":"172.21.15.186:6821/103780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.693672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.674849+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.613406+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.666440+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.734851+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.695804+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.652267+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:02.703157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6835/3811791267":"2023-11-29T11:12:04.618636+0000","172.21.15.186:6833/3811791267":"2023-11-29T11:12:04.618636+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:11.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.475+0000 7fb33bca3700 1 -- 172.21.15.156:0/2635170430 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb3180682f0 msgr2=0x7fb31806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:11.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.475+0000 7fb33bca3700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb3180682f0 0x7fb31806a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb334070670 tx=0x7fb330008030 comp rx=0 tx=0).stop 2023-11-28T11:12:11.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.475+0000 7fb33bca3700 1 -- 172.21.15.156:0/2635170430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33406ec90 msgr2=0x7fb3340c3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:11.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.475+0000 7fb33bca3700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33406ec90 0x7fb3340c3c30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb32400c520 tx=0x7fb32400c8e0 comp rx=0 tx=0).stop 2023-11-28T11:12:11.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.475+0000 7fb33bca3700 1 -- 172.21.15.156:0/2635170430 shutdown_connections 2023-11-28T11:12:11.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.475+0000 7fb33bca3700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fb3180682f0 0x7fb31806a7a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:11.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.475+0000 7fb33bca3700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3340cec70 0x7fb3340c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:11.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.475+0000 7fb33bca3700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33406ec90 0x7fb3340c3c30 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:11.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.475+0000 7fb33bca3700 1 --2- 172.21.15.156:0/2635170430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3340cb0e0 0x7fb33414baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:11.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.475+0000 7fb33bca3700 1 -- 172.21.15.156:0/2635170430 >> 172.21.15.156:0/2635170430 conn(0x7fb3340b12b0 msgr2=0x7fb3340b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:11.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.475+0000 7fb33bca3700 1 -- 172.21.15.156:0/2635170430 shutdown_connections 2023-11-28T11:12:11.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:11.475+0000 7fb33bca3700 1 -- 172.21.15.156:0/2635170430 wait complete. 2023-11-28T11:12:12.488 INFO:tasks.ceph:Scrubbing osd.0 2023-11-28T11:12:12.489 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:12.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.626+0000 7f7ba83a1700 1 Processor -- start 2023-11-28T11:12:12.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.626+0000 7f7ba83a1700 1 -- start start 2023-11-28T11:12:12.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba83a1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7ba00af060 0x7f7ba0070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba83a1700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7ba00af430 0x7f7ba006d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba83a1700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba00713c0 con 0x7f7ba006a270 2023-11-28T11:12:12.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba83a1700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba006dee0 con 0x7f7ba00af060 2023-11-28T11:12:12.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba83a1700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba00716a0 con 0x7f7ba00af430 2023-11-28T11:12:12.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba613d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7ba00af060 0x7f7ba0070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba693e700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7ba006a270 0x7f7ba006a690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51496/0 (socket says 172.21.15.156:51496) 2023-11-28T11:12:12.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba613d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7ba00af060 0x7f7ba0070d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36614/0 (socket says 172.21.15.156:36614) 2023-11-28T11:12:12.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba693e700 1 -- 172.21.15.156:0/2138627890 learned_addr learned my addr 172.21.15.156:0/2138627890 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:12.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba593c700 1 --2- 172.21.15.156:0/2138627890 >> v2:172.21.15.186:3300/0 conn(0x7f7ba00af430 0x7f7ba006d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba613d700 1 -- 172.21.15.156:0/2138627890 >> v2:172.21.15.186:3300/0 conn(0x7f7ba00af430 msgr2=0x7f7ba006d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:12.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba613d700 1 --2- 172.21.15.156:0/2138627890 >> v2:172.21.15.186:3300/0 conn(0x7f7ba00af430 0x7f7ba006d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba613d700 1 -- 172.21.15.156:0/2138627890 >> v1:172.21.15.156:6789/0 conn(0x7f7ba006a270 legacy=0x7f7ba006a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba613d700 1 -- 172.21.15.156:0/2138627890 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba0071980 con 0x7f7ba00af060 2023-11-28T11:12:12.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba613d700 1 --2- 172.21.15.156:0/2138627890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ba00af060 0x7f7ba0070d50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00af800 tx=0x7f7b90009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3690ccbfdf8145 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:12.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba513b700 1 -- 172.21.15.156:0/2138627890 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90011830 con 0x7f7ba00af060 2023-11-28T11:12:12.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba513b700 1 -- 172.21.15.156:0/2138627890 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b90011990 con 0x7f7ba00af060 2023-11-28T11:12:12.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.627+0000 7f7ba513b700 1 -- 172.21.15.156:0/2138627890 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90011d20 con 0x7f7ba00af060 2023-11-28T11:12:12.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.628+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2138627890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ba00af060 msgr2=0x7f7ba0070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.628+0000 7f7ba83a1700 1 --2- 172.21.15.156:0/2138627890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ba00af060 0x7f7ba0070d50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00af800 tx=0x7f7b90009b20 comp rx=0 tx=0).stop 2023-11-28T11:12:12.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.628+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2138627890 shutdown_connections 2023-11-28T11:12:12.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.628+0000 7f7ba83a1700 1 --2- 172.21.15.156:0/2138627890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ba00af060 0x7f7ba0070d50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.628+0000 7f7ba83a1700 1 --2- 172.21.15.156:0/2138627890 >> v2:172.21.15.186:3300/0 conn(0x7f7ba00af430 0x7f7ba006d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.628+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2138627890 >> 172.21.15.156:0/2138627890 conn(0x7f7ba006b380 msgr2=0x7f7ba006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:12.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.628+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2138627890 shutdown_connections 2023-11-28T11:12:12.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.628+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2138627890 wait complete. 2023-11-28T11:12:12.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.630+0000 7f7ba83a1700 1 Processor -- start 2023-11-28T11:12:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.630+0000 7f7ba83a1700 1 -- start start 2023-11-28T11:12:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba83a1700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ba006a270 0x7f7ba00c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba83a1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ba00af060 0x7f7ba00cf340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba83a1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ba00af430 0x7f7ba0156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba83a1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00c80e0 con 0x7f7ba00af430 2023-11-28T11:12:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba83a1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00c83c0 con 0x7f7ba00af060 2023-11-28T11:12:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba83a1700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00c86a0 con 0x7f7ba006a270 2023-11-28T11:12:12.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba693e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ba00af430 0x7f7ba0156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba693e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ba00af430 0x7f7ba0156140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36624/0 (socket says 172.21.15.156:36624) 2023-11-28T11:12:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba693e700 1 -- 172.21.15.156:0/2936734228 learned_addr learned my addr 172.21.15.156:0/2936734228 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba593c700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ba00af060 0x7f7ba00cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba613d700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ba006a270 0x7f7ba00c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba693e700 1 -- 172.21.15.156:0/2936734228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ba006a270 msgr2=0x7f7ba00c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba693e700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ba006a270 0x7f7ba00c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba693e700 1 -- 172.21.15.156:0/2936734228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ba00af060 msgr2=0x7f7ba00cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba693e700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ba00af060 0x7f7ba00cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba693e700 1 -- 172.21.15.156:0/2936734228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900094a0 con 0x7f7ba00af430 2023-11-28T11:12:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba693e700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ba00af430 0x7f7ba0156140 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800ca80 tx=0x7f7b9800ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7b96ffd700 1 -- 172.21.15.156:0/2936734228 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800aaa0 con 0x7f7ba00af430 2023-11-28T11:12:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7b96ffd700 1 -- 172.21.15.156:0/2936734228 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9800ac00 con 0x7f7ba00af430 2023-11-28T11:12:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7b96ffd700 1 -- 172.21.15.156:0/2936734228 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98012400 con 0x7f7ba00af430 2023-11-28T11:12:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2936734228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba0071f40 con 0x7f7ba00af430 2023-11-28T11:12:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.631+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2936734228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba0071c60 con 0x7f7ba00af430 2023-11-28T11:12:12.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.632+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2936734228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7ba00ca660 con 0x7f7ba00af430 2023-11-28T11:12:12.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.633+0000 7f7b96ffd700 1 -- 172.21.15.156:0/2936734228 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b98019030 con 0x7f7ba00af430 2023-11-28T11:12:12.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.633+0000 7f7b96ffd700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7b840682f0 0x7f7b8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.633+0000 7f7b96ffd700 1 -- 172.21.15.156:0/2936734228 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7b980853b0 con 0x7f7ba00af430 2023-11-28T11:12:12.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.633+0000 7f7ba613d700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7b840682f0 0x7f7b8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.633+0000 7f7b96ffd700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] conn(0x7f7b8406e760 0x7f7b84070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.633+0000 7f7b96ffd700 1 -- 172.21.15.156:0/2936734228 --> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b84071240 con 0x7f7b8406e760 2023-11-28T11:12:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.633+0000 7f7b96ffd700 1 -- 172.21.15.156:0/2936734228 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b98087a60 con 0x7f7ba00af430 2023-11-28T11:12:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.633+0000 7f7ba593c700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] conn(0x7f7b8406e760 0x7f7b84070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.634+0000 7f7ba613d700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7b840682f0 0x7f7b8406a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7b900090b0 tx=0x7f7b90008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.634+0000 7f7ba593c700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] conn(0x7f7b8406e760 0x7f7b84070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.634+0000 7f7b96ffd700 1 -- 172.21.15.156:0/2936734228 <== osd.0 v2:172.21.15.156:6802/103563 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7b84071240 con 0x7f7b8406e760 2023-11-28T11:12:12.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.648+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2936734228 --> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7ba00afd10 con 0x7f7b8406e760 2023-11-28T11:12:12.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.648+0000 7f7b96ffd700 1 -- 172.21.15.156:0/2936734228 <== osd.0 v2:172.21.15.156:6802/103563 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7ba00afd10 con 0x7f7b8406e760 2023-11-28T11:12:12.650 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:12:12.650 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:12.650 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:12:12.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.649+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] conn(0x7f7b8406e760 msgr2=0x7f7b84070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.649+0000 7f7ba83a1700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:6802/103563,v1:172.21.15.156:6804/103563] conn(0x7f7b8406e760 0x7f7b84070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.649+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7b840682f0 msgr2=0x7f7b8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.649+0000 7f7ba83a1700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f7b840682f0 0x7f7b8406a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7b900090b0 tx=0x7f7b90008030 comp rx=0 tx=0).stop 2023-11-28T11:12:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.649+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ba00af430 msgr2=0x7f7ba0156140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.649+0000 7f7ba83a1700 1 --2- 172.21.15.156:0/2936734228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ba00af430 0x7f7ba0156140 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800ca80 tx=0x7f7b9800ce40 comp rx=0 tx=0).stop 2023-11-28T11:12:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.649+0000 7f7ba693e700 1 -- 172.21.15.156:0/2936734228 reap_dead start 2023-11-28T11:12:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.649+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2936734228 shutdown_connections 2023-11-28T11:12:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.649+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2936734228 >> 172.21.15.156:0/2936734228 conn(0x7f7ba006b380 msgr2=0x7f7ba00d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:12.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.649+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2936734228 shutdown_connections 2023-11-28T11:12:12.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.649+0000 7f7ba83a1700 1 -- 172.21.15.156:0/2936734228 wait complete. 2023-11-28T11:12:12.662 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-28T11:12:12.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.782+0000 7f71ced5b700 1 Processor -- start 2023-11-28T11:12:12.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.782+0000 7f71ced5b700 1 -- start start 2023-11-28T11:12:12.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71ced5b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71c815cef0 0x7f71c80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71ced5b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f71c806ba90 0x7f71c80ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71ced5b700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c80cc5f0 con 0x7f71c80cbdf0 2023-11-28T11:12:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71ced5b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c815d470 con 0x7f71c806ba90 2023-11-28T11:12:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71ced5b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c815d750 con 0x7f71c815cef0 2023-11-28T11:12:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71cd2f8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f71c806ba90 0x7f71c80ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71cd2f8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f71c806ba90 0x7f71c80ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36634/0 (socket says 172.21.15.156:36634) 2023-11-28T11:12:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71cd2f8700 1 -- 172.21.15.156:0/1726005912 learned_addr learned my addr 172.21.15.156:0/1726005912 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71ccaf7700 1 --2- 172.21.15.156:0/1726005912 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71c815cef0 0x7f71c80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71cd2f8700 1 -- 172.21.15.156:0/1726005912 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71c815cef0 msgr2=0x7f71c80af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71cd2f8700 1 --2- 172.21.15.156:0/1726005912 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71c815cef0 0x7f71c80af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71cd2f8700 1 -- 172.21.15.156:0/1726005912 >> v1:172.21.15.186:6789/0 conn(0x7f71c80cbdf0 legacy=0x7f71c806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.783+0000 7f71cd2f8700 1 -- 172.21.15.156:0/1726005912 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c815da30 con 0x7f71c806ba90 2023-11-28T11:12:12.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.784+0000 7f71cd2f8700 1 --2- 172.21.15.156:0/1726005912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c806ba90 0x7f71c80ca650 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009e50 tx=0x7f71c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a467a4b88f655b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:12.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.784+0000 7f71c77fe700 1 -- 172.21.15.156:0/1726005912 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c000e070 con 0x7f71c806ba90 2023-11-28T11:12:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.784+0000 7f71c77fe700 1 -- 172.21.15.156:0/1726005912 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c000b9c0 con 0x7f71c806ba90 2023-11-28T11:12:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.784+0000 7f71c77fe700 1 -- 172.21.15.156:0/1726005912 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c000bd70 con 0x7f71c806ba90 2023-11-28T11:12:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.784+0000 7f71ced5b700 1 -- 172.21.15.156:0/1726005912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c806ba90 msgr2=0x7f71c80ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.784+0000 7f71ced5b700 1 --2- 172.21.15.156:0/1726005912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c806ba90 0x7f71c80ca650 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009e50 tx=0x7f71c000b330 comp rx=0 tx=0).stop 2023-11-28T11:12:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.784+0000 7f71ced5b700 1 -- 172.21.15.156:0/1726005912 shutdown_connections 2023-11-28T11:12:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.784+0000 7f71ced5b700 1 --2- 172.21.15.156:0/1726005912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c806ba90 0x7f71c80ca650 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.784+0000 7f71ced5b700 1 --2- 172.21.15.156:0/1726005912 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71c815cef0 0x7f71c80af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.784+0000 7f71ced5b700 1 -- 172.21.15.156:0/1726005912 >> 172.21.15.156:0/1726005912 conn(0x7f71c80d0000 msgr2=0x7f71c80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.784+0000 7f71ced5b700 1 -- 172.21.15.156:0/1726005912 shutdown_connections 2023-11-28T11:12:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.784+0000 7f71ced5b700 1 -- 172.21.15.156:0/1726005912 wait complete. 2023-11-28T11:12:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.786+0000 7f71ced5b700 1 Processor -- start 2023-11-28T11:12:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.786+0000 7f71ced5b700 1 -- start start 2023-11-28T11:12:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.786+0000 7f71ced5b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71c806ba90 0x7f71c8151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.786+0000 7f71ced5b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c80cbdf0 0x7f71c80b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.786+0000 7f71ced5b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c815cef0 0x7f71c80b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.786+0000 7f71ced5b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c809a7a0 con 0x7f71c815cef0 2023-11-28T11:12:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.786+0000 7f71ced5b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80b4f10 con 0x7f71c80cbdf0 2023-11-28T11:12:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.786+0000 7f71ced5b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80b51f0 con 0x7f71c806ba90 2023-11-28T11:12:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71cd2f8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c815cef0 0x7f71c80b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71cd2f8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c815cef0 0x7f71c80b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36640/0 (socket says 172.21.15.156:36640) 2023-11-28T11:12:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71cd2f8700 1 -- 172.21.15.156:0/4108592217 learned_addr learned my addr 172.21.15.156:0/4108592217 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71c7fff700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c80cbdf0 0x7f71c80b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71ccaf7700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71c806ba90 0x7f71c8151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71cd2f8700 1 -- 172.21.15.156:0/4108592217 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71c806ba90 msgr2=0x7f71c8151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71cd2f8700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71c806ba90 0x7f71c8151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71cd2f8700 1 -- 172.21.15.156:0/4108592217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c80cbdf0 msgr2=0x7f71c80b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71cd2f8700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c80cbdf0 0x7f71c80b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71cd2f8700 1 -- 172.21.15.156:0/4108592217 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c0009b30 con 0x7f71c815cef0 2023-11-28T11:12:12.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71ccaf7700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71c806ba90 0x7f71c8151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:12.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71c7fff700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c80cbdf0 0x7f71c80b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:12.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71cd2f8700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c815cef0 0x7f71c80b0e20 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f71c806c7b0 tx=0x7f71c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:12.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71c57fa700 1 -- 172.21.15.156:0/4108592217 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c00099b0 con 0x7f71c815cef0 2023-11-28T11:12:12.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71c57fa700 1 -- 172.21.15.156:0/4108592217 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c00092e0 con 0x7f71c815cef0 2023-11-28T11:12:12.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71ced5b700 1 -- 172.21.15.156:0/4108592217 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c815dff0 con 0x7f71c815cef0 2023-11-28T11:12:12.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71c57fa700 1 -- 172.21.15.156:0/4108592217 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c001c640 con 0x7f71c815cef0 2023-11-28T11:12:12.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.787+0000 7f71ced5b700 1 -- 172.21.15.156:0/4108592217 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c815dd10 con 0x7f71c815cef0 2023-11-28T11:12:12.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.788+0000 7f71ced5b700 1 -- 172.21.15.156:0/4108592217 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c80cab90 con 0x7f71c815cef0 2023-11-28T11:12:12.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.788+0000 7f71c57fa700 1 -- 172.21.15.156:0/4108592217 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f71c002c070 con 0x7f71c815cef0 2023-11-28T11:12:12.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.789+0000 7f71c57fa700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71ac0682f0 0x7f71ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.789+0000 7f71c57fa700 1 -- 172.21.15.156:0/4108592217 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f71c00875f0 con 0x7f71c815cef0 2023-11-28T11:12:12.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.789+0000 7f71ccaf7700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71ac0682f0 0x7f71ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.789+0000 7f71ccaf7700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71ac0682f0 0x7f71ac06a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00ba70 tx=0x7f71bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:12.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.793+0000 7f71c57fa700 1 -- 172.21.15.156:0/4108592217 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f71c0054150 con 0x7f71c815cef0 2023-11-28T11:12:12.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.972+0000 7f71ced5b700 1 -- 172.21.15.156:0/4108592217 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f71c80b4da0 con 0x7f71ac0682f0 2023-11-28T11:12:12.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.973+0000 7f71c57fa700 1 -- 172.21.15.156:0/4108592217 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f71c80b4da0 con 0x7f71ac0682f0 2023-11-28T11:12:12.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.975+0000 7f71ced5b700 1 -- 172.21.15.156:0/4108592217 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71ac0682f0 msgr2=0x7f71ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.975+0000 7f71ced5b700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71ac0682f0 0x7f71ac06a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00ba70 tx=0x7f71bc008040 comp rx=0 tx=0).stop 2023-11-28T11:12:12.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.975+0000 7f71ced5b700 1 -- 172.21.15.156:0/4108592217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c815cef0 msgr2=0x7f71c80b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.975+0000 7f71ced5b700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c815cef0 0x7f71c80b0e20 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f71c806c7b0 tx=0x7f71c0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.975+0000 7f71ced5b700 1 -- 172.21.15.156:0/4108592217 shutdown_connections 2023-11-28T11:12:12.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.975+0000 7f71ced5b700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c815cef0 0x7f71c80b0e20 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.975+0000 7f71ced5b700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f71ac0682f0 0x7f71ac06a7a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.975+0000 7f71ced5b700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71c806ba90 0x7f71c8151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.975+0000 7f71ced5b700 1 --2- 172.21.15.156:0/4108592217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c80cbdf0 0x7f71c80b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.975+0000 7f71ced5b700 1 -- 172.21.15.156:0/4108592217 >> 172.21.15.156:0/4108592217 conn(0x7f71c80d0000 msgr2=0x7f71c806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:12.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.975+0000 7f71ced5b700 1 -- 172.21.15.156:0/4108592217 shutdown_connections 2023-11-28T11:12:12.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:12.975+0000 7f71ced5b700 1 -- 172.21.15.156:0/4108592217 wait complete. 2023-11-28T11:12:12.978 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 0 to deep-scrub 2023-11-28T11:12:12.988 INFO:tasks.ceph:Scrubbing osd.1 2023-11-28T11:12:12.988 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:13.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.111+0000 7f4faff5a700 1 Processor -- start 2023-11-28T11:12:13.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.112+0000 7f4faff5a700 1 -- start start 2023-11-28T11:12:13.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.112+0000 7f4faff5a700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4fa80af060 0x7f4fa8070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4faff5a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4fa806a980 0x7f4fa806d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4faff5a700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa80713c0 con 0x7f4fa80af430 2023-11-28T11:12:13.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4faff5a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa806dee0 con 0x7f4fa806a980 2023-11-28T11:12:13.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4faff5a700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa80716a0 con 0x7f4fa80af060 2023-11-28T11:12:13.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4fae4f7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4fa806a980 0x7f4fa806d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4fae4f7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4fa806a980 0x7f4fa806d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36648/0 (socket says 172.21.15.156:36648) 2023-11-28T11:12:13.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4fae4f7700 1 -- 172.21.15.156:0/204578825 learned_addr learned my addr 172.21.15.156:0/204578825 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:13.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4fae4f7700 1 -- 172.21.15.156:0/204578825 >> v2:172.21.15.186:3300/0 conn(0x7f4fa80af060 msgr2=0x7f4fa8070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:13.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4fae4f7700 1 --2- 172.21.15.156:0/204578825 >> v2:172.21.15.186:3300/0 conn(0x7f4fa80af060 0x7f4fa8070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4fae4f7700 1 -- 172.21.15.156:0/204578825 >> v1:172.21.15.186:6789/0 conn(0x7f4fa80af430 legacy=0x7f4fa806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:13.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4fae4f7700 1 -- 172.21.15.156:0/204578825 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa8071980 con 0x7f4fa806a980 2023-11-28T11:12:13.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4fae4f7700 1 --2- 172.21.15.156:0/204578825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa806a980 0x7f4fa806d9a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80725d0 tx=0x7f4fa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05b38ada861ecbd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4faccf4700 1 -- 172.21.15.156:0/204578825 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000e070 con 0x7f4fa806a980 2023-11-28T11:12:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4faccf4700 1 -- 172.21.15.156:0/204578825 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa0003a20 con 0x7f4fa806a980 2023-11-28T11:12:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.113+0000 7f4faccf4700 1 -- 172.21.15.156:0/204578825 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0003dd0 con 0x7f4fa806a980 2023-11-28T11:12:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.114+0000 7f4faff5a700 1 -- 172.21.15.156:0/204578825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa806a980 msgr2=0x7f4fa806d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.114+0000 7f4faff5a700 1 --2- 172.21.15.156:0/204578825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa806a980 0x7f4fa806d9a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80725d0 tx=0x7f4fa000b330 comp rx=0 tx=0).stop 2023-11-28T11:12:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.114+0000 7f4faff5a700 1 -- 172.21.15.156:0/204578825 shutdown_connections 2023-11-28T11:12:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.114+0000 7f4faff5a700 1 --2- 172.21.15.156:0/204578825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa806a980 0x7f4fa806d9a0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.114+0000 7f4faff5a700 1 --2- 172.21.15.156:0/204578825 >> v2:172.21.15.186:3300/0 conn(0x7f4fa80af060 0x7f4fa8070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.114+0000 7f4faff5a700 1 -- 172.21.15.156:0/204578825 >> 172.21.15.156:0/204578825 conn(0x7f4fa806b380 msgr2=0x7f4fa806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.114+0000 7f4faff5a700 1 -- 172.21.15.156:0/204578825 shutdown_connections 2023-11-28T11:12:13.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.114+0000 7f4faff5a700 1 -- 172.21.15.156:0/204578825 wait complete. 2023-11-28T11:12:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.115+0000 7f4faff5a700 1 Processor -- start 2023-11-28T11:12:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.115+0000 7f4faff5a700 1 -- start start 2023-11-28T11:12:13.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4faff5a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4fa806a980 0x7f4fa8156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4faff5a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa80af060 0x7f4fa80b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4faff5a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fa80af430 0x7f4fa80b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4faff5a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa815dd40 con 0x7f4fa80af060 2023-11-28T11:12:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4faff5a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa815e160 con 0x7f4fa80af430 2023-11-28T11:12:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4faff5a700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa815d0c0 con 0x7f4fa806a980 2023-11-28T11:12:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4fad4f5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa80af060 0x7f4fa80b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4fad4f5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa80af060 0x7f4fa80b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36652/0 (socket says 172.21.15.156:36652) 2023-11-28T11:12:13.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4fad4f5700 1 -- 172.21.15.156:0/1003295084 learned_addr learned my addr 172.21.15.156:0/1003295084 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:13.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4fae4f7700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fa80af430 0x7f4fa80b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4fadcf6700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4fa806a980 0x7f4fa8156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4fad4f5700 1 -- 172.21.15.156:0/1003295084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4fa806a980 msgr2=0x7f4fa8156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4fad4f5700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4fa806a980 0x7f4fa8156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4fad4f5700 1 -- 172.21.15.156:0/1003295084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fa80af430 msgr2=0x7f4fa80b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4fad4f5700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fa80af430 0x7f4fa80b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4fad4f5700 1 -- 172.21.15.156:0/1003295084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa0009b30 con 0x7f4fa80af060 2023-11-28T11:12:13.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4fadcf6700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4fa806a980 0x7f4fa8156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.116+0000 7f4fad4f5700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa80af060 0x7f4fa80b1db0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400c4a0 tx=0x7f4fa400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.117+0000 7f4f9e7fc700 1 -- 172.21.15.156:0/1003295084 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa40054e0 con 0x7f4fa80af060 2023-11-28T11:12:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.117+0000 7f4f9e7fc700 1 -- 172.21.15.156:0/1003295084 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa4003680 con 0x7f4fa80af060 2023-11-28T11:12:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.117+0000 7f4faff5a700 1 -- 172.21.15.156:0/1003295084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa8071f40 con 0x7f4fa80af060 2023-11-28T11:12:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.117+0000 7f4f9e7fc700 1 -- 172.21.15.156:0/1003295084 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4003a10 con 0x7f4fa80af060 2023-11-28T11:12:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.117+0000 7f4faff5a700 1 -- 172.21.15.156:0/1003295084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa8071c60 con 0x7f4fa80af060 2023-11-28T11:12:13.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.117+0000 7f4faff5a700 1 -- 172.21.15.156:0/1003295084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4fa80ca660 con 0x7f4fa80af060 2023-11-28T11:12:13.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.118+0000 7f4f9e7fc700 1 -- 172.21.15.156:0/1003295084 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4fa4010070 con 0x7f4fa80af060 2023-11-28T11:12:13.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.118+0000 7f4f9e7fc700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4f8c0682f0 0x7f4f8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.118+0000 7f4f9e7fc700 1 -- 172.21.15.156:0/1003295084 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4fa4085140 con 0x7f4fa80af060 2023-11-28T11:12:13.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.118+0000 7f4fadcf6700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4f8c0682f0 0x7f4f8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.119+0000 7f4f9e7fc700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x7f4f8c06e760 0x7f4f8c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.119+0000 7f4f9e7fc700 1 -- 172.21.15.156:0/1003295084 --> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f8c071240 con 0x7f4f8c06e760 2023-11-28T11:12:13.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.119+0000 7f4f9e7fc700 1 -- 172.21.15.156:0/1003295084 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fa4085580 con 0x7f4fa80af060 2023-11-28T11:12:13.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.119+0000 7f4fae4f7700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x7f4f8c06e760 0x7f4f8c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.119+0000 7f4fadcf6700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4f8c0682f0 0x7f4f8c06a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4f980099d0 tx=0x7f4f98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.119+0000 7f4fae4f7700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x7f4f8c06e760 0x7f4f8c070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.120+0000 7f4f9e7fc700 1 -- 172.21.15.156:0/1003295084 <== osd.1 v2:172.21.15.156:6820/103566 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4f8c071240 con 0x7f4f8c06e760 2023-11-28T11:12:13.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4faff5a700 1 -- 172.21.15.156:0/1003295084 --> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4fa815de80 con 0x7f4f8c06e760 2023-11-28T11:12:13.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4f9e7fc700 1 -- 172.21.15.156:0/1003295084 <== osd.1 v2:172.21.15.156:6820/103566 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4fa815de80 con 0x7f4f8c06e760 2023-11-28T11:12:13.135 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:12:13.135 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:13.136 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:12:13.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4faff5a700 1 -- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x7f4f8c06e760 msgr2=0x7f4f8c070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4faff5a700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:6820/103566,v1:172.21.15.156:6822/103566] conn(0x7f4f8c06e760 0x7f4f8c070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4faff5a700 1 -- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4f8c0682f0 msgr2=0x7f4f8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4faff5a700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f4f8c0682f0 0x7f4f8c06a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4f980099d0 tx=0x7f4f98008040 comp rx=0 tx=0).stop 2023-11-28T11:12:13.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4faff5a700 1 -- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa80af060 msgr2=0x7f4fa80b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4faff5a700 1 --2- 172.21.15.156:0/1003295084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa80af060 0x7f4fa80b1db0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400c4a0 tx=0x7f4fa400c7b0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4fae4f7700 1 -- 172.21.15.156:0/1003295084 reap_dead start 2023-11-28T11:12:13.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4faff5a700 1 -- 172.21.15.156:0/1003295084 shutdown_connections 2023-11-28T11:12:13.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4faff5a700 1 -- 172.21.15.156:0/1003295084 >> 172.21.15.156:0/1003295084 conn(0x7f4fa806b380 msgr2=0x7f4fa80b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4faff5a700 1 -- 172.21.15.156:0/1003295084 shutdown_connections 2023-11-28T11:12:13.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.134+0000 7f4faff5a700 1 -- 172.21.15.156:0/1003295084 wait complete. 2023-11-28T11:12:13.147 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-28T11:12:13.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.268+0000 7ff61d29d700 1 Processor -- start 2023-11-28T11:12:13.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.268+0000 7ff61d29d700 1 -- start start 2023-11-28T11:12:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff61d29d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6180d14a0 0x7ff6180b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff61d29d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff6180b2090 0x7ff6180b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff61d29d700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6180afd90 con 0x7ff61815cbb0 2023-11-28T11:12:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff61d29d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6180d3060 con 0x7ff6180b2090 2023-11-28T11:12:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff61d29d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180affe0 con 0x7ff6180d14a0 2023-11-28T11:12:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff6177fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff6180b2090 0x7ff6180b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff6177fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff6180b2090 0x7ff6180b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36660/0 (socket says 172.21.15.156:36660) 2023-11-28T11:12:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff6177fe700 1 -- 172.21.15.156:0/2132522915 learned_addr learned my addr 172.21.15.156:0/2132522915 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff6167fc700 1 --2- 172.21.15.156:0/2132522915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6180d14a0 0x7ff6180b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff6177fe700 1 -- 172.21.15.156:0/2132522915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6180d14a0 msgr2=0x7ff6180b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff6177fe700 1 --2- 172.21.15.156:0/2132522915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6180d14a0 0x7ff6180b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff6177fe700 1 -- 172.21.15.156:0/2132522915 >> v1:172.21.15.186:6789/0 conn(0x7ff61815cbb0 legacy=0x7ff6180d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff6177fe700 1 -- 172.21.15.156:0/2132522915 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6180b02c0 con 0x7ff6180b2090 2023-11-28T11:12:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.269+0000 7ff6177fe700 1 --2- 172.21.15.156:0/2132522915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6180b2090 0x7ff6180b2e00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff610009e50 tx=0x7ff61000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a5f7f2e5de6fdb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.270+0000 7ff615ffb700 1 -- 172.21.15.156:0/2132522915 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000e070 con 0x7ff6180b2090 2023-11-28T11:12:13.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.270+0000 7ff615ffb700 1 -- 172.21.15.156:0/2132522915 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff610003a20 con 0x7ff6180b2090 2023-11-28T11:12:13.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.270+0000 7ff615ffb700 1 -- 172.21.15.156:0/2132522915 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610003dd0 con 0x7ff6180b2090 2023-11-28T11:12:13.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.270+0000 7ff61d29d700 1 -- 172.21.15.156:0/2132522915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6180b2090 msgr2=0x7ff6180b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.270+0000 7ff61d29d700 1 --2- 172.21.15.156:0/2132522915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6180b2090 0x7ff6180b2e00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff610009e50 tx=0x7ff61000b330 comp rx=0 tx=0).stop 2023-11-28T11:12:13.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.270+0000 7ff61d29d700 1 -- 172.21.15.156:0/2132522915 shutdown_connections 2023-11-28T11:12:13.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.270+0000 7ff61d29d700 1 --2- 172.21.15.156:0/2132522915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6180b2090 0x7ff6180b2e00 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.270+0000 7ff61d29d700 1 --2- 172.21.15.156:0/2132522915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6180d14a0 0x7ff6180b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.270+0000 7ff61d29d700 1 -- 172.21.15.156:0/2132522915 >> 172.21.15.156:0/2132522915 conn(0x7ff61809c430 msgr2=0x7ff6180d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.270+0000 7ff61d29d700 1 -- 172.21.15.156:0/2132522915 shutdown_connections 2023-11-28T11:12:13.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.270+0000 7ff61d29d700 1 -- 172.21.15.156:0/2132522915 wait complete. 2023-11-28T11:12:13.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.271+0000 7ff61d29d700 1 Processor -- start 2023-11-28T11:12:13.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.272+0000 7ff61d29d700 1 -- start start 2023-11-28T11:12:13.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.272+0000 7ff61d29d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6180b2090 0x7ff6181e5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.272+0000 7ff61d29d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6180d14a0 0x7ff61806bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.272+0000 7ff61d29d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61815cbb0 0x7ff61806c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.272+0000 7ff61d29d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6181f6660 con 0x7ff6180d14a0 2023-11-28T11:12:13.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.272+0000 7ff61d29d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6181f59e0 con 0x7ff61815cbb0 2023-11-28T11:12:13.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.272+0000 7ff61d29d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6181f5cc0 con 0x7ff6180b2090 2023-11-28T11:12:13.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.272+0000 7ff6167fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6180d14a0 0x7ff61806bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.272+0000 7ff6167fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6180d14a0 0x7ff61806bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36666/0 (socket says 172.21.15.156:36666) 2023-11-28T11:12:13.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.272+0000 7ff6167fc700 1 -- 172.21.15.156:0/1499785287 learned_addr learned my addr 172.21.15.156:0/1499785287 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:13.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff616ffd700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6180b2090 0x7ff6181e5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff6167fc700 1 -- 172.21.15.156:0/1499785287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6180b2090 msgr2=0x7ff6181e5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff6167fc700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6180b2090 0x7ff6181e5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff6177fe700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61815cbb0 0x7ff61806c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff6167fc700 1 -- 172.21.15.156:0/1499785287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61815cbb0 msgr2=0x7ff61806c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff6167fc700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61815cbb0 0x7ff61806c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff6167fc700 1 -- 172.21.15.156:0/1499785287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610009b30 con 0x7ff6180d14a0 2023-11-28T11:12:13.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff6167fc700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6180d14a0 0x7ff61806bc70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c7f0 tx=0x7ff60800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff5ff7fe700 1 -- 172.21.15.156:0/1499785287 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608011800 con 0x7ff6180d14a0 2023-11-28T11:12:13.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff5ff7fe700 1 -- 172.21.15.156:0/1499785287 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff608011960 con 0x7ff6180d14a0 2023-11-28T11:12:13.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff5ff7fe700 1 -- 172.21.15.156:0/1499785287 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608011cf0 con 0x7ff6180d14a0 2023-11-28T11:12:13.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff61d29d700 1 -- 172.21.15.156:0/1499785287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6180b0880 con 0x7ff6180d14a0 2023-11-28T11:12:13.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.273+0000 7ff61d29d700 1 -- 172.21.15.156:0/1499785287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6180b05a0 con 0x7ff6180d14a0 2023-11-28T11:12:13.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.274+0000 7ff61d29d700 1 -- 172.21.15.156:0/1499785287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6180b0b60 con 0x7ff6180d14a0 2023-11-28T11:12:13.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.275+0000 7ff5ff7fe700 1 -- 172.21.15.156:0/1499785287 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff608021070 con 0x7ff6180d14a0 2023-11-28T11:12:13.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.275+0000 7ff5ff7fe700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6040682f0 0x7ff60406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.275+0000 7ff5ff7fe700 1 -- 172.21.15.156:0/1499785287 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff6080857c0 con 0x7ff6180d14a0 2023-11-28T11:12:13.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.275+0000 7ff616ffd700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6040682f0 0x7ff60406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.275+0000 7ff616ffd700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6040682f0 0x7ff60406a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff618157f70 tx=0x7ff60c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.279+0000 7ff5ff7fe700 1 -- 172.21.15.156:0/1499785287 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff608052320 con 0x7ff6180d14a0 2023-11-28T11:12:13.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.463+0000 7ff61d29d700 1 -- 172.21.15.156:0/1499785287 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6181f67a0 con 0x7ff6040682f0 2023-11-28T11:12:13.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.463+0000 7ff5ff7fe700 1 -- 172.21.15.156:0/1499785287 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff6181f67a0 con 0x7ff6040682f0 2023-11-28T11:12:13.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.465+0000 7ff61d29d700 1 -- 172.21.15.156:0/1499785287 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6040682f0 msgr2=0x7ff60406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.465+0000 7ff61d29d700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6040682f0 0x7ff60406a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff618157f70 tx=0x7ff60c008040 comp rx=0 tx=0).stop 2023-11-28T11:12:13.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.465+0000 7ff61d29d700 1 -- 172.21.15.156:0/1499785287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6180d14a0 msgr2=0x7ff61806bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.465+0000 7ff61d29d700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6180d14a0 0x7ff61806bc70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c7f0 tx=0x7ff60800cbb0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.465+0000 7ff61d29d700 1 -- 172.21.15.156:0/1499785287 shutdown_connections 2023-11-28T11:12:13.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.465+0000 7ff61d29d700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61815cbb0 0x7ff61806c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.465+0000 7ff61d29d700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff6040682f0 0x7ff60406a7a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.465+0000 7ff61d29d700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6180b2090 0x7ff6181e5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.465+0000 7ff61d29d700 1 --2- 172.21.15.156:0/1499785287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6180d14a0 0x7ff61806bc70 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.465+0000 7ff61d29d700 1 -- 172.21.15.156:0/1499785287 >> 172.21.15.156:0/1499785287 conn(0x7ff61809c430 msgr2=0x7ff6180b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.465+0000 7ff61d29d700 1 -- 172.21.15.156:0/1499785287 shutdown_connections 2023-11-28T11:12:13.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.465+0000 7ff61d29d700 1 -- 172.21.15.156:0/1499785287 wait complete. 2023-11-28T11:12:13.468 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 1 to deep-scrub 2023-11-28T11:12:13.479 INFO:tasks.ceph:Scrubbing osd.2 2023-11-28T11:12:13.479 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:13.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.602+0000 7fa1b8799700 1 Processor -- start 2023-11-28T11:12:13.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.602+0000 7fa1b8799700 1 -- start start 2023-11-28T11:12:13.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.603+0000 7fa1b8799700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa1b00af060 0x7fa1b0070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.603+0000 7fa1b8799700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1b006a980 0x7fa1b006d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.603+0000 7fa1b8799700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b00713c0 con 0x7fa1b00af430 2023-11-28T11:12:13.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.603+0000 7fa1b8799700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b006dee0 con 0x7fa1b00af060 2023-11-28T11:12:13.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.603+0000 7fa1b8799700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b00716a0 con 0x7fa1b006a980 2023-11-28T11:12:13.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b6535700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa1b00af060 0x7fa1b0070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b6d36700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1b006a980 0x7fa1b006d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b6d36700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1b006a980 0x7fa1b006d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:40538/0 (socket says 172.21.15.156:40538) 2023-11-28T11:12:13.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b6535700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa1b00af060 0x7fa1b0070d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:47686/0 (socket says 172.21.15.156:47686) 2023-11-28T11:12:13.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b6d36700 1 -- 172.21.15.156:0/3635237858 learned_addr learned my addr 172.21.15.156:0/3635237858 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:13.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b5d34700 1 --1- 172.21.15.156:0/3635237858 >> v1:172.21.15.186:6789/0 conn(0x7fa1b00af430 0x7fa1b006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:33678/0 (socket says 172.21.15.156:33678) 2023-11-28T11:12:13.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b6d36700 1 -- 172.21.15.156:0/3635237858 >> v2:172.21.15.186:3300/0 conn(0x7fa1b00af060 msgr2=0x7fa1b0070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:13.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b6d36700 1 --2- 172.21.15.156:0/3635237858 >> v2:172.21.15.186:3300/0 conn(0x7fa1b00af060 0x7fa1b0070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b6d36700 1 -- 172.21.15.156:0/3635237858 >> v1:172.21.15.186:6789/0 conn(0x7fa1b00af430 legacy=0x7fa1b006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b6d36700 1 -- 172.21.15.156:0/3635237858 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b0071980 con 0x7fa1b006a980 2023-11-28T11:12:13.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b6535700 1 --2- 172.21.15.156:0/3635237858 >> v2:172.21.15.186:3300/0 conn(0x7fa1b00af060 0x7fa1b0070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:13.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b6d36700 1 --2- 172.21.15.156:0/3635237858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1b006a980 0x7fa1b006d9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009e50 tx=0x7fa1a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e87655c1c041065 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b5533700 1 -- 172.21.15.156:0/3635237858 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800e070 con 0x7fa1b006a980 2023-11-28T11:12:13.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b5533700 1 -- 172.21.15.156:0/3635237858 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a800b910 con 0x7fa1b006a980 2023-11-28T11:12:13.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.604+0000 7fa1b5533700 1 -- 172.21.15.156:0/3635237858 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800bcc0 con 0x7fa1b006a980 2023-11-28T11:12:13.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.605+0000 7fa1b8799700 1 -- 172.21.15.156:0/3635237858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1b006a980 msgr2=0x7fa1b006d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.605+0000 7fa1b8799700 1 --2- 172.21.15.156:0/3635237858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1b006a980 0x7fa1b006d9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009e50 tx=0x7fa1a800b330 comp rx=0 tx=0).stop 2023-11-28T11:12:13.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.605+0000 7fa1b8799700 1 -- 172.21.15.156:0/3635237858 shutdown_connections 2023-11-28T11:12:13.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.605+0000 7fa1b8799700 1 --2- 172.21.15.156:0/3635237858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1b006a980 0x7fa1b006d9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.605+0000 7fa1b8799700 1 --2- 172.21.15.156:0/3635237858 >> v2:172.21.15.186:3300/0 conn(0x7fa1b00af060 0x7fa1b0070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.605+0000 7fa1b8799700 1 -- 172.21.15.156:0/3635237858 >> 172.21.15.156:0/3635237858 conn(0x7fa1b006b380 msgr2=0x7fa1b006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.605+0000 7fa1b8799700 1 -- 172.21.15.156:0/3635237858 shutdown_connections 2023-11-28T11:12:13.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.605+0000 7fa1b8799700 1 -- 172.21.15.156:0/3635237858 wait complete. 2023-11-28T11:12:13.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.606+0000 7fa1b8799700 1 Processor -- start 2023-11-28T11:12:13.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.606+0000 7fa1b8799700 1 -- start start 2023-11-28T11:12:13.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.607+0000 7fa1b8799700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1b006a980 0x7fa1b015da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.607+0000 7fa1b8799700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1b00af060 0x7fa1b00c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.607+0000 7fa1b8799700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1b00af430 0x7fa1b00c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.607+0000 7fa1b8799700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b0150140 con 0x7fa1b00af060 2023-11-28T11:12:13.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.607+0000 7fa1b8799700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b00afba0 con 0x7fa1b00af430 2023-11-28T11:12:13.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.607+0000 7fa1b8799700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b015e300 con 0x7fa1b006a980 2023-11-28T11:12:13.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.607+0000 7fa1b5d34700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1b00af060 0x7fa1b00c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.607+0000 7fa1b5d34700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1b00af060 0x7fa1b00c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36678/0 (socket says 172.21.15.156:36678) 2023-11-28T11:12:13.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.607+0000 7fa1b5d34700 1 -- 172.21.15.156:0/893440209 learned_addr learned my addr 172.21.15.156:0/893440209 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:13.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.607+0000 7fa1b6d36700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1b00af430 0x7fa1b00c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.607+0000 7fa1b6535700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1b006a980 0x7fa1b015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.608+0000 7fa1b5d34700 1 -- 172.21.15.156:0/893440209 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1b006a980 msgr2=0x7fa1b015da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.608+0000 7fa1b5d34700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1b006a980 0x7fa1b015da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.608+0000 7fa1b5d34700 1 -- 172.21.15.156:0/893440209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1b00af430 msgr2=0x7fa1b00c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.608+0000 7fa1b5d34700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1b00af430 0x7fa1b00c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.608+0000 7fa1b5d34700 1 -- 172.21.15.156:0/893440209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a00094a0 con 0x7fa1b00af060 2023-11-28T11:12:13.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.608+0000 7fa1b6d36700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1b00af430 0x7fa1b00c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:13.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.608+0000 7fa1b5d34700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1b00af060 0x7fa1b00c8120 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c4f0 tx=0x7fa1ac00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.608+0000 7fa1a6ffd700 1 -- 172.21.15.156:0/893440209 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac005cb0 con 0x7fa1b00af060 2023-11-28T11:12:13.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.608+0000 7fa1a6ffd700 1 -- 172.21.15.156:0/893440209 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ac005e10 con 0x7fa1b00af060 2023-11-28T11:12:13.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.608+0000 7fa1b8799700 1 -- 172.21.15.156:0/893440209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a8009b30 con 0x7fa1b00af060 2023-11-28T11:12:13.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.608+0000 7fa1a6ffd700 1 -- 172.21.15.156:0/893440209 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac0072d0 con 0x7fa1b00af060 2023-11-28T11:12:13.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.608+0000 7fa1b8799700 1 -- 172.21.15.156:0/893440209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b0071f40 con 0x7fa1b00af060 2023-11-28T11:12:13.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.609+0000 7fa1b8799700 1 -- 172.21.15.156:0/893440209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa1b0071c60 con 0x7fa1b00af060 2023-11-28T11:12:13.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.609+0000 7fa1a6ffd700 1 -- 172.21.15.156:0/893440209 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa1ac019030 con 0x7fa1b00af060 2023-11-28T11:12:13.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.610+0000 7fa1a6ffd700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa1940682f0 0x7fa19406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.610+0000 7fa1a6ffd700 1 -- 172.21.15.156:0/893440209 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa1ac085360 con 0x7fa1b00af060 2023-11-28T11:12:13.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.610+0000 7fa1b6535700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa1940682f0 0x7fa19406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.610+0000 7fa1a6ffd700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] conn(0x7fa19406e760 0x7fa194070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.610+0000 7fa1a6ffd700 1 -- 172.21.15.156:0/893440209 --> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa194071240 con 0x7fa19406e760 2023-11-28T11:12:13.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.610+0000 7fa1a6ffd700 1 -- 172.21.15.156:0/893440209 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1ac0857a0 con 0x7fa1b00af060 2023-11-28T11:12:13.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.610+0000 7fa1b6d36700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] conn(0x7fa19406e760 0x7fa194070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.610+0000 7fa1b6535700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa1940682f0 0x7fa19406a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009d70 tx=0x7fa1a0009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.610+0000 7fa1b6d36700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] conn(0x7fa19406e760 0x7fa194070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.611+0000 7fa1a6ffd700 1 -- 172.21.15.156:0/893440209 <== osd.2 v2:172.21.15.156:6800/103564 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa194071240 con 0x7fa19406e760 2023-11-28T11:12:13.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.625+0000 7fa1b8799700 1 -- 172.21.15.156:0/893440209 --> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa1b00ca660 con 0x7fa19406e760 2023-11-28T11:12:13.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.625+0000 7fa1a6ffd700 1 -- 172.21.15.156:0/893440209 <== osd.2 v2:172.21.15.156:6800/103564 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa1b00ca660 con 0x7fa19406e760 2023-11-28T11:12:13.627 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:12:13.627 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:13.627 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:12:13.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.626+0000 7fa1b8799700 1 -- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] conn(0x7fa19406e760 msgr2=0x7fa194070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.626+0000 7fa1b8799700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:6800/103564,v1:172.21.15.156:6801/103564] conn(0x7fa19406e760 0x7fa194070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.626+0000 7fa1b8799700 1 -- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa1940682f0 msgr2=0x7fa19406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.626+0000 7fa1b8799700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa1940682f0 0x7fa19406a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009d70 tx=0x7fa1a0009210 comp rx=0 tx=0).stop 2023-11-28T11:12:13.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.626+0000 7fa1b8799700 1 -- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1b00af060 msgr2=0x7fa1b00c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.626+0000 7fa1b8799700 1 --2- 172.21.15.156:0/893440209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1b00af060 0x7fa1b00c8120 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c4f0 tx=0x7fa1ac00c800 comp rx=0 tx=0).stop 2023-11-28T11:12:13.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.626+0000 7fa1b6d36700 1 -- 172.21.15.156:0/893440209 reap_dead start 2023-11-28T11:12:13.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.626+0000 7fa1b8799700 1 -- 172.21.15.156:0/893440209 shutdown_connections 2023-11-28T11:12:13.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.626+0000 7fa1b8799700 1 -- 172.21.15.156:0/893440209 >> 172.21.15.156:0/893440209 conn(0x7fa1b006b380 msgr2=0x7fa1b00d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.626+0000 7fa1b8799700 1 -- 172.21.15.156:0/893440209 shutdown_connections 2023-11-28T11:12:13.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.626+0000 7fa1b8799700 1 -- 172.21.15.156:0/893440209 wait complete. 2023-11-28T11:12:13.638 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-28T11:12:13.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.758+0000 7ff28bd19700 1 Processor -- start 2023-11-28T11:12:13.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.759+0000 7ff28bd19700 1 -- start start 2023-11-28T11:12:13.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.759+0000 7ff28bd19700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff284071040 0x7ff284071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.759+0000 7ff28bd19700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff28406dee0 0x7ff28406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.759+0000 7ff28bd19700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff284071950 con 0x7ff28406cdc0 2023-11-28T11:12:13.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.759+0000 7ff28bd19700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff284161550 con 0x7ff284071040 2023-11-28T11:12:13.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.759+0000 7ff28bd19700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28415d060 con 0x7ff28406dee0 2023-11-28T11:12:13.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff28a2b6700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff28406cdc0 0x7ff28406d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51568/0 (socket says 172.21.15.156:51568) 2023-11-28T11:12:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff28a2b6700 1 -- 172.21.15.156:0/2483753233 learned_addr learned my addr 172.21.15.156:0/2483753233 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff2892b4700 1 --2- 172.21.15.156:0/2483753233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff28406dee0 0x7ff28406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff289ab5700 1 --2- 172.21.15.156:0/2483753233 >> v2:172.21.15.186:3300/0 conn(0x7ff284071040 0x7ff284071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff288ab3700 1 -- 172.21.15.156:0/2483753233 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196526494 0 0) 0x7ff284071950 con 0x7ff28406cdc0 2023-11-28T11:12:13.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff288ab3700 1 -- 172.21.15.156:0/2483753233 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2700035b0 con 0x7ff28406cdc0 2023-11-28T11:12:13.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff2892b4700 1 -- 172.21.15.156:0/2483753233 >> v2:172.21.15.186:3300/0 conn(0x7ff284071040 msgr2=0x7ff284071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:13.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff2892b4700 1 --2- 172.21.15.156:0/2483753233 >> v2:172.21.15.186:3300/0 conn(0x7ff284071040 0x7ff284071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff2892b4700 1 -- 172.21.15.156:0/2483753233 >> v1:172.21.15.156:6789/0 conn(0x7ff28406cdc0 legacy=0x7ff28406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff2892b4700 1 -- 172.21.15.156:0/2483753233 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28415d340 con 0x7ff28406dee0 2023-11-28T11:12:13.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff2892b4700 1 --2- 172.21.15.156:0/2483753233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff28406dee0 0x7ff28406e330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff280009840 tx=0x7ff280009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e26e2d93d53505d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff288ab3700 1 -- 172.21.15.156:0/2483753233 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280011830 con 0x7ff28406dee0 2023-11-28T11:12:13.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff288ab3700 1 -- 172.21.15.156:0/2483753233 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff280011990 con 0x7ff28406dee0 2023-11-28T11:12:13.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.760+0000 7ff288ab3700 1 -- 172.21.15.156:0/2483753233 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280011d00 con 0x7ff28406dee0 2023-11-28T11:12:13.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.761+0000 7ff28bd19700 1 -- 172.21.15.156:0/2483753233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff28406dee0 msgr2=0x7ff28406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.761+0000 7ff28bd19700 1 --2- 172.21.15.156:0/2483753233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff28406dee0 0x7ff28406e330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff280009840 tx=0x7ff280009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:13.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.761+0000 7ff28bd19700 1 -- 172.21.15.156:0/2483753233 shutdown_connections 2023-11-28T11:12:13.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.761+0000 7ff28bd19700 1 --2- 172.21.15.156:0/2483753233 >> v2:172.21.15.186:3300/0 conn(0x7ff284071040 0x7ff284071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.761+0000 7ff28bd19700 1 --2- 172.21.15.156:0/2483753233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff28406dee0 0x7ff28406e330 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.761+0000 7ff28bd19700 1 -- 172.21.15.156:0/2483753233 >> 172.21.15.156:0/2483753233 conn(0x7ff2840d0000 msgr2=0x7ff2840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.761+0000 7ff28bd19700 1 -- 172.21.15.156:0/2483753233 shutdown_connections 2023-11-28T11:12:13.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.761+0000 7ff28bd19700 1 -- 172.21.15.156:0/2483753233 wait complete. 2023-11-28T11:12:13.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.762+0000 7ff28bd19700 1 Processor -- start 2023-11-28T11:12:13.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.762+0000 7ff28bd19700 1 -- start start 2023-11-28T11:12:13.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28bd19700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff28406cdc0 0x7ff28414a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28bd19700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff28406dee0 0x7ff2840b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28bd19700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff284071040 0x7ff2840cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28bd19700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28414a980 con 0x7ff284071040 2023-11-28T11:12:13.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28bd19700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2841f5bc0 con 0x7ff28406dee0 2023-11-28T11:12:13.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28bd19700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2841f5ea0 con 0x7ff28406cdc0 2023-11-28T11:12:13.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28a2b6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff284071040 0x7ff2840cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28a2b6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff284071040 0x7ff2840cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36696/0 (socket says 172.21.15.156:36696) 2023-11-28T11:12:13.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28a2b6700 1 -- 172.21.15.156:0/3499503273 learned_addr learned my addr 172.21.15.156:0/3499503273 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:13.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff2892b4700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff28406dee0 0x7ff2840b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff289ab5700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff28406cdc0 0x7ff28414a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28a2b6700 1 -- 172.21.15.156:0/3499503273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff28406cdc0 msgr2=0x7ff28414a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28a2b6700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff28406cdc0 0x7ff28414a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28a2b6700 1 -- 172.21.15.156:0/3499503273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff28406dee0 msgr2=0x7ff2840b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28a2b6700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff28406dee0 0x7ff2840b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff28a2b6700 1 -- 172.21.15.156:0/3499503273 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2800094a0 con 0x7ff284071040 2023-11-28T11:12:13.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.763+0000 7ff2892b4700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff28406dee0 0x7ff2840b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:13.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.764+0000 7ff28a2b6700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff284071040 0x7ff2840cf280 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00caa0 tx=0x7ff27c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.764+0000 7ff2767fc700 1 -- 172.21.15.156:0/3499503273 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00aa80 con 0x7ff284071040 2023-11-28T11:12:13.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.764+0000 7ff2767fc700 1 -- 172.21.15.156:0/3499503273 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27c00abe0 con 0x7ff284071040 2023-11-28T11:12:13.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.764+0000 7ff28bd19700 1 -- 172.21.15.156:0/3499503273 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28415d900 con 0x7ff284071040 2023-11-28T11:12:13.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.764+0000 7ff2767fc700 1 -- 172.21.15.156:0/3499503273 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c012400 con 0x7ff284071040 2023-11-28T11:12:13.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.764+0000 7ff28bd19700 1 -- 172.21.15.156:0/3499503273 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28415d620 con 0x7ff284071040 2023-11-28T11:12:13.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.765+0000 7ff28bd19700 1 -- 172.21.15.156:0/3499503273 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28406e870 con 0x7ff284071040 2023-11-28T11:12:13.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.765+0000 7ff2767fc700 1 -- 172.21.15.156:0/3499503273 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff27c019030 con 0x7ff284071040 2023-11-28T11:12:13.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.766+0000 7ff2767fc700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff268068340 0x7ff26806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.766+0000 7ff289ab5700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff268068340 0x7ff26806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.766+0000 7ff2767fc700 1 -- 172.21.15.156:0/3499503273 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff27c085520 con 0x7ff284071040 2023-11-28T11:12:13.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.766+0000 7ff289ab5700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff268068340 0x7ff26806a7f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff278009ad0 tx=0x7ff278008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.770+0000 7ff2767fc700 1 -- 172.21.15.156:0/3499503273 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff27c052140 con 0x7ff284071040 2023-11-28T11:12:13.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.950+0000 7ff28bd19700 1 -- 172.21.15.156:0/3499503273 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2841f6010 con 0x7ff268068340 2023-11-28T11:12:13.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.950+0000 7ff2767fc700 1 -- 172.21.15.156:0/3499503273 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff2841f6010 con 0x7ff268068340 2023-11-28T11:12:13.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.952+0000 7ff28bd19700 1 -- 172.21.15.156:0/3499503273 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff268068340 msgr2=0x7ff26806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.952+0000 7ff28bd19700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff268068340 0x7ff26806a7f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff278009ad0 tx=0x7ff278008040 comp rx=0 tx=0).stop 2023-11-28T11:12:13.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.952+0000 7ff28bd19700 1 -- 172.21.15.156:0/3499503273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff284071040 msgr2=0x7ff2840cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.952+0000 7ff28bd19700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff284071040 0x7ff2840cf280 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00caa0 tx=0x7ff27c00ce60 comp rx=0 tx=0).stop 2023-11-28T11:12:13.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.952+0000 7ff28bd19700 1 -- 172.21.15.156:0/3499503273 shutdown_connections 2023-11-28T11:12:13.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.952+0000 7ff28bd19700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff284071040 0x7ff2840cf280 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.952+0000 7ff28bd19700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff268068340 0x7ff26806a7f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.952+0000 7ff28bd19700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff28406cdc0 0x7ff28414a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.952+0000 7ff28bd19700 1 --2- 172.21.15.156:0/3499503273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff28406dee0 0x7ff2840b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.953+0000 7ff28bd19700 1 -- 172.21.15.156:0/3499503273 >> 172.21.15.156:0/3499503273 conn(0x7ff2840d0000 msgr2=0x7ff28406d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.953+0000 7ff28bd19700 1 -- 172.21.15.156:0/3499503273 shutdown_connections 2023-11-28T11:12:13.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:13.953+0000 7ff28bd19700 1 -- 172.21.15.156:0/3499503273 wait complete. 2023-11-28T11:12:13.958 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 2 to deep-scrub 2023-11-28T11:12:13.966 INFO:tasks.ceph:Scrubbing osd.3 2023-11-28T11:12:13.966 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:14.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.091+0000 7f2fd8ec4700 1 Processor -- start 2023-11-28T11:12:14.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.091+0000 7f2fd8ec4700 1 -- start start 2023-11-28T11:12:14.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd8ec4700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2fd40d0470 0x7f2fd4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd8ec4700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd40cc7b0 con 0x7f2fd41596b0 2023-11-28T11:12:14.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd8ec4700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd40cd610 con 0x7f2fd40cc260 2023-11-28T11:12:14.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd8ec4700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd4159a80 con 0x7f2fd40d0470 2023-11-28T11:12:14.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd259c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2fd41596b0 0x7f2fd40cfd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51578/0 (socket says 172.21.15.156:51578) 2023-11-28T11:12:14.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd259c700 1 -- 172.21.15.156:0/4125628397 learned_addr learned my addr 172.21.15.156:0/4125628397 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:14.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd1d9b700 1 --2- 172.21.15.156:0/4125628397 >> v2:172.21.15.186:3300/0 conn(0x7f2fd40d0470 0x7f2fd4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744740983 0 0) 0x7f2fd40cc7b0 con 0x7f2fd41596b0 2023-11-28T11:12:14.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fbc0035b0 con 0x7f2fd41596b0 2023-11-28T11:12:14.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858563053 0 0) 0x7f2fd40cd610 con 0x7f2fd40cc260 2023-11-28T11:12:14.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fd40cc7b0 con 0x7f2fd40cc260 2023-11-28T11:12:14.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4066750124 0 0) 0x7f2fbc0035b0 con 0x7f2fd41596b0 2023-11-28T11:12:14.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 >> v2:172.21.15.186:3300/0 conn(0x7f2fd40d0470 msgr2=0x7f2fd4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 --2- 172.21.15.156:0/4125628397 >> v2:172.21.15.186:3300/0 conn(0x7f2fd40d0470 0x7f2fd4071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 >> v1:172.21.15.186:6789/0 conn(0x7f2fd40cc260 legacy=0x7f2fd406cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd40cca00 con 0x7f2fd41596b0 2023-11-28T11:12:14.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f2fc4001670 con 0x7f2fd41596b0 2023-11-28T11:12:14.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 >> v1:172.21.15.156:6789/0 conn(0x7f2fd41596b0 legacy=0x7f2fd40cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 --2- 172.21.15.156:0/4125628397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fbc003b90 0x7f2fbc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 --2- 172.21.15.156:0/4125628397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fbc004850 0x7f2fbc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 --2- 172.21.15.156:0/4125628397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fbc005200 0x7f2fbc00dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4159a80 con 0x7f2fbc003b90 2023-11-28T11:12:14.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc00e270 con 0x7f2fbc005200 2023-11-28T11:12:14.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.092+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc00e430 con 0x7f2fbc004850 2023-11-28T11:12:14.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd259c700 1 --2- 172.21.15.156:0/4125628397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fbc003b90 0x7f2fbc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd2d9d700 1 --2- 172.21.15.156:0/4125628397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fbc005200 0x7f2fbc00dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd1d9b700 1 --2- 172.21.15.156:0/4125628397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fbc004850 0x7f2fbc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd259c700 1 -- 172.21.15.156:0/4125628397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fbc004850 msgr2=0x7f2fbc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd259c700 1 --2- 172.21.15.156:0/4125628397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fbc004850 0x7f2fbc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd259c700 1 -- 172.21.15.156:0/4125628397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fbc005200 msgr2=0x7f2fbc00dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd259c700 1 --2- 172.21.15.156:0/4125628397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fbc005200 0x7f2fbc00dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd259c700 1 -- 172.21.15.156:0/4125628397 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc0035b0 con 0x7f2fbc003b90 2023-11-28T11:12:14.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd2d9d700 1 -- 172.21.15.156:0/4125628397 reap_dead start 2023-11-28T11:12:14.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd259c700 1 --2- 172.21.15.156:0/4125628397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fbc003b90 0x7f2fbc003f80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c210 tx=0x7f2fbc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c5cdd697829d8a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400cd50 con 0x7f2fbc003b90 2023-11-28T11:12:14.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc4005960 con 0x7f2fbc003b90 2023-11-28T11:12:14.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.093+0000 7f2fd159a700 1 -- 172.21.15.156:0/4125628397 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4005d10 con 0x7f2fbc003b90 2023-11-28T11:12:14.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.094+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/4125628397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fbc003b90 msgr2=0x7f2fbc003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.094+0000 7f2fd8ec4700 1 --2- 172.21.15.156:0/4125628397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fbc003b90 0x7f2fbc003f80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c210 tx=0x7f2fbc005880 comp rx=0 tx=0).stop 2023-11-28T11:12:14.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.094+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/4125628397 shutdown_connections 2023-11-28T11:12:14.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.094+0000 7f2fd8ec4700 1 --2- 172.21.15.156:0/4125628397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fbc003b90 0x7f2fbc003f80 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.094+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/4125628397 >> 172.21.15.156:0/4125628397 conn(0x7f2fd406b380 msgr2=0x7f2fd406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.094+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/4125628397 shutdown_connections 2023-11-28T11:12:14.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.094+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/4125628397 wait complete. 2023-11-28T11:12:14.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.095+0000 7f2fd8ec4700 1 Processor -- start 2023-11-28T11:12:14.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.095+0000 7f2fd8ec4700 1 -- start start 2023-11-28T11:12:14.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd8ec4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fbc003b90 0x7f2fd40c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd8ec4700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd41544b0 0x7f2fd40c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd8ec4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd4154920 0x7f2fd4159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd8ec4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40c3e40 con 0x7f2fbc003b90 2023-11-28T11:12:14.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd8ec4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd41f6740 con 0x7f2fd4154920 2023-11-28T11:12:14.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd8ec4700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40c52d0 con 0x7f2fd41544b0 2023-11-28T11:12:14.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fbc003b90 0x7f2fd40c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fbc003b90 0x7f2fd40c3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36712/0 (socket says 172.21.15.156:36712) 2023-11-28T11:12:14.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd259c700 1 -- 172.21.15.156:0/273419507 learned_addr learned my addr 172.21.15.156:0/273419507 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:14.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd2d9d700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd4154920 0x7f2fd4159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd1d9b700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd41544b0 0x7f2fd40c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd259c700 1 -- 172.21.15.156:0/273419507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd41544b0 msgr2=0x7f2fd40c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd259c700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd41544b0 0x7f2fd40c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd259c700 1 -- 172.21.15.156:0/273419507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd4154920 msgr2=0x7f2fd4159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd259c700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd4154920 0x7f2fd4159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd259c700 1 -- 172.21.15.156:0/273419507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd40ccfc0 con 0x7f2fbc003b90 2023-11-28T11:12:14.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fd259c700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fbc003b90 0x7f2fd40c3900 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00e240 tx=0x7f2fc4001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.096+0000 7f2fc2ffd700 1 -- 172.21.15.156:0/273419507 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4004870 con 0x7f2fbc003b90 2023-11-28T11:12:14.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.097+0000 7f2fc2ffd700 1 -- 172.21.15.156:0/273419507 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc40049d0 con 0x7f2fbc003b90 2023-11-28T11:12:14.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.097+0000 7f2fc2ffd700 1 -- 172.21.15.156:0/273419507 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4004d60 con 0x7f2fbc003b90 2023-11-28T11:12:14.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.097+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/273419507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd40ccce0 con 0x7f2fbc003b90 2023-11-28T11:12:14.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.097+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/273419507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd40cd2a0 con 0x7f2fbc003b90 2023-11-28T11:12:14.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.097+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/273419507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2fd4068690 con 0x7f2fbc003b90 2023-11-28T11:12:14.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.098+0000 7f2fc2ffd700 1 -- 172.21.15.156:0/273419507 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2fc4010070 con 0x7f2fbc003b90 2023-11-28T11:12:14.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.098+0000 7f2fc2ffd700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2fb40682f0 0x7f2fb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.098+0000 7f2fc2ffd700 1 -- 172.21.15.156:0/273419507 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2fc4086330 con 0x7f2fbc003b90 2023-11-28T11:12:14.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.098+0000 7f2fd1d9b700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2fb40682f0 0x7f2fb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.099+0000 7f2fc2ffd700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] conn(0x7f2fb406e760 0x7f2fb4070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.099+0000 7f2fc2ffd700 1 -- 172.21.15.156:0/273419507 --> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2fb4071240 con 0x7f2fb406e760 2023-11-28T11:12:14.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.099+0000 7f2fc2ffd700 1 -- 172.21.15.156:0/273419507 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fc4086770 con 0x7f2fbc003b90 2023-11-28T11:12:14.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.099+0000 7f2fd2d9d700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] conn(0x7f2fb406e760 0x7f2fb4070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.099+0000 7f2fd1d9b700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2fb40682f0 0x7f2fb406a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40d0860 tx=0x7f2fc8009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.099+0000 7f2fd2d9d700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] conn(0x7f2fb406e760 0x7f2fb4070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.100+0000 7f2fc2ffd700 1 -- 172.21.15.156:0/273419507 <== osd.3 v2:172.21.15.156:6816/103567 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2fb4071240 con 0x7f2fb406e760 2023-11-28T11:12:14.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.114+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/273419507 --> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2fd40711f0 con 0x7f2fb406e760 2023-11-28T11:12:14.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.114+0000 7f2fc2ffd700 1 -- 172.21.15.156:0/273419507 <== osd.3 v2:172.21.15.156:6816/103567 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2fd40711f0 con 0x7f2fb406e760 2023-11-28T11:12:14.115 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:12:14.116 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:14.116 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:12:14.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.114+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] conn(0x7f2fb406e760 msgr2=0x7f2fb4070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.114+0000 7f2fd8ec4700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:6816/103567,v1:172.21.15.156:6817/103567] conn(0x7f2fb406e760 0x7f2fb4070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.114+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2fb40682f0 msgr2=0x7f2fb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.114+0000 7f2fd8ec4700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f2fb40682f0 0x7f2fb406a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40d0860 tx=0x7f2fc8009210 comp rx=0 tx=0).stop 2023-11-28T11:12:14.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.114+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fbc003b90 msgr2=0x7f2fd40c3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.114+0000 7f2fd8ec4700 1 --2- 172.21.15.156:0/273419507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fbc003b90 0x7f2fd40c3900 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00e240 tx=0x7f2fc4001980 comp rx=0 tx=0).stop 2023-11-28T11:12:14.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.114+0000 7f2fd2d9d700 1 -- 172.21.15.156:0/273419507 reap_dead start 2023-11-28T11:12:14.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.115+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/273419507 shutdown_connections 2023-11-28T11:12:14.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.115+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/273419507 >> 172.21.15.156:0/273419507 conn(0x7f2fd406b380 msgr2=0x7f2fd40cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.115+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/273419507 shutdown_connections 2023-11-28T11:12:14.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.115+0000 7f2fd8ec4700 1 -- 172.21.15.156:0/273419507 wait complete. 2023-11-28T11:12:14.127 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-28T11:12:14.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.247+0000 7f41fd8ef700 1 Processor -- start 2023-11-28T11:12:14.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.248+0000 7f41fd8ef700 1 -- start start 2023-11-28T11:12:14.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.248+0000 7f41fd8ef700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f41f8071040 0x7f41f8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.248+0000 7f41fd8ef700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f41f8160540 0x7f41f8160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.248+0000 7f41fd8ef700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f8071950 con 0x7f41f806dee0 2023-11-28T11:12:14.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.248+0000 7f41fd8ef700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f815d060 con 0x7f41f8071040 2023-11-28T11:12:14.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.248+0000 7f41fd8ef700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f815d340 con 0x7f41f8160540 2023-11-28T11:12:14.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.249+0000 7f41f6ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f41f8071040 0x7f41f8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.249+0000 7f41f6ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f41f8071040 0x7f41f8071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36722/0 (socket says 172.21.15.156:36722) 2023-11-28T11:12:14.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.249+0000 7f41f6ffd700 1 -- 172.21.15.156:0/3278293761 learned_addr learned my addr 172.21.15.156:0/3278293761 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:14.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.249+0000 7f41f77fe700 1 --2- 172.21.15.156:0/3278293761 >> v2:172.21.15.186:3300/0 conn(0x7f41f8160540 0x7f41f8160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.249+0000 7f41f6ffd700 1 -- 172.21.15.156:0/3278293761 >> v2:172.21.15.186:3300/0 conn(0x7f41f8160540 msgr2=0x7f41f8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.249+0000 7f41f6ffd700 1 --2- 172.21.15.156:0/3278293761 >> v2:172.21.15.186:3300/0 conn(0x7f41f8160540 0x7f41f8160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.249+0000 7f41f6ffd700 1 -- 172.21.15.156:0/3278293761 >> v1:172.21.15.186:6789/0 conn(0x7f41f806dee0 legacy=0x7f41f806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.249+0000 7f41f6ffd700 1 -- 172.21.15.156:0/3278293761 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f815d620 con 0x7f41f8071040 2023-11-28T11:12:14.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.249+0000 7f41f6ffd700 1 --2- 172.21.15.156:0/3278293761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f8071040 0x7f41f8071410 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f41f80c3a10 tx=0x7f41e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ae7269f4f9a67b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.249+0000 7f41f5ffb700 1 -- 172.21.15.156:0/3278293761 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0004230 con 0x7f41f8071040 2023-11-28T11:12:14.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.249+0000 7f41f5ffb700 1 -- 172.21.15.156:0/3278293761 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e0004390 con 0x7f41f8071040 2023-11-28T11:12:14.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.249+0000 7f41f5ffb700 1 -- 172.21.15.156:0/3278293761 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e00036a0 con 0x7f41f8071040 2023-11-28T11:12:14.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.250+0000 7f41fd8ef700 1 -- 172.21.15.156:0/3278293761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f8071040 msgr2=0x7f41f8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.250+0000 7f41fd8ef700 1 --2- 172.21.15.156:0/3278293761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f8071040 0x7f41f8071410 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f41f80c3a10 tx=0x7f41e0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.250+0000 7f41fd8ef700 1 -- 172.21.15.156:0/3278293761 shutdown_connections 2023-11-28T11:12:14.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.250+0000 7f41fd8ef700 1 --2- 172.21.15.156:0/3278293761 >> v2:172.21.15.186:3300/0 conn(0x7f41f8160540 0x7f41f8160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.250+0000 7f41fd8ef700 1 --2- 172.21.15.156:0/3278293761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f8071040 0x7f41f8071410 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.250+0000 7f41fd8ef700 1 -- 172.21.15.156:0/3278293761 >> 172.21.15.156:0/3278293761 conn(0x7f41f80d0000 msgr2=0x7f41f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.250+0000 7f41fd8ef700 1 -- 172.21.15.156:0/3278293761 shutdown_connections 2023-11-28T11:12:14.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.250+0000 7f41fd8ef700 1 -- 172.21.15.156:0/3278293761 wait complete. 2023-11-28T11:12:14.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.251+0000 7f41fd8ef700 1 Processor -- start 2023-11-28T11:12:14.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.252+0000 7f41fd8ef700 1 -- start start 2023-11-28T11:12:14.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.252+0000 7f41fd8ef700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41f806dee0 0x7f41f81e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.252+0000 7f41fd8ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f8071040 0x7f41f80b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.252+0000 7f41fd8ef700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f8160540 0x7f41f80b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.252+0000 7f41fd8ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f80b58d0 con 0x7f41f8071040 2023-11-28T11:12:14.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.252+0000 7f41fd8ef700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f80b4d00 con 0x7f41f8160540 2023-11-28T11:12:14.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.252+0000 7f41fd8ef700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f8142140 con 0x7f41f806dee0 2023-11-28T11:12:14.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41f67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f8071040 0x7f41f80b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41f67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f8071040 0x7f41f80b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36730/0 (socket says 172.21.15.156:36730) 2023-11-28T11:12:14.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41f67fc700 1 -- 172.21.15.156:0/4287967197 learned_addr learned my addr 172.21.15.156:0/4287967197 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:14.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41f6ffd700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41f806dee0 0x7f41f81e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41f77fe700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f8160540 0x7f41f80b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41f67fc700 1 -- 172.21.15.156:0/4287967197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41f806dee0 msgr2=0x7f41f81e9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41f67fc700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41f806dee0 0x7f41f81e9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41f67fc700 1 -- 172.21.15.156:0/4287967197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f8160540 msgr2=0x7f41f80b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41f67fc700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f8160540 0x7f41f80b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41f67fc700 1 -- 172.21.15.156:0/4287967197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e0009550 con 0x7f41f8071040 2023-11-28T11:12:14.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41f67fc700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f8071040 0x7f41f80b2e30 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f41e800b4d0 tx=0x7f41e800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41effff700 1 -- 172.21.15.156:0/4287967197 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e800bea0 con 0x7f41f8071040 2023-11-28T11:12:14.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41effff700 1 -- 172.21.15.156:0/4287967197 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e8011830 con 0x7f41f8071040 2023-11-28T11:12:14.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41fd8ef700 1 -- 172.21.15.156:0/4287967197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f815ccd0 con 0x7f41f8071040 2023-11-28T11:12:14.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41effff700 1 -- 172.21.15.156:0/4287967197 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8011be0 con 0x7f41f8071040 2023-11-28T11:12:14.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.253+0000 7f41fd8ef700 1 -- 172.21.15.156:0/4287967197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f815d900 con 0x7f41f8071040 2023-11-28T11:12:14.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.254+0000 7f41fd8ef700 1 -- 172.21.15.156:0/4287967197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f806a270 con 0x7f41f8071040 2023-11-28T11:12:14.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.254+0000 7f41effff700 1 -- 172.21.15.156:0/4287967197 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f41e800f050 con 0x7f41f8071040 2023-11-28T11:12:14.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.255+0000 7f41effff700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f41d80682f0 0x7f41d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.255+0000 7f41effff700 1 -- 172.21.15.156:0/4287967197 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f41e8085370 con 0x7f41f8071040 2023-11-28T11:12:14.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.255+0000 7f41f6ffd700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f41d80682f0 0x7f41d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.255+0000 7f41f6ffd700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f41d80682f0 0x7f41d806a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009840 tx=0x7f41e0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.259+0000 7f41effff700 1 -- 172.21.15.156:0/4287967197 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f41e8051e70 con 0x7f41f8071040 2023-11-28T11:12:14.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.442+0000 7f41fd8ef700 1 -- 172.21.15.156:0/4287967197 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f41f806ecd0 con 0x7f41d80682f0 2023-11-28T11:12:14.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.442+0000 7f41effff700 1 -- 172.21.15.156:0/4287967197 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f41f806ecd0 con 0x7f41d80682f0 2023-11-28T11:12:14.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.444+0000 7f41fd8ef700 1 -- 172.21.15.156:0/4287967197 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f41d80682f0 msgr2=0x7f41d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.444+0000 7f41fd8ef700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f41d80682f0 0x7f41d806a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009840 tx=0x7f41e0008030 comp rx=0 tx=0).stop 2023-11-28T11:12:14.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.444+0000 7f41fd8ef700 1 -- 172.21.15.156:0/4287967197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f8071040 msgr2=0x7f41f80b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.444+0000 7f41fd8ef700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f8071040 0x7f41f80b2e30 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f41e800b4d0 tx=0x7f41e800b7e0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.444+0000 7f41fd8ef700 1 -- 172.21.15.156:0/4287967197 shutdown_connections 2023-11-28T11:12:14.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.444+0000 7f41fd8ef700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f8160540 0x7f41f80b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.444+0000 7f41fd8ef700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f41d80682f0 0x7f41d806a7a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.444+0000 7f41fd8ef700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41f806dee0 0x7f41f81e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.444+0000 7f41fd8ef700 1 --2- 172.21.15.156:0/4287967197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f8071040 0x7f41f80b2e30 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.444+0000 7f41fd8ef700 1 -- 172.21.15.156:0/4287967197 >> 172.21.15.156:0/4287967197 conn(0x7f41f80d0000 msgr2=0x7f41f815f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.444+0000 7f41fd8ef700 1 -- 172.21.15.156:0/4287967197 shutdown_connections 2023-11-28T11:12:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.444+0000 7f41fd8ef700 1 -- 172.21.15.156:0/4287967197 wait complete. 2023-11-28T11:12:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 3 to deep-scrub 2023-11-28T11:12:14.457 INFO:tasks.ceph:Scrubbing osd.4 2023-11-28T11:12:14.457 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:14.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.580+0000 7f3ab8a5d700 1 Processor -- start 2023-11-28T11:12:14.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.580+0000 7f3ab8a5d700 1 -- start start 2023-11-28T11:12:14.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab8a5d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3ab406a980 0x7f3ab4070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab8a5d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ab40af430 0x7f3ab406d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab8a5d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab40713c0 con 0x7f3ab40af060 2023-11-28T11:12:14.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab8a5d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab406dee0 con 0x7f3ab406a980 2023-11-28T11:12:14.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab8a5d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab40716a0 con 0x7f3ab40af430 2023-11-28T11:12:14.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab259c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3ab40af060 0x7f3ab406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51614/0 (socket says 172.21.15.156:51614) 2023-11-28T11:12:14.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab259c700 1 -- 172.21.15.156:0/3344606790 learned_addr learned my addr 172.21.15.156:0/3344606790 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:14.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab1d9b700 1 --2- 172.21.15.156:0/3344606790 >> v2:172.21.15.186:3300/0 conn(0x7f3ab406a980 0x7f3ab4070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab2d9d700 1 --2- 172.21.15.156:0/3344606790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ab40af430 0x7f3ab406d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313319431 0 0) 0x7f3ab40713c0 con 0x7f3ab40af060 2023-11-28T11:12:14.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a9c0035b0 con 0x7f3ab40af060 2023-11-28T11:12:14.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3905524992 0 0) 0x7f3a9c0035b0 con 0x7f3ab40af060 2023-11-28T11:12:14.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ab40af430 msgr2=0x7f3ab406d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab159a700 1 --2- 172.21.15.156:0/3344606790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ab40af430 0x7f3ab406d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 >> v2:172.21.15.186:3300/0 conn(0x7f3ab406a980 msgr2=0x7f3ab4070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.581+0000 7f3ab159a700 1 --2- 172.21.15.156:0/3344606790 >> v2:172.21.15.186:3300/0 conn(0x7f3ab406a980 0x7f3ab4070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab4071980 con 0x7f3ab40af060 2023-11-28T11:12:14.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2785687173 0 0) 0x7f3aa8001670 con 0x7f3ab40af060 2023-11-28T11:12:14.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 >> v1:172.21.15.156:6789/0 conn(0x7f3ab40af060 legacy=0x7f3ab406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab159a700 1 --2- 172.21.15.156:0/3344606790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ab40af430 0x7f3a9c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab159a700 1 --2- 172.21.15.156:0/3344606790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a9c003f80 0x7f3a9c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab159a700 1 --2- 172.21.15.156:0/3344606790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a9c004d90 0x7f3a9c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab406dee0 con 0x7f3a9c004d90 2023-11-28T11:12:14.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab40716a0 con 0x7f3a9c003f80 2023-11-28T11:12:14.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c00ba40 con 0x7f3ab40af430 2023-11-28T11:12:14.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab2d9d700 1 --2- 172.21.15.156:0/3344606790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a9c004d90 0x7f3a9c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab259c700 1 --2- 172.21.15.156:0/3344606790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ab40af430 0x7f3a9c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab1d9b700 1 --2- 172.21.15.156:0/3344606790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a9c003f80 0x7f3a9c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab2d9d700 1 -- 172.21.15.156:0/3344606790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ab40af430 msgr2=0x7f3a9c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab2d9d700 1 --2- 172.21.15.156:0/3344606790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ab40af430 0x7f3a9c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab2d9d700 1 -- 172.21.15.156:0/3344606790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a9c003f80 msgr2=0x7f3a9c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab2d9d700 1 --2- 172.21.15.156:0/3344606790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a9c003f80 0x7f3a9c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab2d9d700 1 -- 172.21.15.156:0/3344606790 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0035b0 con 0x7f3a9c004d90 2023-11-28T11:12:14.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab2d9d700 1 --2- 172.21.15.156:0/3344606790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a9c004d90 0x7f3a9c005200 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009fc0 tx=0x7f3a9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0ccb337bafe0ed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00ebd0 con 0x7f3a9c004d90 2023-11-28T11:12:14.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac00f040 con 0x7f3a9c004d90 2023-11-28T11:12:14.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.582+0000 7f3ab159a700 1 -- 172.21.15.156:0/3344606790 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac014480 con 0x7f3a9c004d90 2023-11-28T11:12:14.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.583+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/3344606790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a9c004d90 msgr2=0x7f3a9c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.583+0000 7f3ab8a5d700 1 --2- 172.21.15.156:0/3344606790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a9c004d90 0x7f3a9c005200 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009fc0 tx=0x7f3a9c005880 comp rx=0 tx=0).stop 2023-11-28T11:12:14.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.583+0000 7f3ab2d9d700 1 -- 172.21.15.156:0/3344606790 reap_dead start 2023-11-28T11:12:14.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.583+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/3344606790 shutdown_connections 2023-11-28T11:12:14.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.583+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/3344606790 >> 172.21.15.156:0/3344606790 conn(0x7f3ab406b380 msgr2=0x7f3ab406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.583+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/3344606790 shutdown_connections 2023-11-28T11:12:14.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.583+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/3344606790 wait complete. 2023-11-28T11:12:14.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.584+0000 7f3ab8a5d700 1 Processor -- start 2023-11-28T11:12:14.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.584+0000 7f3ab8a5d700 1 -- start start 2023-11-28T11:12:14.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab8a5d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a9c004d90 0x7f3ab415c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab8a5d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ab40b2520 0x7f3ab40af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab8a5d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40d00b0 0x7f3ab40d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab8a5d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab415e4f0 con 0x7f3ab40d00b0 2023-11-28T11:12:14.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab8a5d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab415e2f0 con 0x7f3a9c004d90 2023-11-28T11:12:14.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab8a5d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab40d0a60 con 0x7f3ab40b2520 2023-11-28T11:12:14.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40d00b0 0x7f3ab40d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40d00b0 0x7f3ab40d0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36750/0 (socket says 172.21.15.156:36750) 2023-11-28T11:12:14.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab2d9d700 1 -- 172.21.15.156:0/2424086553 learned_addr learned my addr 172.21.15.156:0/2424086553 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:14.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab259c700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a9c004d90 0x7f3ab415c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab1d9b700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ab40b2520 0x7f3ab40af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab2d9d700 1 -- 172.21.15.156:0/2424086553 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ab40b2520 msgr2=0x7f3ab40af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab2d9d700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ab40b2520 0x7f3ab40af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab2d9d700 1 -- 172.21.15.156:0/2424086553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a9c004d90 msgr2=0x7f3ab415c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab2d9d700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a9c004d90 0x7f3ab415c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.585+0000 7f3ab2d9d700 1 -- 172.21.15.156:0/2424086553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab4071f40 con 0x7f3ab40d00b0 2023-11-28T11:12:14.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.586+0000 7f3ab2d9d700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40d00b0 0x7f3ab40d0520 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3aac016000 tx=0x7f3aac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.586+0000 7f3aa2ffd700 1 -- 172.21.15.156:0/2424086553 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac007000 con 0x7f3ab40d00b0 2023-11-28T11:12:14.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.586+0000 7f3aa2ffd700 1 -- 172.21.15.156:0/2424086553 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac022070 con 0x7f3ab40d00b0 2023-11-28T11:12:14.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.586+0000 7f3aa2ffd700 1 -- 172.21.15.156:0/2424086553 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac029480 con 0x7f3ab40d00b0 2023-11-28T11:12:14.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.586+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/2424086553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab4071c60 con 0x7f3ab40d00b0 2023-11-28T11:12:14.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.586+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/2424086553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab406d9a0 con 0x7f3ab40d00b0 2023-11-28T11:12:14.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.587+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/2424086553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ab4068690 con 0x7f3ab40d00b0 2023-11-28T11:12:14.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.587+0000 7f3aa2ffd700 1 -- 172.21.15.156:0/2424086553 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3aac01e430 con 0x7f3ab40d00b0 2023-11-28T11:12:14.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.588+0000 7f3aa2ffd700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3a940682f0 0x7f3a9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.588+0000 7f3aa2ffd700 1 -- 172.21.15.156:0/2424086553 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3aac024070 con 0x7f3ab40d00b0 2023-11-28T11:12:14.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.588+0000 7f3ab259c700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3a940682f0 0x7f3a9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.588+0000 7f3aa2ffd700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] conn(0x7f3a9406e760 0x7f3a94070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.588+0000 7f3aa2ffd700 1 -- 172.21.15.156:0/2424086553 --> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a94071240 con 0x7f3a9406e760 2023-11-28T11:12:14.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.588+0000 7f3aa2ffd700 1 -- 172.21.15.156:0/2424086553 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3aac0882a0 con 0x7f3ab40d00b0 2023-11-28T11:12:14.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.588+0000 7f3ab259c700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3a940682f0 0x7f3a9406a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8001050 tx=0x7f3aa8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.588+0000 7f3ab1d9b700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] conn(0x7f3a9406e760 0x7f3a94070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.589+0000 7f3ab1d9b700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] conn(0x7f3a9406e760 0x7f3a94070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.589+0000 7f3aa2ffd700 1 -- 172.21.15.156:0/2424086553 <== osd.4 v2:172.21.15.186:6824/103781 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3a94071240 con 0x7f3a9406e760 2023-11-28T11:12:14.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.603+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/2424086553 --> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3ab415e630 con 0x7f3a9406e760 2023-11-28T11:12:14.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.604+0000 7f3aa2ffd700 1 -- 172.21.15.156:0/2424086553 <== osd.4 v2:172.21.15.186:6824/103781 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3ab415e630 con 0x7f3a9406e760 2023-11-28T11:12:14.605 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:12:14.605 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:14.605 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:12:14.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.604+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/2424086553 >> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] conn(0x7f3a9406e760 msgr2=0x7f3a94070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.604+0000 7f3ab8a5d700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.186:6824/103781,v1:172.21.15.186:6825/103781] conn(0x7f3a9406e760 0x7f3a94070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.604+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/2424086553 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3a940682f0 msgr2=0x7f3a9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.604+0000 7f3ab8a5d700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f3a940682f0 0x7f3a9406a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8001050 tx=0x7f3aa8009040 comp rx=0 tx=0).stop 2023-11-28T11:12:14.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.604+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/2424086553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40d00b0 msgr2=0x7f3ab40d0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.604+0000 7f3ab8a5d700 1 --2- 172.21.15.156:0/2424086553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40d00b0 0x7f3ab40d0520 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3aac016000 tx=0x7f3aac009750 comp rx=0 tx=0).stop 2023-11-28T11:12:14.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.604+0000 7f3ab2d9d700 1 -- 172.21.15.156:0/2424086553 reap_dead start 2023-11-28T11:12:14.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.604+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/2424086553 shutdown_connections 2023-11-28T11:12:14.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.604+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/2424086553 >> 172.21.15.156:0/2424086553 conn(0x7f3ab406b380 msgr2=0x7f3ab40ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.604+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/2424086553 shutdown_connections 2023-11-28T11:12:14.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.604+0000 7f3ab8a5d700 1 -- 172.21.15.156:0/2424086553 wait complete. 2023-11-28T11:12:14.617 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-28T11:12:14.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.738+0000 7f81d70aa700 1 Processor -- start 2023-11-28T11:12:14.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.738+0000 7f81d70aa700 1 -- start start 2023-11-28T11:12:14.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81d70aa700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f81d01658c0 0x7f81d0167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81d70aa700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d015d060 con 0x7f81d006dee0 2023-11-28T11:12:14.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81d70aa700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d015d340 con 0x7f81d0071040 2023-11-28T11:12:14.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81d70aa700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d0161550 con 0x7f81d01658c0 2023-11-28T11:12:14.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81d5647700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f81d01658c0 0x7f81d0167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81cffff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f81d006dee0 0x7f81d006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51638/0 (socket says 172.21.15.156:51638) 2023-11-28T11:12:14.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81d5647700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f81d01658c0 0x7f81d0167ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36762/0 (socket says 172.21.15.156:36762) 2023-11-28T11:12:14.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81cffff700 1 -- 172.21.15.156:0/961679536 learned_addr learned my addr 172.21.15.156:0/961679536 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:14.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81d5647700 1 -- 172.21.15.156:0/961679536 >> v1:172.21.15.186:6789/0 conn(0x7f81d0071040 legacy=0x7f81d0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81d5647700 1 -- 172.21.15.156:0/961679536 >> v1:172.21.15.156:6789/0 conn(0x7f81d006dee0 legacy=0x7f81d006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81d5647700 1 -- 172.21.15.156:0/961679536 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d015d620 con 0x7f81d01658c0 2023-11-28T11:12:14.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81cf7fe700 1 -- 172.21.15.156:0/961679536 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336961227 0 0) 0x7f81d015d060 con 0x7f81d006dee0 2023-11-28T11:12:14.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81d5647700 1 --2- 172.21.15.156:0/961679536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d01658c0 0x7f81d0167ca0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009d90 tx=0x7f81c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c87a179e6d7e33f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.739+0000 7f81cf7fe700 1 -- 172.21.15.156:0/961679536 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c800e070 con 0x7f81d01658c0 2023-11-28T11:12:14.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.740+0000 7f81cf7fe700 1 -- 172.21.15.156:0/961679536 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c800bbd0 con 0x7f81d01658c0 2023-11-28T11:12:14.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.740+0000 7f81cf7fe700 1 -- 172.21.15.156:0/961679536 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8012430 con 0x7f81d01658c0 2023-11-28T11:12:14.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.740+0000 7f81d70aa700 1 -- 172.21.15.156:0/961679536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d01658c0 msgr2=0x7f81d0167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.740+0000 7f81d70aa700 1 --2- 172.21.15.156:0/961679536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d01658c0 0x7f81d0167ca0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009d90 tx=0x7f81c800b330 comp rx=0 tx=0).stop 2023-11-28T11:12:14.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.740+0000 7f81d70aa700 1 -- 172.21.15.156:0/961679536 shutdown_connections 2023-11-28T11:12:14.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.740+0000 7f81d70aa700 1 --2- 172.21.15.156:0/961679536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d01658c0 0x7f81d0167ca0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.740+0000 7f81d70aa700 1 -- 172.21.15.156:0/961679536 >> 172.21.15.156:0/961679536 conn(0x7f81d00d0000 msgr2=0x7f81d00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.740+0000 7f81d70aa700 1 -- 172.21.15.156:0/961679536 shutdown_connections 2023-11-28T11:12:14.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.740+0000 7f81d70aa700 1 -- 172.21.15.156:0/961679536 wait complete. 2023-11-28T11:12:14.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.742+0000 7f81d70aa700 1 Processor -- start 2023-11-28T11:12:14.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.742+0000 7f81d70aa700 1 -- start start 2023-11-28T11:12:14.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d70aa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81d006dee0 0x7f81d00c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d70aa700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81d0071040 0x7f81d00acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d70aa700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d01658c0 0x7f81d0156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d70aa700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00ae090 con 0x7f81d01658c0 2023-11-28T11:12:14.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d70aa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d0157560 con 0x7f81d006dee0 2023-11-28T11:12:14.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d70aa700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00ad480 con 0x7f81d0071040 2023-11-28T11:12:14.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d5647700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d01658c0 0x7f81d0156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d5647700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d01658c0 0x7f81d0156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36768/0 (socket says 172.21.15.156:36768) 2023-11-28T11:12:14.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d5647700 1 -- 172.21.15.156:0/1819352786 learned_addr learned my addr 172.21.15.156:0/1819352786 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:14.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81cffff700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81d0071040 0x7f81d00acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d4e46700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81d006dee0 0x7f81d00c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d5647700 1 -- 172.21.15.156:0/1819352786 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81d0071040 msgr2=0x7f81d00acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d5647700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81d0071040 0x7f81d00acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d5647700 1 -- 172.21.15.156:0/1819352786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81d006dee0 msgr2=0x7f81d00c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d5647700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81d006dee0 0x7f81d00c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d5647700 1 -- 172.21.15.156:0/1819352786 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c8009a70 con 0x7f81d01658c0 2023-11-28T11:12:14.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d5647700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d01658c0 0x7f81d0156aa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f81c8014040 tx=0x7f81c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81cd7fa700 1 -- 172.21.15.156:0/1819352786 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c801b810 con 0x7f81d01658c0 2023-11-28T11:12:14.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81cd7fa700 1 -- 172.21.15.156:0/1819352786 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c8020070 con 0x7f81d01658c0 2023-11-28T11:12:14.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81d70aa700 1 -- 172.21.15.156:0/1819352786 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d015ccd0 con 0x7f81d01658c0 2023-11-28T11:12:14.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.743+0000 7f81cd7fa700 1 -- 172.21.15.156:0/1819352786 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c801bbb0 con 0x7f81d01658c0 2023-11-28T11:12:14.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.744+0000 7f81d70aa700 1 -- 172.21.15.156:0/1819352786 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d015d900 con 0x7f81d01658c0 2023-11-28T11:12:14.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.744+0000 7f81d70aa700 1 -- 172.21.15.156:0/1819352786 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d006a270 con 0x7f81d01658c0 2023-11-28T11:12:14.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.745+0000 7f81cd7fa700 1 -- 172.21.15.156:0/1819352786 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f81c801bd10 con 0x7f81d01658c0 2023-11-28T11:12:14.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.745+0000 7f81cd7fa700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f81b40682f0 0x7f81b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.745+0000 7f81cd7fa700 1 -- 172.21.15.156:0/1819352786 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f81c8022070 con 0x7f81d01658c0 2023-11-28T11:12:14.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.745+0000 7f81d4e46700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f81b40682f0 0x7f81b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.746+0000 7f81d4e46700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f81b40682f0 0x7f81b406a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f81c4000f60 tx=0x7f81c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.749+0000 7f81cd7fa700 1 -- 172.21.15.156:0/1819352786 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f81c80549c0 con 0x7f81d01658c0 2023-11-28T11:12:14.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.931+0000 7f81d70aa700 1 -- 172.21.15.156:0/1819352786 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f81d00ae1d0 con 0x7f81b40682f0 2023-11-28T11:12:14.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.931+0000 7f81cd7fa700 1 -- 172.21.15.156:0/1819352786 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f81d00ae1d0 con 0x7f81b40682f0 2023-11-28T11:12:14.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.933+0000 7f81d70aa700 1 -- 172.21.15.156:0/1819352786 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f81b40682f0 msgr2=0x7f81b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.933+0000 7f81d70aa700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f81b40682f0 0x7f81b406a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f81c4000f60 tx=0x7f81c4009040 comp rx=0 tx=0).stop 2023-11-28T11:12:14.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.933+0000 7f81d70aa700 1 -- 172.21.15.156:0/1819352786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d01658c0 msgr2=0x7f81d0156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.933+0000 7f81d70aa700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d01658c0 0x7f81d0156aa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f81c8014040 tx=0x7f81c8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.933+0000 7f81d70aa700 1 -- 172.21.15.156:0/1819352786 shutdown_connections 2023-11-28T11:12:14.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.933+0000 7f81d70aa700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d01658c0 0x7f81d0156aa0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.933+0000 7f81d70aa700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81d006dee0 0x7f81d00c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.933+0000 7f81d70aa700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f81b40682f0 0x7f81b406a7a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.933+0000 7f81d70aa700 1 --2- 172.21.15.156:0/1819352786 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81d0071040 0x7f81d00acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.933+0000 7f81d70aa700 1 -- 172.21.15.156:0/1819352786 >> 172.21.15.156:0/1819352786 conn(0x7f81d00d0000 msgr2=0x7f81d00cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.933+0000 7f81d70aa700 1 -- 172.21.15.156:0/1819352786 shutdown_connections 2023-11-28T11:12:14.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:14.933+0000 7f81d70aa700 1 -- 172.21.15.156:0/1819352786 wait complete. 2023-11-28T11:12:14.937 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 4 to deep-scrub 2023-11-28T11:12:14.946 INFO:tasks.ceph:Scrubbing osd.5 2023-11-28T11:12:14.947 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:15.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.071+0000 7f85345d4700 1 Processor -- start 2023-11-28T11:12:15.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.071+0000 7f85345d4700 1 -- start start 2023-11-28T11:12:15.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.072+0000 7f85345d4700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f852c1596b0 0x7f852c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.072+0000 7f85345d4700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f852c0d0470 0x7f852c06cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.072+0000 7f85345d4700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c0cc7b0 con 0x7f852c0cc260 2023-11-28T11:12:15.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.072+0000 7f85345d4700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852c0cd610 con 0x7f852c1596b0 2023-11-28T11:12:15.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.072+0000 7f85345d4700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c159a80 con 0x7f852c0d0470 2023-11-28T11:12:15.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.072+0000 7f8531b6f700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f852c0cc260 0x7f852c0cfd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51652/0 (socket says 172.21.15.156:51652) 2023-11-28T11:12:15.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.072+0000 7f8531b6f700 1 -- 172.21.15.156:0/3789777782 learned_addr learned my addr 172.21.15.156:0/3789777782 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:15.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.072+0000 7f8532370700 1 --2- 172.21.15.156:0/3789777782 >> v2:172.21.15.186:3300/0 conn(0x7f852c1596b0 0x7f852c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.072+0000 7f853136e700 1 -- 172.21.15.156:0/3789777782 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476052488 0 0) 0x7f852c0cc7b0 con 0x7f852c0cc260 2023-11-28T11:12:15.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.072+0000 7f853136e700 1 -- 172.21.15.156:0/3789777782 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85180035b0 con 0x7f852c0cc260 2023-11-28T11:12:15.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.072+0000 7f8532b71700 1 --2- 172.21.15.156:0/3789777782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f852c0d0470 0x7f852c06cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.073+0000 7f8532370700 1 -- 172.21.15.156:0/3789777782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f852c0d0470 msgr2=0x7f852c06cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:15.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.073+0000 7f8532370700 1 --2- 172.21.15.156:0/3789777782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f852c0d0470 0x7f852c06cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.073+0000 7f8532370700 1 -- 172.21.15.156:0/3789777782 >> v1:172.21.15.156:6789/0 conn(0x7f852c0cc260 legacy=0x7f852c0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.073+0000 7f8532370700 1 -- 172.21.15.156:0/3789777782 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0cca00 con 0x7f852c1596b0 2023-11-28T11:12:15.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.073+0000 7f8532b71700 1 --2- 172.21.15.156:0/3789777782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f852c0d0470 0x7f852c06cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:15.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.073+0000 7f8532370700 1 --2- 172.21.15.156:0/3789777782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f852c1596b0 0x7f852c071410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f851c009840 tx=0x7f851c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0437389f7a1c5e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.073+0000 7f853136e700 1 -- 172.21.15.156:0/3789777782 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c011830 con 0x7f852c1596b0 2023-11-28T11:12:15.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.073+0000 7f853136e700 1 -- 172.21.15.156:0/3789777782 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c011990 con 0x7f852c1596b0 2023-11-28T11:12:15.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.073+0000 7f853136e700 1 -- 172.21.15.156:0/3789777782 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c011d00 con 0x7f852c1596b0 2023-11-28T11:12:15.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.073+0000 7f85345d4700 1 -- 172.21.15.156:0/3789777782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f852c1596b0 msgr2=0x7f852c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.073+0000 7f85345d4700 1 --2- 172.21.15.156:0/3789777782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f852c1596b0 0x7f852c071410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f851c009840 tx=0x7f851c009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:15.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.074+0000 7f85345d4700 1 -- 172.21.15.156:0/3789777782 shutdown_connections 2023-11-28T11:12:15.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.074+0000 7f85345d4700 1 --2- 172.21.15.156:0/3789777782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f852c0d0470 0x7f852c06cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.074+0000 7f85345d4700 1 --2- 172.21.15.156:0/3789777782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f852c1596b0 0x7f852c071410 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.074+0000 7f85345d4700 1 -- 172.21.15.156:0/3789777782 >> 172.21.15.156:0/3789777782 conn(0x7f852c06b380 msgr2=0x7f852c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:15.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.074+0000 7f85345d4700 1 -- 172.21.15.156:0/3789777782 shutdown_connections 2023-11-28T11:12:15.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.074+0000 7f85345d4700 1 -- 172.21.15.156:0/3789777782 wait complete. 2023-11-28T11:12:15.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.075+0000 7f85345d4700 1 Processor -- start 2023-11-28T11:12:15.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.075+0000 7f85345d4700 1 -- start start 2023-11-28T11:12:15.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.075+0000 7f85345d4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f852c0cc260 0x7f852c1f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.075+0000 7f85345d4700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f852c0d0470 0x7f852c152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.075+0000 7f85345d4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c1596b0 0x7f852c1f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.075+0000 7f85345d4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0c77c0 con 0x7f852c1596b0 2023-11-28T11:12:15.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.075+0000 7f85345d4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c1f36f0 con 0x7f852c0cc260 2023-11-28T11:12:15.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.075+0000 7f85345d4700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c1f6330 con 0x7f852c0d0470 2023-11-28T11:12:15.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8532b71700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c1596b0 0x7f852c1f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8532b71700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c1596b0 0x7f852c1f31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36782/0 (socket says 172.21.15.156:36782) 2023-11-28T11:12:15.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8532b71700 1 -- 172.21.15.156:0/3755674956 learned_addr learned my addr 172.21.15.156:0/3755674956 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:15.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8532b71700 1 -- 172.21.15.156:0/3755674956 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f852c0d0470 msgr2=0x7f852c152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8532370700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f852c0cc260 0x7f852c1f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8531b6f700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f852c0d0470 0x7f852c152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8532b71700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f852c0d0470 0x7f852c152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8532b71700 1 -- 172.21.15.156:0/3755674956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f852c0cc260 msgr2=0x7f852c1f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8532b71700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f852c0cc260 0x7f852c1f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8532b71700 1 -- 172.21.15.156:0/3755674956 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8524009a70 con 0x7f852c1596b0 2023-11-28T11:12:15.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8532370700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f852c0cc260 0x7f852c1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:15.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8532b71700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c1596b0 0x7f852c1f31b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f852400efc0 tx=0x7f852400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8522ffd700 1 -- 172.21.15.156:0/3755674956 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852400ce00 con 0x7f852c1596b0 2023-11-28T11:12:15.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8522ffd700 1 -- 172.21.15.156:0/3755674956 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852400f040 con 0x7f852c1596b0 2023-11-28T11:12:15.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f85345d4700 1 -- 172.21.15.156:0/3755674956 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c0094a0 con 0x7f852c1596b0 2023-11-28T11:12:15.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f8522ffd700 1 -- 172.21.15.156:0/3755674956 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85240146c0 con 0x7f852c1596b0 2023-11-28T11:12:15.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.076+0000 7f85345d4700 1 -- 172.21.15.156:0/3755674956 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c0ccfc0 con 0x7f852c1596b0 2023-11-28T11:12:15.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.077+0000 7f85345d4700 1 -- 172.21.15.156:0/3755674956 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f852c0ccce0 con 0x7f852c1596b0 2023-11-28T11:12:15.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.078+0000 7f8522ffd700 1 -- 172.21.15.156:0/3755674956 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8524014820 con 0x7f852c1596b0 2023-11-28T11:12:15.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.078+0000 7f8522ffd700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f85100682f0 0x7f851006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.078+0000 7f8522ffd700 1 -- 172.21.15.156:0/3755674956 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f85240868a0 con 0x7f852c1596b0 2023-11-28T11:12:15.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.078+0000 7f8532370700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f85100682f0 0x7f851006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.078+0000 7f8522ffd700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x7f851006e760 0x7f8510070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.078+0000 7f8522ffd700 1 -- 172.21.15.156:0/3755674956 --> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8510071240 con 0x7f851006e760 2023-11-28T11:12:15.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.078+0000 7f8522ffd700 1 -- 172.21.15.156:0/3755674956 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f852408b2a0 con 0x7f852c1596b0 2023-11-28T11:12:15.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.078+0000 7f8532370700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f85100682f0 0x7f851006a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f851c0090b0 tx=0x7f851c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.078+0000 7f8531b6f700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x7f851006e760 0x7f8510070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.079+0000 7f8531b6f700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x7f851006e760 0x7f8510070b70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.080+0000 7f8522ffd700 1 -- 172.21.15.156:0/3755674956 <== osd.5 v2:172.21.15.186:6808/103778 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8510071240 con 0x7f851006e760 2023-11-28T11:12:15.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.093+0000 7f85345d4700 1 -- 172.21.15.156:0/3755674956 --> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f852c0cd2a0 con 0x7f851006e760 2023-11-28T11:12:15.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.094+0000 7f8522ffd700 1 -- 172.21.15.156:0/3755674956 <== osd.5 v2:172.21.15.186:6808/103778 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f852c0cd2a0 con 0x7f851006e760 2023-11-28T11:12:15.095 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:12:15.096 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:15.096 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:12:15.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.094+0000 7f85345d4700 1 -- 172.21.15.156:0/3755674956 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x7f851006e760 msgr2=0x7f8510070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.094+0000 7f85345d4700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.186:6808/103778,v1:172.21.15.186:6809/103778] conn(0x7f851006e760 0x7f8510070b70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.095+0000 7f85345d4700 1 -- 172.21.15.156:0/3755674956 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f85100682f0 msgr2=0x7f851006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.095+0000 7f85345d4700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f85100682f0 0x7f851006a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f851c0090b0 tx=0x7f851c008030 comp rx=0 tx=0).stop 2023-11-28T11:12:15.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.095+0000 7f85345d4700 1 -- 172.21.15.156:0/3755674956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c1596b0 msgr2=0x7f852c1f31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.095+0000 7f85345d4700 1 --2- 172.21.15.156:0/3755674956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c1596b0 0x7f852c1f31b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f852400efc0 tx=0x7f852400c3a0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.095+0000 7f8532b71700 1 -- 172.21.15.156:0/3755674956 reap_dead start 2023-11-28T11:12:15.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.095+0000 7f85345d4700 1 -- 172.21.15.156:0/3755674956 shutdown_connections 2023-11-28T11:12:15.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.095+0000 7f85345d4700 1 -- 172.21.15.156:0/3755674956 >> 172.21.15.156:0/3755674956 conn(0x7f852c06b380 msgr2=0x7f852c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:15.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.095+0000 7f85345d4700 1 -- 172.21.15.156:0/3755674956 shutdown_connections 2023-11-28T11:12:15.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.095+0000 7f85345d4700 1 -- 172.21.15.156:0/3755674956 wait complete. 2023-11-28T11:12:15.107 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-28T11:12:15.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.228+0000 7fe8761e8700 1 Processor -- start 2023-11-28T11:12:15.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.228+0000 7fe8761e8700 1 -- start start 2023-11-28T11:12:15.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.228+0000 7fe8761e8700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe87015c9b0 0x7fe8700b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.228+0000 7fe8761e8700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700cd7c0 0x7fe8700cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe8761e8700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8700d1950 con 0x7fe870161340 2023-11-28T11:12:15.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe8761e8700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8700b5800 con 0x7fe87015c9b0 2023-11-28T11:12:15.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe8761e8700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700ce140 con 0x7fe8700cd7c0 2023-11-28T11:12:15.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe86effd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe870161340 0x7fe8700d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51666/0 (socket says 172.21.15.156:51666) 2023-11-28T11:12:15.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe86effd700 1 -- 172.21.15.156:0/2658369320 learned_addr learned my addr 172.21.15.156:0/2658369320 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:15.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe86f7fe700 1 --2- 172.21.15.156:0/2658369320 >> v2:172.21.15.186:3300/0 conn(0x7fe87015c9b0 0x7fe8700b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe86ffff700 1 --2- 172.21.15.156:0/2658369320 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700cd7c0 0x7fe8700cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe86e7fc700 1 -- 172.21.15.156:0/2658369320 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304227873 0 0) 0x7fe8700d1950 con 0x7fe870161340 2023-11-28T11:12:15.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe86e7fc700 1 -- 172.21.15.156:0/2658369320 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8580035b0 con 0x7fe870161340 2023-11-28T11:12:15.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe86f7fe700 1 -- 172.21.15.156:0/2658369320 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700cd7c0 msgr2=0x7fe8700cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:15.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe86f7fe700 1 --2- 172.21.15.156:0/2658369320 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700cd7c0 0x7fe8700cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe86f7fe700 1 -- 172.21.15.156:0/2658369320 >> v1:172.21.15.156:6789/0 conn(0x7fe870161340 legacy=0x7fe8700d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe86f7fe700 1 -- 172.21.15.156:0/2658369320 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8700d24f0 con 0x7fe87015c9b0 2023-11-28T11:12:15.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe86f7fe700 1 --2- 172.21.15.156:0/2658369320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe87015c9b0 0x7fe8700b4bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe860009840 tx=0x7fe860009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db21b00c94d6fc06 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.229+0000 7fe86e7fc700 1 -- 172.21.15.156:0/2658369320 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860011830 con 0x7fe87015c9b0 2023-11-28T11:12:15.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.230+0000 7fe86e7fc700 1 -- 172.21.15.156:0/2658369320 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe860011990 con 0x7fe87015c9b0 2023-11-28T11:12:15.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.230+0000 7fe86e7fc700 1 -- 172.21.15.156:0/2658369320 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860011d20 con 0x7fe87015c9b0 2023-11-28T11:12:15.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.230+0000 7fe8761e8700 1 -- 172.21.15.156:0/2658369320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe87015c9b0 msgr2=0x7fe8700b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.230+0000 7fe8761e8700 1 --2- 172.21.15.156:0/2658369320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe87015c9b0 0x7fe8700b4bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe860009840 tx=0x7fe860009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:15.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.230+0000 7fe8761e8700 1 -- 172.21.15.156:0/2658369320 shutdown_connections 2023-11-28T11:12:15.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.230+0000 7fe8761e8700 1 --2- 172.21.15.156:0/2658369320 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700cd7c0 0x7fe8700cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.230+0000 7fe8761e8700 1 --2- 172.21.15.156:0/2658369320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe87015c9b0 0x7fe8700b4bf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.230+0000 7fe8761e8700 1 -- 172.21.15.156:0/2658369320 >> 172.21.15.156:0/2658369320 conn(0x7fe870072600 msgr2=0x7fe8700cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:15.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.230+0000 7fe8761e8700 1 -- 172.21.15.156:0/2658369320 shutdown_connections 2023-11-28T11:12:15.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.230+0000 7fe8761e8700 1 -- 172.21.15.156:0/2658369320 wait complete. 2023-11-28T11:12:15.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.231+0000 7fe8761e8700 1 Processor -- start 2023-11-28T11:12:15.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.232+0000 7fe8761e8700 1 -- start start 2023-11-28T11:12:15.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.232+0000 7fe8761e8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8700cd7c0 0x7fe870156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.232+0000 7fe8761e8700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe87015c9b0 0x7fe8700cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.232+0000 7fe8761e8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe870161340 0x7fe87015e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.232+0000 7fe8761e8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe870157140 con 0x7fe8700cd7c0 2023-11-28T11:12:15.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.232+0000 7fe8761e8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700cbdc0 con 0x7fe870161340 2023-11-28T11:12:15.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.232+0000 7fe8761e8700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700cc0a0 con 0x7fe87015c9b0 2023-11-28T11:12:15.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.232+0000 7fe86f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8700cd7c0 0x7fe870156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.232+0000 7fe86f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8700cd7c0 0x7fe870156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36794/0 (socket says 172.21.15.156:36794) 2023-11-28T11:12:15.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.232+0000 7fe86f7fe700 1 -- 172.21.15.156:0/2615445157 learned_addr learned my addr 172.21.15.156:0/2615445157 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:15.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe86ffff700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe870161340 0x7fe87015e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe86effd700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe87015c9b0 0x7fe8700cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe86f7fe700 1 -- 172.21.15.156:0/2615445157 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe87015c9b0 msgr2=0x7fe8700cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe86f7fe700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe87015c9b0 0x7fe8700cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe86f7fe700 1 -- 172.21.15.156:0/2615445157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe870161340 msgr2=0x7fe87015e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe86f7fe700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe870161340 0x7fe87015e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe86f7fe700 1 -- 172.21.15.156:0/2615445157 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8600094a0 con 0x7fe8700cd7c0 2023-11-28T11:12:15.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe86ffff700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe870161340 0x7fe87015e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:15.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe86f7fe700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8700cd7c0 0x7fe870156c00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe860009fd0 tx=0x7fe860004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe857fff700 1 -- 172.21.15.156:0/2615445157 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8600036a0 con 0x7fe8700cd7c0 2023-11-28T11:12:15.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe857fff700 1 -- 172.21.15.156:0/2615445157 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86001e070 con 0x7fe8700cd7c0 2023-11-28T11:12:15.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe8761e8700 1 -- 172.21.15.156:0/2615445157 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8700d1300 con 0x7fe8700cd7c0 2023-11-28T11:12:15.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe857fff700 1 -- 172.21.15.156:0/2615445157 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860003a60 con 0x7fe8700cd7c0 2023-11-28T11:12:15.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.233+0000 7fe8761e8700 1 -- 172.21.15.156:0/2615445157 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8700b5520 con 0x7fe8700cd7c0 2023-11-28T11:12:15.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.234+0000 7fe8761e8700 1 -- 172.21.15.156:0/2615445157 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8700d15e0 con 0x7fe8700cd7c0 2023-11-28T11:12:15.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.234+0000 7fe857fff700 1 -- 172.21.15.156:0/2615445157 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe86000f050 con 0x7fe8700cd7c0 2023-11-28T11:12:15.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.235+0000 7fe857fff700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe85c0682f0 0x7fe85c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.235+0000 7fe857fff700 1 -- 172.21.15.156:0/2615445157 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe8600868e0 con 0x7fe8700cd7c0 2023-11-28T11:12:15.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.235+0000 7fe86effd700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe85c0682f0 0x7fe85c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.235+0000 7fe86effd700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe85c0682f0 0x7fe85c06a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe87015fa00 tx=0x7fe864009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.239+0000 7fe857fff700 1 -- 172.21.15.156:0/2615445157 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe860053440 con 0x7fe8700cd7c0 2023-11-28T11:12:15.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.421+0000 7fe8761e8700 1 -- 172.21.15.156:0/2615445157 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8700cc210 con 0x7fe85c0682f0 2023-11-28T11:12:15.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.421+0000 7fe857fff700 1 -- 172.21.15.156:0/2615445157 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe8700cc210 con 0x7fe85c0682f0 2023-11-28T11:12:15.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.423+0000 7fe8761e8700 1 -- 172.21.15.156:0/2615445157 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe85c0682f0 msgr2=0x7fe85c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.423+0000 7fe8761e8700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe85c0682f0 0x7fe85c06a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe87015fa00 tx=0x7fe864009040 comp rx=0 tx=0).stop 2023-11-28T11:12:15.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.423+0000 7fe8761e8700 1 -- 172.21.15.156:0/2615445157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8700cd7c0 msgr2=0x7fe870156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.423+0000 7fe8761e8700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8700cd7c0 0x7fe870156c00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe860009fd0 tx=0x7fe860004750 comp rx=0 tx=0).stop 2023-11-28T11:12:15.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.423+0000 7fe8761e8700 1 -- 172.21.15.156:0/2615445157 shutdown_connections 2023-11-28T11:12:15.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.423+0000 7fe8761e8700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe870161340 0x7fe87015e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.423+0000 7fe8761e8700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8700cd7c0 0x7fe870156c00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.423+0000 7fe8761e8700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fe85c0682f0 0x7fe85c06a7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.423+0000 7fe8761e8700 1 --2- 172.21.15.156:0/2615445157 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe87015c9b0 0x7fe8700cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.423+0000 7fe8761e8700 1 -- 172.21.15.156:0/2615445157 >> 172.21.15.156:0/2615445157 conn(0x7fe870072600 msgr2=0x7fe8700b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:15.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.423+0000 7fe8761e8700 1 -- 172.21.15.156:0/2615445157 shutdown_connections 2023-11-28T11:12:15.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.423+0000 7fe8761e8700 1 -- 172.21.15.156:0/2615445157 wait complete. 2023-11-28T11:12:15.427 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 5 to deep-scrub 2023-11-28T11:12:15.436 INFO:tasks.ceph:Scrubbing osd.6 2023-11-28T11:12:15.437 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:15.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.559+0000 7fa5654ac700 1 Processor -- start 2023-11-28T11:12:15.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.559+0000 7fa5654ac700 1 -- start start 2023-11-28T11:12:15.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.560+0000 7fa5654ac700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa5600cd910 0x7fa5600cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.560+0000 7fa5654ac700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa5600cfef0 0x7fa5600d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.560+0000 7fa5654ac700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa56006bdd0 0x7fa56006c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.560+0000 7fa5654ac700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56006b9b0 con 0x7fa5600cd910 2023-11-28T11:12:15.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.560+0000 7fa5654ac700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56006b380 con 0x7fa5600cfef0 2023-11-28T11:12:15.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.560+0000 7fa5654ac700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56006c790 con 0x7fa56006bdd0 2023-11-28T11:12:15.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.560+0000 7fa55effd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa5600cd910 0x7fa5600cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.560+0000 7fa55effd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa5600cd910 0x7fa5600cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36802/0 (socket says 172.21.15.156:36802) 2023-11-28T11:12:15.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.560+0000 7fa55effd700 1 -- 172.21.15.156:0/2323555274 learned_addr learned my addr 172.21.15.156:0/2323555274 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:15.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.560+0000 7fa55e7fc700 1 --2- 172.21.15.156:0/2323555274 >> v2:172.21.15.186:3300/0 conn(0x7fa5600cfef0 0x7fa5600d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa55f7fe700 1 --2- 172.21.15.156:0/2323555274 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa56006bdd0 0x7fa56006c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa55effd700 1 -- 172.21.15.156:0/2323555274 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa56006bdd0 msgr2=0x7fa56006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:15.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa55effd700 1 --2- 172.21.15.156:0/2323555274 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa56006bdd0 0x7fa56006c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa55effd700 1 -- 172.21.15.156:0/2323555274 >> v2:172.21.15.186:3300/0 conn(0x7fa5600cfef0 msgr2=0x7fa5600d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:15.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa55effd700 1 --2- 172.21.15.156:0/2323555274 >> v2:172.21.15.186:3300/0 conn(0x7fa5600cfef0 0x7fa5600d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa55effd700 1 -- 172.21.15.156:0/2323555274 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5600ce220 con 0x7fa5600cd910 2023-11-28T11:12:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa55e7fc700 1 --2- 172.21.15.156:0/2323555274 >> v2:172.21.15.186:3300/0 conn(0x7fa5600cfef0 0x7fa5600d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa55effd700 1 --2- 172.21.15.156:0/2323555274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5600cd910 0x7fa5600cdce0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa560070a60 tx=0x7fa554009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ace85f38e3f0218 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa55dffb700 1 -- 172.21.15.156:0/2323555274 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55400b870 con 0x7fa5600cd910 2023-11-28T11:12:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa55dffb700 1 -- 172.21.15.156:0/2323555274 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55400b9d0 con 0x7fa5600cd910 2023-11-28T11:12:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa55dffb700 1 -- 172.21.15.156:0/2323555274 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55400bd60 con 0x7fa5600cd910 2023-11-28T11:12:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa5654ac700 1 -- 172.21.15.156:0/2323555274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5600cd910 msgr2=0x7fa5600cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa5654ac700 1 --2- 172.21.15.156:0/2323555274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5600cd910 0x7fa5600cdce0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa560070a60 tx=0x7fa554009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa5654ac700 1 -- 172.21.15.156:0/2323555274 shutdown_connections 2023-11-28T11:12:15.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa5654ac700 1 --2- 172.21.15.156:0/2323555274 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa56006bdd0 0x7fa56006c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa5654ac700 1 --2- 172.21.15.156:0/2323555274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5600cd910 0x7fa5600cdce0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa5654ac700 1 --2- 172.21.15.156:0/2323555274 >> v2:172.21.15.186:3300/0 conn(0x7fa5600cfef0 0x7fa5600d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa5654ac700 1 -- 172.21.15.156:0/2323555274 >> 172.21.15.156:0/2323555274 conn(0x7fa56006df70 msgr2=0x7fa56006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:15.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.561+0000 7fa5654ac700 1 -- 172.21.15.156:0/2323555274 shutdown_connections 2023-11-28T11:12:15.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.562+0000 7fa5654ac700 1 -- 172.21.15.156:0/2323555274 wait complete. 2023-11-28T11:12:15.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.563+0000 7fa5654ac700 1 Processor -- start 2023-11-28T11:12:15.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.563+0000 7fa5654ac700 1 -- start start 2023-11-28T11:12:15.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa5654ac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa56006bdd0 0x7fa56014bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa5654ac700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5600cd910 0x7fa56014b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa5654ac700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5600cfef0 0x7fa5601e89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa5654ac700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56015c2c0 con 0x7fa5600cd910 2023-11-28T11:12:15.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa5654ac700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56015e540 con 0x7fa56006bdd0 2023-11-28T11:12:15.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa5654ac700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56015e820 con 0x7fa5600cfef0 2023-11-28T11:12:15.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa55e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5600cd910 0x7fa56014b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa55e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5600cd910 0x7fa56014b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36810/0 (socket says 172.21.15.156:36810) 2023-11-28T11:12:15.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa55e7fc700 1 -- 172.21.15.156:0/2441861072 learned_addr learned my addr 172.21.15.156:0/2441861072 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:15.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa55f7fe700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5600cfef0 0x7fa5601e89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa55effd700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa56006bdd0 0x7fa56014bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa55e7fc700 1 -- 172.21.15.156:0/2441861072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5600cfef0 msgr2=0x7fa5601e89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa55e7fc700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5600cfef0 0x7fa5601e89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa55e7fc700 1 -- 172.21.15.156:0/2441861072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa56006bdd0 msgr2=0x7fa56014bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa55e7fc700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa56006bdd0 0x7fa56014bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa55e7fc700 1 -- 172.21.15.156:0/2441861072 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa554009550 con 0x7fa5600cd910 2023-11-28T11:12:15.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa55effd700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa56006bdd0 0x7fa56014bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:15.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa55e7fc700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5600cd910 0x7fa56014b210 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c750 tx=0x7fa55000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa5477fe700 1 -- 172.21.15.156:0/2441861072 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550011800 con 0x7fa5600cd910 2023-11-28T11:12:15.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa5477fe700 1 -- 172.21.15.156:0/2441861072 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa550011960 con 0x7fa5600cd910 2023-11-28T11:12:15.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa5654ac700 1 -- 172.21.15.156:0/2441861072 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56015e160 con 0x7fa5600cd910 2023-11-28T11:12:15.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa5477fe700 1 -- 172.21.15.156:0/2441861072 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550011cd0 con 0x7fa5600cd910 2023-11-28T11:12:15.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.564+0000 7fa5654ac700 1 -- 172.21.15.156:0/2441861072 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56015de80 con 0x7fa5600cd910 2023-11-28T11:12:15.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.565+0000 7fa5654ac700 1 -- 172.21.15.156:0/2441861072 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa56006b5d0 con 0x7fa5600cd910 2023-11-28T11:12:15.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.566+0000 7fa5477fe700 1 -- 172.21.15.156:0/2441861072 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa550020070 con 0x7fa5600cd910 2023-11-28T11:12:15.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.566+0000 7fa5477fe700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa54c0682f0 0x7fa54c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.566+0000 7fa5477fe700 1 -- 172.21.15.156:0/2441861072 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa550085810 con 0x7fa5600cd910 2023-11-28T11:12:15.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.566+0000 7fa55effd700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa54c0682f0 0x7fa54c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.566+0000 7fa5477fe700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x7fa54c06e760 0x7fa54c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.566+0000 7fa5477fe700 1 -- 172.21.15.156:0/2441861072 --> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa54c071240 con 0x7fa54c06e760 2023-11-28T11:12:15.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.566+0000 7fa5477fe700 1 -- 172.21.15.156:0/2441861072 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa55008a2a0 con 0x7fa5600cd910 2023-11-28T11:12:15.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.566+0000 7fa55f7fe700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x7fa54c06e760 0x7fa54c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.566+0000 7fa55effd700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa54c0682f0 0x7fa54c06a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa554009840 tx=0x7fa554008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.567+0000 7fa55f7fe700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x7fa54c06e760 0x7fa54c070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.568+0000 7fa5477fe700 1 -- 172.21.15.156:0/2441861072 <== osd.6 v2:172.21.15.186:6800/103779 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa54c071240 con 0x7fa54c06e760 2023-11-28T11:12:15.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.581+0000 7fa5654ac700 1 -- 172.21.15.156:0/2441861072 --> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa560068690 con 0x7fa54c06e760 2023-11-28T11:12:15.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.582+0000 7fa5477fe700 1 -- 172.21.15.156:0/2441861072 <== osd.6 v2:172.21.15.186:6800/103779 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa560068690 con 0x7fa54c06e760 2023-11-28T11:12:15.583 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:12:15.583 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:15.584 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:12:15.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.582+0000 7fa5654ac700 1 -- 172.21.15.156:0/2441861072 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x7fa54c06e760 msgr2=0x7fa54c070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.582+0000 7fa5654ac700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.186:6800/103779,v1:172.21.15.186:6801/103779] conn(0x7fa54c06e760 0x7fa54c070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.582+0000 7fa5654ac700 1 -- 172.21.15.156:0/2441861072 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa54c0682f0 msgr2=0x7fa54c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.582+0000 7fa5654ac700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fa54c0682f0 0x7fa54c06a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa554009840 tx=0x7fa554008030 comp rx=0 tx=0).stop 2023-11-28T11:12:15.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.582+0000 7fa5654ac700 1 -- 172.21.15.156:0/2441861072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5600cd910 msgr2=0x7fa56014b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.582+0000 7fa5654ac700 1 --2- 172.21.15.156:0/2441861072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5600cd910 0x7fa56014b210 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c750 tx=0x7fa55000cb10 comp rx=0 tx=0).stop 2023-11-28T11:12:15.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.582+0000 7fa55f7fe700 1 -- 172.21.15.156:0/2441861072 reap_dead start 2023-11-28T11:12:15.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.582+0000 7fa5654ac700 1 -- 172.21.15.156:0/2441861072 shutdown_connections 2023-11-28T11:12:15.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.582+0000 7fa5654ac700 1 -- 172.21.15.156:0/2441861072 >> 172.21.15.156:0/2441861072 conn(0x7fa56006df70 msgr2=0x7fa56006c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:15.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.582+0000 7fa5654ac700 1 -- 172.21.15.156:0/2441861072 shutdown_connections 2023-11-28T11:12:15.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.583+0000 7fa5654ac700 1 -- 172.21.15.156:0/2441861072 wait complete. 2023-11-28T11:12:15.595 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-28T11:12:15.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.717+0000 7f11eba14700 1 Processor -- start 2023-11-28T11:12:15.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.717+0000 7f11eba14700 1 -- start start 2023-11-28T11:12:15.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11eba14700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f11e406dee0 0x7f11e406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11eba14700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e415d060 con 0x7f11e4160540 2023-11-28T11:12:15.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11eba14700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e415d340 con 0x7f11e4071040 2023-11-28T11:12:15.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11eba14700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e415d620 con 0x7f11e406dee0 2023-11-28T11:12:15.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11e8faf700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f11e406dee0 0x7f11e406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11e9fb1700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f11e4160540 0x7f11e4160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51700/0 (socket says 172.21.15.156:51700) 2023-11-28T11:12:15.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11e8faf700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f11e406dee0 0x7f11e406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36820/0 (socket says 172.21.15.156:36820) 2023-11-28T11:12:15.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11e9fb1700 1 -- 172.21.15.156:0/4213894154 learned_addr learned my addr 172.21.15.156:0/4213894154 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:15.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11e8faf700 1 -- 172.21.15.156:0/4213894154 >> v1:172.21.15.186:6789/0 conn(0x7f11e4071040 legacy=0x7f11e4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11e8faf700 1 -- 172.21.15.156:0/4213894154 >> v1:172.21.15.156:6789/0 conn(0x7f11e4160540 legacy=0x7f11e4160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11e8faf700 1 -- 172.21.15.156:0/4213894154 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e415d900 con 0x7f11e406dee0 2023-11-28T11:12:15.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11dbfff700 1 -- 172.21.15.156:0/4213894154 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729044676 0 0) 0x7f11e415d060 con 0x7f11e4160540 2023-11-28T11:12:15.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11e8faf700 1 --2- 172.21.15.156:0/4213894154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11e406dee0 0x7f11e406e330 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009840 tx=0x7f11d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c67f56ae8a41750 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.718+0000 7f11dbfff700 1 -- 172.21.15.156:0/4213894154 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4011830 con 0x7f11e406dee0 2023-11-28T11:12:15.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.719+0000 7f11dbfff700 1 -- 172.21.15.156:0/4213894154 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d4011990 con 0x7f11e406dee0 2023-11-28T11:12:15.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.719+0000 7f11dbfff700 1 -- 172.21.15.156:0/4213894154 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4011d20 con 0x7f11e406dee0 2023-11-28T11:12:15.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.719+0000 7f11eba14700 1 -- 172.21.15.156:0/4213894154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11e406dee0 msgr2=0x7f11e406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.719+0000 7f11eba14700 1 --2- 172.21.15.156:0/4213894154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11e406dee0 0x7f11e406e330 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009840 tx=0x7f11d4009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:15.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.719+0000 7f11eba14700 1 -- 172.21.15.156:0/4213894154 shutdown_connections 2023-11-28T11:12:15.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.719+0000 7f11eba14700 1 --2- 172.21.15.156:0/4213894154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11e406dee0 0x7f11e406e330 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.719+0000 7f11eba14700 1 -- 172.21.15.156:0/4213894154 >> 172.21.15.156:0/4213894154 conn(0x7f11e40d0000 msgr2=0x7f11e40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:15.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.719+0000 7f11eba14700 1 -- 172.21.15.156:0/4213894154 shutdown_connections 2023-11-28T11:12:15.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.719+0000 7f11eba14700 1 -- 172.21.15.156:0/4213894154 wait complete. 2023-11-28T11:12:15.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.720+0000 7f11eba14700 1 Processor -- start 2023-11-28T11:12:15.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.720+0000 7f11eba14700 1 -- start start 2023-11-28T11:12:15.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11eba14700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11e406dee0 0x7f11e40ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11eba14700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11e4071040 0x7f11e40b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11eba14700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11e4160540 0x7f11e40b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11eba14700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40ae7c0 con 0x7f11e406dee0 2023-11-28T11:12:15.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11eba14700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40aeaa0 con 0x7f11e4160540 2023-11-28T11:12:15.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11eba14700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40aed80 con 0x7f11e4071040 2023-11-28T11:12:15.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11e97b0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11e406dee0 0x7f11e40ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11e97b0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11e406dee0 0x7f11e40ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36824/0 (socket says 172.21.15.156:36824) 2023-11-28T11:12:15.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11e97b0700 1 -- 172.21.15.156:0/1898578244 learned_addr learned my addr 172.21.15.156:0/1898578244 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:15.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11e8faf700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11e4071040 0x7f11e40b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11e9fb1700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11e4160540 0x7f11e40b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11e97b0700 1 -- 172.21.15.156:0/1898578244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11e4071040 msgr2=0x7f11e40b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.721+0000 7f11e97b0700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11e4071040 0x7f11e40b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.722+0000 7f11e97b0700 1 -- 172.21.15.156:0/1898578244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11e4160540 msgr2=0x7f11e40b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.722+0000 7f11e97b0700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11e4160540 0x7f11e40b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.722+0000 7f11e97b0700 1 -- 172.21.15.156:0/1898578244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d40094a0 con 0x7f11e406dee0 2023-11-28T11:12:15.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.722+0000 7f11e9fb1700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11e4160540 0x7f11e40b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:15.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.722+0000 7f11e97b0700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11e406dee0 0x7f11e40ade40 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f11e000c4d0 tx=0x7f11e000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.722+0000 7f11d9ffb700 1 -- 172.21.15.156:0/1898578244 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e00052d0 con 0x7f11e406dee0 2023-11-28T11:12:15.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.722+0000 7f11d9ffb700 1 -- 172.21.15.156:0/1898578244 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e0005430 con 0x7f11e406dee0 2023-11-28T11:12:15.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.722+0000 7f11eba14700 1 -- 172.21.15.156:0/1898578244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e415ccd0 con 0x7f11e406dee0 2023-11-28T11:12:15.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.722+0000 7f11d9ffb700 1 -- 172.21.15.156:0/1898578244 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0003880 con 0x7f11e406dee0 2023-11-28T11:12:15.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.722+0000 7f11eba14700 1 -- 172.21.15.156:0/1898578244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e406e870 con 0x7f11e406dee0 2023-11-28T11:12:15.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.723+0000 7f11eba14700 1 -- 172.21.15.156:0/1898578244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e406a270 con 0x7f11e406dee0 2023-11-28T11:12:15.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.723+0000 7f11d9ffb700 1 -- 172.21.15.156:0/1898578244 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f11e0019030 con 0x7f11e406dee0 2023-11-28T11:12:15.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.724+0000 7f11d9ffb700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f11c80682f0 0x7f11c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:15.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.724+0000 7f11d9ffb700 1 -- 172.21.15.156:0/1898578244 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f11e0084b30 con 0x7f11e406dee0 2023-11-28T11:12:15.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.724+0000 7f11e8faf700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f11c80682f0 0x7f11c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:15.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.724+0000 7f11e8faf700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f11c80682f0 0x7f11c806a7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f11d40090b0 tx=0x7f11d401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:15.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.728+0000 7f11d9ffb700 1 -- 172.21.15.156:0/1898578244 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f11e004b5e0 con 0x7f11e406dee0 2023-11-28T11:12:15.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.911+0000 7f11eba14700 1 -- 172.21.15.156:0/1898578244 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f11e40b4ca0 con 0x7f11c80682f0 2023-11-28T11:12:15.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.912+0000 7f11d9ffb700 1 -- 172.21.15.156:0/1898578244 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f11e40b4ca0 con 0x7f11c80682f0 2023-11-28T11:12:15.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.914+0000 7f11eba14700 1 -- 172.21.15.156:0/1898578244 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f11c80682f0 msgr2=0x7f11c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.914+0000 7f11eba14700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f11c80682f0 0x7f11c806a7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f11d40090b0 tx=0x7f11d401c040 comp rx=0 tx=0).stop 2023-11-28T11:12:15.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.914+0000 7f11eba14700 1 -- 172.21.15.156:0/1898578244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11e406dee0 msgr2=0x7f11e40ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:15.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.914+0000 7f11eba14700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11e406dee0 0x7f11e40ade40 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f11e000c4d0 tx=0x7f11e000c7e0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.914+0000 7f11eba14700 1 -- 172.21.15.156:0/1898578244 shutdown_connections 2023-11-28T11:12:15.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.914+0000 7f11eba14700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11e4160540 0x7f11e40b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.914+0000 7f11eba14700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11e406dee0 0x7f11e40ade40 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.914+0000 7f11eba14700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f11c80682f0 0x7f11c806a7a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.914+0000 7f11eba14700 1 --2- 172.21.15.156:0/1898578244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11e4071040 0x7f11e40b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:15.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.914+0000 7f11eba14700 1 -- 172.21.15.156:0/1898578244 >> 172.21.15.156:0/1898578244 conn(0x7f11e40d0000 msgr2=0x7f11e406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:15.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.914+0000 7f11eba14700 1 -- 172.21.15.156:0/1898578244 shutdown_connections 2023-11-28T11:12:15.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:15.914+0000 7f11eba14700 1 -- 172.21.15.156:0/1898578244 wait complete. 2023-11-28T11:12:15.917 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 6 to deep-scrub 2023-11-28T11:12:15.927 INFO:tasks.ceph:Scrubbing osd.7 2023-11-28T11:12:15.927 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:12:16.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.052+0000 7f1ebbc73700 1 Processor -- start 2023-11-28T11:12:16.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.052+0000 7f1ebbc73700 1 -- start start 2023-11-28T11:12:16.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.053+0000 7f1ebbc73700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1eb415c6e0 0x7f1eb40cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.053+0000 7f1ebbc73700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb406b3b0 con 0x7f1eb40d0120 2023-11-28T11:12:16.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.053+0000 7f1ebbc73700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb406a310 con 0x7f1eb40cc570 2023-11-28T11:12:16.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.053+0000 7f1ebbc73700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb406a5f0 con 0x7f1eb415c6e0 2023-11-28T11:12:16.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.053+0000 7f1eb9a0f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1eb415c6e0 0x7f1eb40cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.053+0000 7f1eb9a0f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1eb415c6e0 0x7f1eb40cfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36832/0 (socket says 172.21.15.156:36832) 2023-11-28T11:12:16.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.053+0000 7f1eb920e700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f1eb40d0120 0x7f1eb40cbe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51712/0 (socket says 172.21.15.156:51712) 2023-11-28T11:12:16.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.053+0000 7f1eb9a0f700 1 -- 172.21.15.156:0/960064968 learned_addr learned my addr 172.21.15.156:0/960064968 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:16.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.053+0000 7f1eb9a0f700 1 -- 172.21.15.156:0/960064968 >> v1:172.21.15.186:6789/0 conn(0x7f1eb40cc570 legacy=0x7f1eb40cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:16.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.053+0000 7f1eb9a0f700 1 -- 172.21.15.156:0/960064968 >> v1:172.21.15.156:6789/0 conn(0x7f1eb40d0120 legacy=0x7f1eb40cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.053+0000 7f1eb9a0f700 1 -- 172.21.15.156:0/960064968 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb406a8d0 con 0x7f1eb415c6e0 2023-11-28T11:12:16.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.053+0000 7f1eb8a0d700 1 -- 172.21.15.156:0/960064968 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628469708 0 0) 0x7f1eb406b3b0 con 0x7f1eb40d0120 2023-11-28T11:12:16.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.054+0000 7f1eb9a0f700 1 --2- 172.21.15.156:0/960064968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb415c6e0 0x7f1eb40cfbe0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400bb90 tx=0x7f1ea400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ea56d8cc2f92d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.054+0000 7f1eb8a0d700 1 -- 172.21.15.156:0/960064968 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea400dd10 con 0x7f1eb415c6e0 2023-11-28T11:12:16.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.054+0000 7f1eb8a0d700 1 -- 172.21.15.156:0/960064968 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea400de70 con 0x7f1eb415c6e0 2023-11-28T11:12:16.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.054+0000 7f1eb8a0d700 1 -- 172.21.15.156:0/960064968 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4004230 con 0x7f1eb415c6e0 2023-11-28T11:12:16.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.054+0000 7f1ebbc73700 1 -- 172.21.15.156:0/960064968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb415c6e0 msgr2=0x7f1eb40cfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.054+0000 7f1ebbc73700 1 --2- 172.21.15.156:0/960064968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb415c6e0 0x7f1eb40cfbe0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400bb90 tx=0x7f1ea400bea0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.054+0000 7f1ebbc73700 1 -- 172.21.15.156:0/960064968 shutdown_connections 2023-11-28T11:12:16.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.054+0000 7f1ebbc73700 1 --2- 172.21.15.156:0/960064968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb415c6e0 0x7f1eb40cfbe0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.054+0000 7f1ebbc73700 1 -- 172.21.15.156:0/960064968 >> 172.21.15.156:0/960064968 conn(0x7f1eb40b3080 msgr2=0x7f1eb40cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:16.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.054+0000 7f1ebbc73700 1 -- 172.21.15.156:0/960064968 shutdown_connections 2023-11-28T11:12:16.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.054+0000 7f1ebbc73700 1 -- 172.21.15.156:0/960064968 wait complete. 2023-11-28T11:12:16.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.055+0000 7f1ebbc73700 1 Processor -- start 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.056+0000 7f1ebbc73700 1 -- start start 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.056+0000 7f1ebbc73700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1eb40cc570 0x7f1eb40c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.056+0000 7f1ebbc73700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb40d0120 0x7f1eb40991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.056+0000 7f1ebbc73700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1eb415c6e0 0x7f1eb40c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.056+0000 7f1ebbc73700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40c75f0 con 0x7f1eb40d0120 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.056+0000 7f1ebbc73700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb409aa30 con 0x7f1eb40cc570 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.056+0000 7f1ebbc73700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb409ad10 con 0x7f1eb415c6e0 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.056+0000 7f1eb920e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb40d0120 0x7f1eb40991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.056+0000 7f1eb920e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb40d0120 0x7f1eb40991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36840/0 (socket says 172.21.15.156:36840) 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.056+0000 7f1eb920e700 1 -- 172.21.15.156:0/4214637717 learned_addr learned my addr 172.21.15.156:0/4214637717 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1eb9a0f700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1eb40cc570 0x7f1eb40c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1eba210700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1eb415c6e0 0x7f1eb40c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1eb920e700 1 -- 172.21.15.156:0/4214637717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1eb415c6e0 msgr2=0x7f1eb40c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1eb920e700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1eb415c6e0 0x7f1eb40c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1eb920e700 1 -- 172.21.15.156:0/4214637717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1eb40cc570 msgr2=0x7f1eb40c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1eb920e700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1eb40cc570 0x7f1eb40c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1eb920e700 1 -- 172.21.15.156:0/4214637717 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea400b7f0 con 0x7f1eb40d0120 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1eba210700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1eb415c6e0 0x7f1eb40c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:16.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1eb920e700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb40d0120 0x7f1eb40991b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c550 tx=0x7f1eb000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1eaa7fc700 1 -- 172.21.15.156:0/4214637717 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0004c80 con 0x7f1eb40d0120 2023-11-28T11:12:16.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1eaa7fc700 1 -- 172.21.15.156:0/4214637717 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0004e00 con 0x7f1eb40d0120 2023-11-28T11:12:16.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1ebbc73700 1 -- 172.21.15.156:0/4214637717 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb406ae90 con 0x7f1eb40d0120 2023-11-28T11:12:16.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1eaa7fc700 1 -- 172.21.15.156:0/4214637717 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0007310 con 0x7f1eb40d0120 2023-11-28T11:12:16.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.057+0000 7f1ebbc73700 1 -- 172.21.15.156:0/4214637717 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb406abb0 con 0x7f1eb40d0120 2023-11-28T11:12:16.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.058+0000 7f1ebbc73700 1 -- 172.21.15.156:0/4214637717 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1eb40b5240 con 0x7f1eb40d0120 2023-11-28T11:12:16.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.058+0000 7f1eaa7fc700 1 -- 172.21.15.156:0/4214637717 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1eb001a020 con 0x7f1eb40d0120 2023-11-28T11:12:16.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.059+0000 7f1eaa7fc700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1e980682f0 0x7f1e9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.059+0000 7f1eaa7fc700 1 -- 172.21.15.156:0/4214637717 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1eb0085e30 con 0x7f1eb40d0120 2023-11-28T11:12:16.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.059+0000 7f1eb9a0f700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1e980682f0 0x7f1e9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.059+0000 7f1eaa7fc700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x7f1e9806e760 0x7f1e98070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.059+0000 7f1eaa7fc700 1 -- 172.21.15.156:0/4214637717 --> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e98071240 con 0x7f1e9806e760 2023-11-28T11:12:16.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.059+0000 7f1eaa7fc700 1 -- 172.21.15.156:0/4214637717 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1eb0086270 con 0x7f1eb40d0120 2023-11-28T11:12:16.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.059+0000 7f1eba210700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x7f1e9806e760 0x7f1e98070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.059+0000 7f1eb9a0f700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1e980682f0 0x7f1e9806a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400b340 tx=0x7f1ea4013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.060+0000 7f1eba210700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x7f1e9806e760 0x7f1e98070b70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.060+0000 7f1eaa7fc700 1 -- 172.21.15.156:0/4214637717 <== osd.7 v2:172.21.15.186:6816/103780 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1e98071240 con 0x7f1e9806e760 2023-11-28T11:12:16.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.074+0000 7f1ebbc73700 1 -- 172.21.15.156:0/4214637717 --> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1eb409b200 con 0x7f1e9806e760 2023-11-28T11:12:16.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.075+0000 7f1eaa7fc700 1 -- 172.21.15.156:0/4214637717 <== osd.7 v2:172.21.15.186:6816/103780 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1eb409b200 con 0x7f1e9806e760 2023-11-28T11:12:16.077 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T11:12:16.077 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:12:16.077 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T11:12:16.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.075+0000 7f1ebbc73700 1 -- 172.21.15.156:0/4214637717 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x7f1e9806e760 msgr2=0x7f1e98070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.075+0000 7f1ebbc73700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.186:6816/103780,v1:172.21.15.186:6817/103780] conn(0x7f1e9806e760 0x7f1e98070b70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.075+0000 7f1ebbc73700 1 -- 172.21.15.156:0/4214637717 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1e980682f0 msgr2=0x7f1e9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.075+0000 7f1ebbc73700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f1e980682f0 0x7f1e9806a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400b340 tx=0x7f1ea4013000 comp rx=0 tx=0).stop 2023-11-28T11:12:16.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.075+0000 7f1ebbc73700 1 -- 172.21.15.156:0/4214637717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb40d0120 msgr2=0x7f1eb40991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.075+0000 7f1ebbc73700 1 --2- 172.21.15.156:0/4214637717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb40d0120 0x7f1eb40991b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c550 tx=0x7f1eb000c910 comp rx=0 tx=0).stop 2023-11-28T11:12:16.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.075+0000 7f1eba210700 1 -- 172.21.15.156:0/4214637717 reap_dead start 2023-11-28T11:12:16.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.075+0000 7f1ebbc73700 1 -- 172.21.15.156:0/4214637717 shutdown_connections 2023-11-28T11:12:16.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.075+0000 7f1ebbc73700 1 -- 172.21.15.156:0/4214637717 >> 172.21.15.156:0/4214637717 conn(0x7f1eb40b3080 msgr2=0x7f1eb40cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:16.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.075+0000 7f1ebbc73700 1 -- 172.21.15.156:0/4214637717 shutdown_connections 2023-11-28T11:12:16.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.076+0000 7f1ebbc73700 1 -- 172.21.15.156:0/4214637717 wait complete. 2023-11-28T11:12:16.088 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-28T11:12:16.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.208+0000 7ff05759b700 1 Processor -- start 2023-11-28T11:12:16.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.208+0000 7ff05759b700 1 -- start start 2023-11-28T11:12:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.209+0000 7ff05759b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff050158ce0 0x7ff05015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.209+0000 7ff05759b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0500ce4b0 0x7ff05006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.209+0000 7ff05759b700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05006bf10 con 0x7ff05006fb30 2023-11-28T11:12:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.209+0000 7ff05759b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05015d730 con 0x7ff0500ce4b0 2023-11-28T11:12:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.209+0000 7ff05759b700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0501590b0 con 0x7ff050158ce0 2023-11-28T11:12:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.209+0000 7ff055b38700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0500ce4b0 0x7ff05006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.209+0000 7ff055b38700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0500ce4b0 0x7ff05006c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36852/0 (socket says 172.21.15.156:36852) 2023-11-28T11:12:16.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.209+0000 7ff055b38700 1 -- 172.21.15.156:0/572736391 learned_addr learned my addr 172.21.15.156:0/572736391 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:16.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff055337700 1 --2- 172.21.15.156:0/572736391 >> v2:172.21.15.186:3300/0 conn(0x7ff050158ce0 0x7ff05015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff055b38700 1 -- 172.21.15.156:0/572736391 >> v2:172.21.15.186:3300/0 conn(0x7ff050158ce0 msgr2=0x7ff05015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:16.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff055b38700 1 --2- 172.21.15.156:0/572736391 >> v2:172.21.15.186:3300/0 conn(0x7ff050158ce0 0x7ff05015d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff055b38700 1 -- 172.21.15.156:0/572736391 >> v1:172.21.15.186:6789/0 conn(0x7ff05006fb30 legacy=0x7ff0500cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:16.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff055b38700 1 -- 172.21.15.156:0/572736391 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0500cb0d0 con 0x7ff0500ce4b0 2023-11-28T11:12:16.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff055337700 1 --2- 172.21.15.156:0/572736391 >> v2:172.21.15.186:3300/0 conn(0x7ff050158ce0 0x7ff05015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:16.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff055b38700 1 --2- 172.21.15.156:0/572736391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0500ce4b0 0x7ff05006c3a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff05006fa30 tx=0x7ff04800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962814d6ea4270b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff047fff700 1 -- 172.21.15.156:0/572736391 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048010070 con 0x7ff0500ce4b0 2023-11-28T11:12:16.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff047fff700 1 -- 172.21.15.156:0/572736391 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04800db40 con 0x7ff0500ce4b0 2023-11-28T11:12:16.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff047fff700 1 -- 172.21.15.156:0/572736391 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048003a20 con 0x7ff0500ce4b0 2023-11-28T11:12:16.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff05759b700 1 -- 172.21.15.156:0/572736391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0500ce4b0 msgr2=0x7ff05006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff05759b700 1 --2- 172.21.15.156:0/572736391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0500ce4b0 0x7ff05006c3a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff05006fa30 tx=0x7ff04800d430 comp rx=0 tx=0).stop 2023-11-28T11:12:16.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff05759b700 1 -- 172.21.15.156:0/572736391 shutdown_connections 2023-11-28T11:12:16.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff05759b700 1 --2- 172.21.15.156:0/572736391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0500ce4b0 0x7ff05006c3a0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff05759b700 1 --2- 172.21.15.156:0/572736391 >> v2:172.21.15.186:3300/0 conn(0x7ff050158ce0 0x7ff05015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.210+0000 7ff05759b700 1 -- 172.21.15.156:0/572736391 >> 172.21.15.156:0/572736391 conn(0x7ff0500cb670 msgr2=0x7ff0500cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:16.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.211+0000 7ff05759b700 1 -- 172.21.15.156:0/572736391 shutdown_connections 2023-11-28T11:12:16.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.211+0000 7ff05759b700 1 -- 172.21.15.156:0/572736391 wait complete. 2023-11-28T11:12:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.212+0000 7ff05759b700 1 Processor -- start 2023-11-28T11:12:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.212+0000 7ff05759b700 1 -- start start 2023-11-28T11:12:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.212+0000 7ff05759b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff05006fb30 0x7ff05015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.212+0000 7ff05759b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0500ce4b0 0x7ff050156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.212+0000 7ff05759b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff050158ce0 0x7ff05015bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.212+0000 7ff05759b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05015c2d0 con 0x7ff050158ce0 2023-11-28T11:12:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.212+0000 7ff05759b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05015b2f0 con 0x7ff0500ce4b0 2023-11-28T11:12:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.212+0000 7ff05759b700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0501ee350 con 0x7ff05006fb30 2023-11-28T11:12:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff055b38700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff050158ce0 0x7ff05015bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff055b38700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff050158ce0 0x7ff05015bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36858/0 (socket says 172.21.15.156:36858) 2023-11-28T11:12:16.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff055b38700 1 -- 172.21.15.156:0/3647119383 learned_addr learned my addr 172.21.15.156:0/3647119383 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:16.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff054b36700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0500ce4b0 0x7ff050156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff055b38700 1 -- 172.21.15.156:0/3647119383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff05006fb30 msgr2=0x7ff05015ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff055337700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff05006fb30 0x7ff05015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff055b38700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff05006fb30 0x7ff05015ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff055b38700 1 -- 172.21.15.156:0/3647119383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0500ce4b0 msgr2=0x7ff050156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff055b38700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0500ce4b0 0x7ff050156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff055b38700 1 -- 172.21.15.156:0/3647119383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04800bd40 con 0x7ff050158ce0 2023-11-28T11:12:16.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff054b36700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0500ce4b0 0x7ff050156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:16.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff055337700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff05006fb30 0x7ff05015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:16.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff055b38700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff050158ce0 0x7ff05015bd90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff04800d470 tx=0x7ff048004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff045ffb700 1 -- 172.21.15.156:0/3647119383 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048021670 con 0x7ff050158ce0 2023-11-28T11:12:16.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff045ffb700 1 -- 172.21.15.156:0/3647119383 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0480217f0 con 0x7ff050158ce0 2023-11-28T11:12:16.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff05759b700 1 -- 172.21.15.156:0/3647119383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05006b8c0 con 0x7ff050158ce0 2023-11-28T11:12:16.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff045ffb700 1 -- 172.21.15.156:0/3647119383 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048021ba0 con 0x7ff050158ce0 2023-11-28T11:12:16.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.213+0000 7ff05759b700 1 -- 172.21.15.156:0/3647119383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0500cb3b0 con 0x7ff050158ce0 2023-11-28T11:12:16.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.214+0000 7ff05759b700 1 -- 172.21.15.156:0/3647119383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05006bba0 con 0x7ff050158ce0 2023-11-28T11:12:16.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.215+0000 7ff045ffb700 1 -- 172.21.15.156:0/3647119383 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff048014070 con 0x7ff050158ce0 2023-11-28T11:12:16.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.215+0000 7ff045ffb700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0340682f0 0x7ff03406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.215+0000 7ff045ffb700 1 -- 172.21.15.156:0/3647119383 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff048089910 con 0x7ff050158ce0 2023-11-28T11:12:16.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.215+0000 7ff055337700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0340682f0 0x7ff03406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.216+0000 7ff055337700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0340682f0 0x7ff03406a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff0500c58f0 tx=0x7ff04c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.220+0000 7ff045ffb700 1 -- 172.21.15.156:0/3647119383 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff048056470 con 0x7ff050158ce0 2023-11-28T11:12:16.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.402+0000 7ff05759b700 1 -- 172.21.15.156:0/3647119383 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0500745c0 con 0x7ff0340682f0 2023-11-28T11:12:16.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.402+0000 7ff045ffb700 1 -- 172.21.15.156:0/3647119383 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff0500745c0 con 0x7ff0340682f0 2023-11-28T11:12:16.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.404+0000 7ff05759b700 1 -- 172.21.15.156:0/3647119383 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0340682f0 msgr2=0x7ff03406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.404+0000 7ff05759b700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0340682f0 0x7ff03406a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff0500c58f0 tx=0x7ff04c008040 comp rx=0 tx=0).stop 2023-11-28T11:12:16.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.404+0000 7ff05759b700 1 -- 172.21.15.156:0/3647119383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff050158ce0 msgr2=0x7ff05015bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.404+0000 7ff05759b700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff050158ce0 0x7ff05015bd90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff04800d470 tx=0x7ff048004ad0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.404+0000 7ff05759b700 1 -- 172.21.15.156:0/3647119383 shutdown_connections 2023-11-28T11:12:16.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.404+0000 7ff05759b700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff050158ce0 0x7ff05015bd90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.404+0000 7ff05759b700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7ff0340682f0 0x7ff03406a7a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.404+0000 7ff05759b700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff05006fb30 0x7ff05015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.404+0000 7ff05759b700 1 --2- 172.21.15.156:0/3647119383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0500ce4b0 0x7ff050156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.404+0000 7ff05759b700 1 -- 172.21.15.156:0/3647119383 >> 172.21.15.156:0/3647119383 conn(0x7ff0500cb670 msgr2=0x7ff0501580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:16.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.405+0000 7ff05759b700 1 -- 172.21.15.156:0/3647119383 shutdown_connections 2023-11-28T11:12:16.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.405+0000 7ff05759b700 1 -- 172.21.15.156:0/3647119383 wait complete. 2023-11-28T11:12:16.408 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 7 to deep-scrub 2023-11-28T11:12:16.418 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.537+0000 7fc913cae700 1 Processor -- start 2023-11-28T11:12:16.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.537+0000 7fc913cae700 1 -- start start 2023-11-28T11:12:16.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.538+0000 7fc913cae700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc90c06dee0 0x7fc90c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.538+0000 7fc913cae700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90c15d060 con 0x7fc90c160540 2023-11-28T11:12:16.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.538+0000 7fc913cae700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90c15d340 con 0x7fc90c071040 2023-11-28T11:12:16.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.538+0000 7fc913cae700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90c15d620 con 0x7fc90c06dee0 2023-11-28T11:12:16.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.538+0000 7fc911249700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc90c06dee0 0x7fc90c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.538+0000 7fc91224b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc90c160540 0x7fc90c160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51744/0 (socket says 172.21.15.156:51744) 2023-11-28T11:12:16.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.538+0000 7fc911249700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc90c06dee0 0x7fc90c06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36862/0 (socket says 172.21.15.156:36862) 2023-11-28T11:12:16.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.538+0000 7fc91224b700 1 -- 172.21.15.156:0/2477917464 learned_addr learned my addr 172.21.15.156:0/2477917464 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:16.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.538+0000 7fc911249700 1 -- 172.21.15.156:0/2477917464 >> v1:172.21.15.186:6789/0 conn(0x7fc90c071040 legacy=0x7fc90c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.538+0000 7fc911249700 1 -- 172.21.15.156:0/2477917464 >> v1:172.21.15.156:6789/0 conn(0x7fc90c160540 legacy=0x7fc90c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.538+0000 7fc911249700 1 -- 172.21.15.156:0/2477917464 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c15d900 con 0x7fc90c06dee0 2023-11-28T11:12:16.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.538+0000 7fc910a48700 1 -- 172.21.15.156:0/2477917464 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331361589 0 0) 0x7fc90c15d060 con 0x7fc90c160540 2023-11-28T11:12:16.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.539+0000 7fc911249700 1 --2- 172.21.15.156:0/2477917464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c06dee0 0x7fc90c06e330 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc908009840 tx=0x7fc908009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ddea3b39058f76 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.539+0000 7fc910a48700 1 -- 172.21.15.156:0/2477917464 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908011830 con 0x7fc90c06dee0 2023-11-28T11:12:16.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.539+0000 7fc910a48700 1 -- 172.21.15.156:0/2477917464 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc908011990 con 0x7fc90c06dee0 2023-11-28T11:12:16.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.539+0000 7fc910a48700 1 -- 172.21.15.156:0/2477917464 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908011d20 con 0x7fc90c06dee0 2023-11-28T11:12:16.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.539+0000 7fc913cae700 1 -- 172.21.15.156:0/2477917464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c06dee0 msgr2=0x7fc90c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.539+0000 7fc913cae700 1 --2- 172.21.15.156:0/2477917464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c06dee0 0x7fc90c06e330 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc908009840 tx=0x7fc908009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:16.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.539+0000 7fc913cae700 1 -- 172.21.15.156:0/2477917464 shutdown_connections 2023-11-28T11:12:16.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.539+0000 7fc913cae700 1 --2- 172.21.15.156:0/2477917464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c06dee0 0x7fc90c06e330 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.539+0000 7fc913cae700 1 -- 172.21.15.156:0/2477917464 >> 172.21.15.156:0/2477917464 conn(0x7fc90c0d0000 msgr2=0x7fc90c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:16.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.539+0000 7fc913cae700 1 -- 172.21.15.156:0/2477917464 shutdown_connections 2023-11-28T11:12:16.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.539+0000 7fc913cae700 1 -- 172.21.15.156:0/2477917464 wait complete. 2023-11-28T11:12:16.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.541+0000 7fc913cae700 1 Processor -- start 2023-11-28T11:12:16.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.541+0000 7fc913cae700 1 -- start start 2023-11-28T11:12:16.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.541+0000 7fc913cae700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc90c06dee0 0x7fc90c153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.541+0000 7fc913cae700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c071040 0x7fc90c0b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.541+0000 7fc913cae700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc90c160540 0x7fc90c15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.541+0000 7fc913cae700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c159120 con 0x7fc90c071040 2023-11-28T11:12:16.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.541+0000 7fc913cae700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0b0920 con 0x7fc90c06dee0 2023-11-28T11:12:16.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.541+0000 7fc913cae700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0b0bd0 con 0x7fc90c160540 2023-11-28T11:12:16.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.541+0000 7fc911249700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c071040 0x7fc90c0b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.541+0000 7fc911249700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c071040 0x7fc90c0b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36870/0 (socket says 172.21.15.156:36870) 2023-11-28T11:12:16.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.541+0000 7fc911249700 1 -- 172.21.15.156:0/530877298 learned_addr learned my addr 172.21.15.156:0/530877298 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:16.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc911249700 1 -- 172.21.15.156:0/530877298 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc90c160540 msgr2=0x7fc90c15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc911249700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc90c160540 0x7fc90c15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc911a4a700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc90c06dee0 0x7fc90c153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc911249700 1 -- 172.21.15.156:0/530877298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc90c06dee0 msgr2=0x7fc90c153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc911249700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc90c06dee0 0x7fc90c153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc911249700 1 -- 172.21.15.156:0/530877298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9080094a0 con 0x7fc90c071040 2023-11-28T11:12:16.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc911a4a700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc90c06dee0 0x7fc90c153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:16.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc911249700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c071040 0x7fc90c0b03e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc908014040 tx=0x7fc908004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc9027fc700 1 -- 172.21.15.156:0/530877298 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90801a7e0 con 0x7fc90c071040 2023-11-28T11:12:16.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc9027fc700 1 -- 172.21.15.156:0/530877298 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90801a940 con 0x7fc90c071040 2023-11-28T11:12:16.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc913cae700 1 -- 172.21.15.156:0/530877298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c15ccd0 con 0x7fc90c071040 2023-11-28T11:12:16.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc9027fc700 1 -- 172.21.15.156:0/530877298 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90801acd0 con 0x7fc90c071040 2023-11-28T11:12:16.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.542+0000 7fc913cae700 1 -- 172.21.15.156:0/530877298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c06e870 con 0x7fc90c071040 2023-11-28T11:12:16.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.543+0000 7fc913cae700 1 -- 172.21.15.156:0/530877298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90c06a270 con 0x7fc90c071040 2023-11-28T11:12:16.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.543+0000 7fc9027fc700 1 -- 172.21.15.156:0/530877298 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc90802b030 con 0x7fc90c071040 2023-11-28T11:12:16.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.544+0000 7fc9027fc700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc8f00682f0 0x7fc8f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.544+0000 7fc9027fc700 1 -- 172.21.15.156:0/530877298 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc908086c40 con 0x7fc90c071040 2023-11-28T11:12:16.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.544+0000 7fc911a4a700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc8f00682f0 0x7fc8f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.544+0000 7fc911a4a700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc8f00682f0 0x7fc8f006a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000f60 tx=0x7fc8fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.548+0000 7fc9027fc700 1 -- 172.21.15.156:0/530877298 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc90808c020 con 0x7fc90c071040 2023-11-28T11:12:16.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.726+0000 7fc913cae700 1 -- 172.21.15.156:0/530877298 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc90c0720a0 con 0x7fc8f00682f0 2023-11-28T11:12:16.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.729+0000 7fc9027fc700 1 -- 172.21.15.156:0/530877298 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72789 (secure 0 0 0) 0x7fc90c0720a0 con 0x7fc8f00682f0 2023-11-28T11:12:16.731 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:16.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.731+0000 7fc913cae700 1 -- 172.21.15.156:0/530877298 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc8f00682f0 msgr2=0x7fc8f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.731+0000 7fc913cae700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc8f00682f0 0x7fc8f006a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000f60 tx=0x7fc8fc009040 comp rx=0 tx=0).stop 2023-11-28T11:12:16.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.731+0000 7fc913cae700 1 -- 172.21.15.156:0/530877298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c071040 msgr2=0x7fc90c0b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.731+0000 7fc913cae700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c071040 0x7fc90c0b03e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc908014040 tx=0x7fc908004750 comp rx=0 tx=0).stop 2023-11-28T11:12:16.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.731+0000 7fc913cae700 1 -- 172.21.15.156:0/530877298 shutdown_connections 2023-11-28T11:12:16.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.731+0000 7fc913cae700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fc8f00682f0 0x7fc8f006a7a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.731+0000 7fc913cae700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc90c160540 0x7fc90c15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.731+0000 7fc913cae700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc90c06dee0 0x7fc90c153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.731+0000 7fc913cae700 1 --2- 172.21.15.156:0/530877298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c071040 0x7fc90c0b03e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.731+0000 7fc913cae700 1 -- 172.21.15.156:0/530877298 >> 172.21.15.156:0/530877298 conn(0x7fc90c0d0000 msgr2=0x7fc90c15f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:16.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.731+0000 7fc913cae700 1 -- 172.21.15.156:0/530877298 shutdown_connections 2023-11-28T11:12:16.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:16.731+0000 7fc913cae700 1 -- 172.21.15.156:0/530877298 wait complete. 2023-11-28T11:12:16.734 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:12:16.745 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2023-11-28T11:12:16.579309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":93128,"kb_used_data":7624,"kb_used_omap":0,"kb_used_meta":85440,"kb_avail":754881592,"statfs":{"total":773094113280,"available":772998750208,"internally_reserved":0,"allocated":7806976,"data_stored":4109648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87490560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.613335"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.483444+0000","last_change":"2023-11-28T11:08:11.971084+0000","last_active":"2023-11-28T11:12:05.483444+0000","last_peered":"2023-11-28T11:12:05.483444+0000","last_clean":"2023-11-28T11:12:05.483444+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:12:05.483444+0000","last_undegraded":"2023-11-28T11:12:05.483444+0000","last_fullsized":"2023-11-28T11:12:05.483444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.968881+0000","last_change":"2023-11-28T11:08:11.954303+0000","last_active":"2023-11-28T11:12:06.968881+0000","last_peered":"2023-11-28T11:12:06.968881+0000","last_clean":"2023-11-28T11:12:06.968881+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:12:06.968881+0000","last_undegraded":"2023-11-28T11:12:06.968881+0000","last_fullsized":"2023-11-28T11:12:06.968881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:25.295892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:16.553010+0000","last_change":"2023-11-28T11:12:16.552989+0000","last_active":"2023-11-28T11:12:16.553010+0000","last_peered":"2023-11-28T11:12:16.553010+0000","last_clean":"2023-11-28T11:12:16.553010+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:12:16.553010+0000","last_undegraded":"2023-11-28T11:12:16.553010+0000","last_fullsized":"2023-11-28T11:12:16.553010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:16.552895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:16.552895+0000","last_clean_scrub_stamp":"2023-11-28T11:12:16.552895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:27:14.331572+0000","scrub_duration":0.0034665149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.968394+0000","last_change":"2023-11-28T11:08:11.953777+0000","last_active":"2023-11-28T11:12:06.968394+0000","last_peered":"2023-11-28T11:12:06.968394+0000","last_clean":"2023-11-28T11:12:06.968394+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:12:06.968394+0000","last_undegraded":"2023-11-28T11:12:06.968394+0000","last_fullsized":"2023-11-28T11:12:06.968394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":210,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:11.565023+0000","last_change":"2023-11-28T11:08:08.109924+0000","last_active":"2023-11-28T11:12:11.565023+0000","last_peered":"2023-11-28T11:12:11.565023+0000","last_clean":"2023-11-28T11:12:11.565023+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:12:11.565023+0000","last_undegraded":"2023-11-28T11:12:11.565023+0000","last_fullsized":"2023-11-28T11:12:11.565023+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:06.915564+0000","last_clean_scrub_stamp":"2023-11-28T11:08:06.915564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:11.565000+0000","last_change":"2023-11-28T11:08:11.955456+0000","last_active":"2023-11-28T11:12:11.565000+0000","last_peered":"2023-11-28T11:12:11.565000+0000","last_clean":"2023-11-28T11:12:11.565000+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:12:11.565000+0000","last_undegraded":"2023-11-28T11:12:11.565000+0000","last_fullsized":"2023-11-28T11:12:11.565000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:08.060570+0000","last_change":"2023-11-28T11:08:11.971230+0000","last_active":"2023-11-28T11:12:08.060570+0000","last_peered":"2023-11-28T11:12:08.060570+0000","last_clean":"2023-11-28T11:12:08.060570+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:12:08.060570+0000","last_undegraded":"2023-11-28T11:12:08.060570+0000","last_fullsized":"2023-11-28T11:12:08.060570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.483296+0000","last_change":"2023-11-28T11:08:11.958429+0000","last_active":"2023-11-28T11:12:05.483296+0000","last_peered":"2023-11-28T11:12:05.483296+0000","last_clean":"2023-11-28T11:12:05.483296+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:12:05.483296+0000","last_undegraded":"2023-11-28T11:12:05.483296+0000","last_fullsized":"2023-11-28T11:12:05.483296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:03.543520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:07.987719+0000","last_change":"2023-11-28T11:08:11.958501+0000","last_active":"2023-11-28T11:12:07.987719+0000","last_peered":"2023-11-28T11:12:07.987719+0000","last_clean":"2023-11-28T11:12:07.987719+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:12:07.987719+0000","last_undegraded":"2023-11-28T11:12:07.987719+0000","last_fullsized":"2023-11-28T11:12:07.987719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:10.930752+0000","last_clean_scrub_stamp":"2023-11-28T11:08:10.930752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11124,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":9728,"kb_avail":94360716,"statfs":{"total":96636764160,"available":96625373184,"internally_reserved":0,"allocated":1421312,"data_stored":956482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9961472},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.32000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55200000000000005,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":1.006,"15min":1.006},"last":0.60199999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67100000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.46800000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60799999999999998,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.0640000000000001,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.496},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.44700000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71399999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.65600000000000003,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.751,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71199999999999997,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.53500000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.57399999999999995,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78500000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61699999999999999,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84699999999999998,"5min":1.018,"15min":1.018},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.435,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":1.123,"15min":1.123},"last":0.63}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9584,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362256,"statfs":{"total":96636764160,"available":96626950144,"internally_reserved":0,"allocated":827392,"data_stored":366114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.62,"15min":0.71899999999999997},"min":{"1min":0.36799999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70299999999999996,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.63100000000000001,"15min":0.70999999999999996},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.54400000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.69299999999999995,"15min":0.81899999999999995},"min":{"1min":0.435,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70099999999999996,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.65700000000000003,"15min":0.76700000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68700000000000006,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.34799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.65800000000000003,"15min":0.79200000000000004},"min":{"1min":0.376,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69999999999999996,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.62},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.625,"15min":0.69499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67100000000000004,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.51200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.39900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72399999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60699999999999998,"15min":0.66100000000000003},"min":{"1min":0.37,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67400000000000004,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.45900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.51000000000000001,"15min":0.52300000000000002},"min":{"1min":0.33900000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.58699999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.55300000000000005,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.63700000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.42999999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59899999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.505,"5min":0.58399999999999996,"15min":0.59899999999999998},"min":{"1min":0.41499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74099999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76600000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.41699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68700000000000006,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13936,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357904,"statfs":{"total":96636764160,"available":96622493696,"internally_reserved":0,"allocated":827392,"data_stored":366114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.53800000000000003,"15min":0.626},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.59499999999999997,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.371,"15min":0.38},"min":{"1min":0.247,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56100000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.26600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.57499999999999996,"15min":0.66700000000000004},"min":{"1min":0.26300000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.48599999999999999,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.372},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40600000000000003,"15min":0.41599999999999998},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.53000000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28299999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56399999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.377,"15min":0.375},"min":{"1min":0.313,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51500000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.307}]},{"osd":4,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":1.139,"15min":1.373},"min":{"1min":0.40600000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79800000000000004,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86799999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45000000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80200000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":1.1679999999999999,"15min":1.403},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85299999999999998,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.5}]},{"osd":6,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":1.1719999999999999,"15min":1.4079999999999999},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.433,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.62},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88700000000000001,"5min":1.018,"15min":1.018},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.84699999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.47499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14964,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356876,"statfs":{"total":96636764160,"available":96621441024,"internally_reserved":0,"allocated":1421312,"data_stored":956482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35099999999999998,"15min":0.35199999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.502,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.311},{"interface":"front","average":{"1min":0.374,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.24199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.26300000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.40000000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.52000000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.379,"15min":0.378},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.59199999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.35599999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.376,"15min":0.37},"min":{"1min":0.30199999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58299999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.41799999999999998,"15min":0.42199999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61299999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.377}]},{"osd":4,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.47699999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79900000000000004,"5min":0.879,"15min":0.879},"last":0.748},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.47699999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86699999999999999,"5min":0.878,"15min":0.878},"last":0.56899999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58399999999999996,"15min":0.56100000000000005},"min":{"1min":0.45100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.627,"15min":0.626},"min":{"1min":0.496,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.59699999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.497,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.48499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88200000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58299999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.629,"15min":0.623},"min":{"1min":0.51400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.83299999999999996,"5min":1.026,"15min":1.026},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.52800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.60399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9712,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362128,"statfs":{"total":96636764160,"available":96626819072,"internally_reserved":0,"allocated":827392,"data_stored":366114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.31900000000000001},"min":{"1min":0.23200000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54500000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.38600000000000001,"15min":0.37},"min":{"1min":0.312,"5min":0.245,"15min":0.245},"max":{"1min":0.77400000000000002,"5min":1.131,"15min":1.131},"last":0.36599999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.33900000000000002},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58099999999999996,"5min":1.151,"15min":1.151},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.35999999999999999,"15min":0.34100000000000003},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.74099999999999999,"5min":1.177,"15min":1.177},"last":0.33600000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34399999999999997,"15min":0.33500000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.50900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.309},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39300000000000002,"15min":0.373},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59399999999999997,"5min":1.109,"15min":1.109},"last":0.38400000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71199999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.47499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":1.502,"15min":1.502},"last":0.64600000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79000000000000004,"5min":1.323,"15min":1.323},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.65200000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.503,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86199999999999999,"5min":1.464,"15min":1.464},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":1.355,"15min":1.355},"last":0.57299999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.50900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82499999999999996,"5min":1.421,"15min":1.421},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.443,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81200000000000006,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.60499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14064,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357776,"statfs":{"total":96636764160,"available":96622362624,"internally_reserved":0,"allocated":827392,"data_stored":366114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33800000000000002,"15min":0.34300000000000003},"min":{"1min":0.248,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.5,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.32200000000000001},"min":{"1min":0.27000000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44900000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.376}]},{"osd":1,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.25700000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47399999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39100000000000001,"15min":0.40500000000000003},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.45900000000000002,"5min":1.365,"15min":1.365},"last":0.34899999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38,"15min":0.373},"min":{"1min":0.32400000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48799999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.40400000000000003},"min":{"1min":0.34000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56799999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.38800000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.41399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77900000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54000000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.75600000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.56799999999999995},"min":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.63600000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79600000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.439,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":9968,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361872,"statfs":{"total":96636764160,"available":96626556928,"internally_reserved":0,"allocated":827392,"data_stored":366114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.629,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.55600000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":4.9740000000000002,"15min":4.9740000000000002},"last":0.57499999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.81200000000000006,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.97699999999999998,"15min":1.161},"min":{"1min":0.44700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":12.263,"15min":12.263},"last":0.72999999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80300000000000005,"5min":1.524,"15min":1.524},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76100000000000001,"5min":1.236,"15min":1.236},"last":0.55900000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56200000000000006,"15min":0.54300000000000004},"min":{"1min":0.44500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74299999999999999,"5min":1.54,"15min":1.54},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.69599999999999995,"15min":0.72199999999999998},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.85799999999999998,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59799999999999998,"15min":0.55100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95499999999999996,"5min":5.0220000000000002,"15min":5.0220000000000002},"last":0.47799999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.93600000000000005,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":1.923,"15min":1.923},"last":0.51400000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82399999999999995,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.996,"5min":1.49,"15min":1.49},"last":0.499}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9776,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362064,"statfs":{"total":96636764160,"available":96626753536,"internally_reserved":0,"allocated":827392,"data_stored":366114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.501},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40799999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69599999999999995,"5min":0.75,"15min":0.75},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.437}]},{"osd":2,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.495,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.54800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46300000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.502,"15min":0.499},"min":{"1min":0.36899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60299999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.42999999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:16.748 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-28T11:08:10.930752+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:16.749 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-28T11:08:10.930752+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:16.749 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-28T11:08:10.930752+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:16.749 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-28T11:08:06.915564+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:16.749 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T11:08:10.930752+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:16.749 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-28T11:08:10.930752+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:16.750 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-28T11:08:10.930752+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:16.750 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-28T11:08:10.930752+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=12, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:12:16.750 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T11:12:21.673 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:21.676 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:36.754 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:36.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.891+0000 7f747a34f700 1 Processor -- start 2023-11-28T11:12:36.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.891+0000 7f747a34f700 1 -- start start 2023-11-28T11:12:36.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f747a34f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f747406dee0 0x7f747406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f747a34f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7474160540 0x7f74741609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f747a34f700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f747415d060 con 0x7f7474071040 2023-11-28T11:12:36.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f747a34f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747415d340 con 0x7f747406dee0 2023-11-28T11:12:36.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f747a34f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747415d620 con 0x7f7474160540 2023-11-28T11:12:36.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f74737fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f747406dee0 0x7f747406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f74737fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f747406dee0 0x7f747406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36878/0 (socket says 172.21.15.156:36878) 2023-11-28T11:12:36.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f74737fe700 1 -- 172.21.15.156:0/1012290641 learned_addr learned my addr 172.21.15.156:0/1012290641 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:36.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f74788ec700 1 --2- 172.21.15.156:0/1012290641 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7474160540 0x7f74741609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f74737fe700 1 -- 172.21.15.156:0/1012290641 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7474160540 msgr2=0x7f74741609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:36.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f74737fe700 1 --2- 172.21.15.156:0/1012290641 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7474160540 0x7f74741609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f74737fe700 1 -- 172.21.15.156:0/1012290641 >> v1:172.21.15.186:6789/0 conn(0x7f7474071040 legacy=0x7f7474071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:36.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f74737fe700 1 -- 172.21.15.156:0/1012290641 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747415d900 con 0x7f747406dee0 2023-11-28T11:12:36.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.892+0000 7f74737fe700 1 --2- 172.21.15.156:0/1012290641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f747406dee0 0x7f747406e330 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f74640098f0 tx=0x7f7464009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9901840e0131152 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:36.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.893+0000 7f7472ffd700 1 -- 172.21.15.156:0/1012290641 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746400b8a0 con 0x7f747406dee0 2023-11-28T11:12:36.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.893+0000 7f7472ffd700 1 -- 172.21.15.156:0/1012290641 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746400ba00 con 0x7f747406dee0 2023-11-28T11:12:36.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.893+0000 7f7472ffd700 1 -- 172.21.15.156:0/1012290641 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746400bd90 con 0x7f747406dee0 2023-11-28T11:12:36.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.893+0000 7f747a34f700 1 -- 172.21.15.156:0/1012290641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f747406dee0 msgr2=0x7f747406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.893+0000 7f747a34f700 1 --2- 172.21.15.156:0/1012290641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f747406dee0 0x7f747406e330 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f74640098f0 tx=0x7f7464009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:36.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.893+0000 7f747a34f700 1 -- 172.21.15.156:0/1012290641 shutdown_connections 2023-11-28T11:12:36.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.893+0000 7f747a34f700 1 --2- 172.21.15.156:0/1012290641 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7474160540 0x7f74741609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.893+0000 7f747a34f700 1 --2- 172.21.15.156:0/1012290641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f747406dee0 0x7f747406e330 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.893+0000 7f747a34f700 1 -- 172.21.15.156:0/1012290641 >> 172.21.15.156:0/1012290641 conn(0x7f74740d0000 msgr2=0x7f74740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:36.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.893+0000 7f747a34f700 1 -- 172.21.15.156:0/1012290641 shutdown_connections 2023-11-28T11:12:36.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.893+0000 7f747a34f700 1 -- 172.21.15.156:0/1012290641 wait complete. 2023-11-28T11:12:36.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.894+0000 7f747a34f700 1 Processor -- start 2023-11-28T11:12:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.895+0000 7f747a34f700 1 -- start start 2023-11-28T11:12:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.895+0000 7f747a34f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f747406dee0 0x7f747415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.895+0000 7f747a34f700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7474071040 0x7f74740c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.895+0000 7f747a34f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7474160540 0x7f747415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.895+0000 7f747a34f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74740c3fe0 con 0x7f747406dee0 2023-11-28T11:12:36.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.895+0000 7f747a34f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74740c4e20 con 0x7f7474160540 2023-11-28T11:12:36.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.895+0000 7f747a34f700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74740c4210 con 0x7f7474071040 2023-11-28T11:12:36.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.895+0000 7f7473fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f747406dee0 0x7f747415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f7473fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f747406dee0 0x7f747415b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36882/0 (socket says 172.21.15.156:36882) 2023-11-28T11:12:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f7473fff700 1 -- 172.21.15.156:0/364632205 learned_addr learned my addr 172.21.15.156:0/364632205 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f74788ec700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7474160540 0x7f747415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f74737fe700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7474071040 0x7f74740c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f7473fff700 1 -- 172.21.15.156:0/364632205 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7474071040 msgr2=0x7f74740c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f7473fff700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7474071040 0x7f74740c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f7473fff700 1 -- 172.21.15.156:0/364632205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7474160540 msgr2=0x7f747415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f7473fff700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7474160540 0x7f747415f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f7473fff700 1 -- 172.21.15.156:0/364632205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7464009550 con 0x7f747406dee0 2023-11-28T11:12:36.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f74788ec700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7474160540 0x7f747415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:36.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f7473fff700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f747406dee0 0x7f747415b6d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f746800c450 tx=0x7f746800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:36.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f7470ff9700 1 -- 172.21.15.156:0/364632205 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74680052d0 con 0x7f747406dee0 2023-11-28T11:12:36.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f7470ff9700 1 -- 172.21.15.156:0/364632205 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7468005430 con 0x7f747406dee0 2023-11-28T11:12:36.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f747a34f700 1 -- 172.21.15.156:0/364632205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f747415ccd0 con 0x7f747406dee0 2023-11-28T11:12:36.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f7470ff9700 1 -- 172.21.15.156:0/364632205 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74680038c0 con 0x7f747406dee0 2023-11-28T11:12:36.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.896+0000 7f747a34f700 1 -- 172.21.15.156:0/364632205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747406e870 con 0x7f747406dee0 2023-11-28T11:12:36.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.897+0000 7f747a34f700 1 -- 172.21.15.156:0/364632205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747406a270 con 0x7f747406dee0 2023-11-28T11:12:36.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.898+0000 7f7470ff9700 1 -- 172.21.15.156:0/364632205 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7468019030 con 0x7f747406dee0 2023-11-28T11:12:36.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.898+0000 7f7470ff9700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74600682f0 0x7f746006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:36.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.898+0000 7f7470ff9700 1 -- 172.21.15.156:0/364632205 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7468084b80 con 0x7f747406dee0 2023-11-28T11:12:36.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.898+0000 7f74737fe700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74600682f0 0x7f746006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:36.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.898+0000 7f74737fe700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74600682f0 0x7f746006a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7464014040 tx=0x7f7464008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:36.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:36.902+0000 7f7470ff9700 1 -- 172.21.15.156:0/364632205 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f74680516e0 con 0x7f747406dee0 2023-11-28T11:12:37.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.083+0000 7f747a34f700 1 -- 172.21.15.156:0/364632205 --> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74740720a0 con 0x7f74600682f0 2023-11-28T11:12:37.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.086+0000 7f7470ff9700 1 -- 172.21.15.156:0/364632205 <== mgr.4107 v2:172.21.15.156:6832/103293 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73121 (secure 0 0 0) 0x7f74740720a0 con 0x7f74600682f0 2023-11-28T11:12:37.088 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:37.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.088+0000 7f747a34f700 1 -- 172.21.15.156:0/364632205 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74600682f0 msgr2=0x7f746006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.088+0000 7f747a34f700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74600682f0 0x7f746006a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7464014040 tx=0x7f7464008030 comp rx=0 tx=0).stop 2023-11-28T11:12:37.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.088+0000 7f747a34f700 1 -- 172.21.15.156:0/364632205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f747406dee0 msgr2=0x7f747415b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.088+0000 7f747a34f700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f747406dee0 0x7f747415b6d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f746800c450 tx=0x7f746800c810 comp rx=0 tx=0).stop 2023-11-28T11:12:37.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.088+0000 7f747a34f700 1 -- 172.21.15.156:0/364632205 shutdown_connections 2023-11-28T11:12:37.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.088+0000 7f747a34f700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7474160540 0x7f747415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.088+0000 7f747a34f700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f747406dee0 0x7f747415b6d0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.088+0000 7f747a34f700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7f74600682f0 0x7f746006a7a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.088+0000 7f747a34f700 1 --2- 172.21.15.156:0/364632205 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7474071040 0x7f74740c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.088+0000 7f747a34f700 1 -- 172.21.15.156:0/364632205 >> 172.21.15.156:0/364632205 conn(0x7f74740d0000 msgr2=0x7f74740cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:37.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.088+0000 7f747a34f700 1 -- 172.21.15.156:0/364632205 shutdown_connections 2023-11-28T11:12:37.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.088+0000 7f747a34f700 1 -- 172.21.15.156:0/364632205 wait complete. 2023-11-28T11:12:37.092 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T11:12:37.101 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2023-11-28T11:12:36.583331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":93128,"kb_used_data":7624,"kb_used_omap":0,"kb_used_meta":85440,"kb_avail":754881592,"statfs":{"total":773094113280,"available":772998750208,"internally_reserved":0,"allocated":7806976,"data_stored":4109648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87490560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002421"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":70,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:20.947490+0000","last_change":"2023-11-28T11:12:20.947450+0000","last_active":"2023-11-28T11:12:20.947490+0000","last_peered":"2023-11-28T11:12:20.947490+0000","last_clean":"2023-11-28T11:12:20.947490+0000","last_became_active":"2023-11-28T11:08:11.970735+0000","last_became_peered":"2023-11-28T11:08:11.970735+0000","last_unstale":"2023-11-28T11:12:20.947490+0000","last_undegraded":"2023-11-28T11:12:20.947490+0000","last_fullsized":"2023-11-28T11:12:20.947490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:20.947316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:20.947316+0000","last_clean_scrub_stamp":"2023-11-28T11:12:20.947316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:24:44.265735+0000","scrub_duration":0.0036445269999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":72,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:22.567834+0000","last_change":"2023-11-28T11:12:22.567792+0000","last_active":"2023-11-28T11:12:22.567834+0000","last_peered":"2023-11-28T11:12:22.567834+0000","last_clean":"2023-11-28T11:12:22.567834+0000","last_became_active":"2023-11-28T11:08:11.954039+0000","last_became_peered":"2023-11-28T11:08:11.954039+0000","last_unstale":"2023-11-28T11:12:22.567834+0000","last_undegraded":"2023-11-28T11:12:22.567834+0000","last_fullsized":"2023-11-28T11:12:22.567834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:22.567678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:22.567678+0000","last_clean_scrub_stamp":"2023-11-28T11:12:22.567678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:46.157097+0000","scrub_duration":0.0043053270000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":71,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:16.553020+0000","last_change":"2023-11-28T11:12:16.552989+0000","last_active":"2023-11-28T11:12:16.553020+0000","last_peered":"2023-11-28T11:12:16.553020+0000","last_clean":"2023-11-28T11:12:16.553020+0000","last_became_active":"2023-11-28T11:08:11.951863+0000","last_became_peered":"2023-11-28T11:08:11.951863+0000","last_unstale":"2023-11-28T11:12:16.553020+0000","last_undegraded":"2023-11-28T11:12:16.553020+0000","last_fullsized":"2023-11-28T11:12:16.553020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:16.552895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:16.552895+0000","last_clean_scrub_stamp":"2023-11-28T11:12:16.552895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:27:14.331572+0000","scrub_duration":0.0034665149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":72,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:16.654522+0000","last_change":"2023-11-28T11:12:16.654452+0000","last_active":"2023-11-28T11:12:16.654522+0000","last_peered":"2023-11-28T11:12:16.654522+0000","last_clean":"2023-11-28T11:12:16.654522+0000","last_became_active":"2023-11-28T11:08:11.953399+0000","last_became_peered":"2023-11-28T11:08:11.953399+0000","last_unstale":"2023-11-28T11:12:16.654522+0000","last_undegraded":"2023-11-28T11:12:16.654522+0000","last_fullsized":"2023-11-28T11:12:16.654522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:16.654324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:16.654324+0000","last_clean_scrub_stamp":"2023-11-28T11:12:16.654324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:41.196515+0000","scrub_duration":0.0035623880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"68'119","reported_seq":226,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:20.136708+0000","last_change":"2023-11-28T11:12:20.136650+0000","last_active":"2023-11-28T11:12:20.136708+0000","last_peered":"2023-11-28T11:12:20.136708+0000","last_clean":"2023-11-28T11:12:20.136708+0000","last_became_active":"2023-11-28T11:08:08.109027+0000","last_became_peered":"2023-11-28T11:08:08.109027+0000","last_unstale":"2023-11-28T11:12:20.136708+0000","last_undegraded":"2023-11-28T11:12:20.136708+0000","last_fullsized":"2023-11-28T11:12:20.136708+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'119","last_scrub_stamp":"2023-11-28T11:12:20.136532+0000","last_deep_scrub":"68'119","last_deep_scrub_stamp":"2023-11-28T11:12:20.136532+0000","last_clean_scrub_stamp":"2023-11-28T11:12:20.136532+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:43.725951+0000","scrub_duration":0.0070494629999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:24.107826+0000","last_change":"2023-11-28T11:12:24.107777+0000","last_active":"2023-11-28T11:12:24.107826+0000","last_peered":"2023-11-28T11:12:24.107826+0000","last_clean":"2023-11-28T11:12:24.107826+0000","last_became_active":"2023-11-28T11:08:11.955133+0000","last_became_peered":"2023-11-28T11:08:11.955133+0000","last_unstale":"2023-11-28T11:12:24.107826+0000","last_undegraded":"2023-11-28T11:12:24.107826+0000","last_fullsized":"2023-11-28T11:12:24.107826+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:24.107666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:24.107666+0000","last_clean_scrub_stamp":"2023-11-28T11:12:24.107666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:30:21.789960+0000","scrub_duration":0.0040658259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":71,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:23.085296+0000","last_change":"2023-11-28T11:12:23.085258+0000","last_active":"2023-11-28T11:12:23.085296+0000","last_peered":"2023-11-28T11:12:23.085296+0000","last_clean":"2023-11-28T11:12:23.085296+0000","last_became_active":"2023-11-28T11:08:11.970954+0000","last_became_peered":"2023-11-28T11:08:11.970954+0000","last_unstale":"2023-11-28T11:12:23.085296+0000","last_undegraded":"2023-11-28T11:12:23.085296+0000","last_fullsized":"2023-11-28T11:12:23.085296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:23.085143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:23.085143+0000","last_clean_scrub_stamp":"2023-11-28T11:12:23.085143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:17.008927+0000","scrub_duration":0.0040968539999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":70,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:15.015662+0000","last_change":"2023-11-28T11:12:15.015605+0000","last_active":"2023-11-28T11:12:15.015662+0000","last_peered":"2023-11-28T11:12:15.015662+0000","last_clean":"2023-11-28T11:12:15.015662+0000","last_became_active":"2023-11-28T11:08:11.958165+0000","last_became_peered":"2023-11-28T11:08:11.958165+0000","last_unstale":"2023-11-28T11:12:15.015662+0000","last_undegraded":"2023-11-28T11:12:15.015662+0000","last_fullsized":"2023-11-28T11:12:15.015662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:15.015493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:15.015493+0000","last_clean_scrub_stamp":"2023-11-28T11:12:15.015493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:39.557683+0000","scrub_duration":0.0039817460000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":66,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:21.305863+0000","last_change":"2023-11-28T11:12:21.305802+0000","last_active":"2023-11-28T11:12:21.305863+0000","last_peered":"2023-11-28T11:12:21.305863+0000","last_clean":"2023-11-28T11:12:21.305863+0000","last_became_active":"2023-11-28T11:08:11.958160+0000","last_became_peered":"2023-11-28T11:08:11.958160+0000","last_unstale":"2023-11-28T11:12:21.305863+0000","last_undegraded":"2023-11-28T11:12:21.305863+0000","last_fullsized":"2023-11-28T11:12:21.305863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:21.305694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:21.305694+0000","last_clean_scrub_stamp":"2023-11-28T11:12:21.305694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:18.988161+0000","scrub_duration":0.0035656680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11124,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":9728,"kb_avail":94360716,"statfs":{"total":96636764160,"available":96625373184,"internally_reserved":0,"allocated":1421312,"data_stored":956482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9961472},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.32000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.498},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55200000000000005,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":1.006,"15min":1.006},"last":0.61599999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67100000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.46800000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57199999999999995}]},{"osd":2,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60799999999999998,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.0640000000000001,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73999999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.58799999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71399999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.65600000000000003,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.751,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71199999999999997,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.46700000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.57399999999999995,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78500000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.436},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52000000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61699999999999999,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84699999999999998,"5min":1.018,"15min":1.018},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.435,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":1.123,"15min":1.123},"last":0.55500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9584,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362256,"statfs":{"total":96636764160,"available":96626950144,"internally_reserved":0,"allocated":827392,"data_stored":366114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.62,"15min":0.71899999999999997},"min":{"1min":0.36799999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70299999999999996,"5min":5.1239999999999997,"15min":5.1239999999999997},"last":0.5},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.63100000000000001,"15min":0.70999999999999996},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.53800000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.69299999999999995,"15min":0.81899999999999995},"min":{"1min":0.435,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70099999999999996,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.65700000000000003,"15min":0.76700000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68700000000000006,"5min":5.1509999999999998,"15min":5.1509999999999998},"last":0.39000000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.65800000000000003,"15min":0.79200000000000004},"min":{"1min":0.376,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69999999999999996,"5min":6.0620000000000003,"15min":6.0620000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.625,"15min":0.69499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67100000000000004,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.42899999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.39900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72399999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60699999999999998,"15min":0.66100000000000003},"min":{"1min":0.37,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67400000000000004,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.42199999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.51000000000000001,"15min":0.52300000000000002},"min":{"1min":0.33900000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.58699999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.55300000000000005,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.63700000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.34000000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59899999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.371},{"interface":"front","average":{"1min":0.505,"5min":0.58399999999999996,"15min":0.59899999999999998},"min":{"1min":0.41499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74099999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.55400000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76600000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.41699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68700000000000006,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13936,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357904,"statfs":{"total":96636764160,"available":96622493696,"internally_reserved":0,"allocated":827392,"data_stored":366114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.47699999999999998,"15min":0.60499999999999998},"min":{"1min":0.23499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.57099999999999995,"5min":7.0519999999999996,"15min":7.0519999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.36899999999999999,"15min":0.379},"min":{"1min":0.214,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54800000000000004,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.318}]},{"osd":2,"last update":"Tue Nov 28 11:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.51400000000000001,"15min":0.64700000000000002},"min":{"1min":0.30299999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59499999999999997,"5min":7.0890000000000004,"15min":7.0890000000000004},"last":0.374},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40600000000000003,"15min":0.41599999999999998},"min":{"1min":0.29099999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.69199999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.44400000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42299999999999999,"15min":0.433},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.624,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.377,"15min":0.375},"min":{"1min":0.26400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54500000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.35399999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.96399999999999997,"15min":1.3149999999999999},"min":{"1min":0.41699999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76600000000000001,"5min":18.024999999999999,"15min":18.024999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.44400000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.746,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.46899999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83099999999999996,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.98599999999999999,"15min":1.3420000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.877,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.50600000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.997,"15min":1.349},"min":{"1min":0.45300000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.91800000000000004,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89200000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90600000000000003,"5min":1.018,"15min":1.018},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93899999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.49299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14964,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356876,"statfs":{"total":96636764160,"available":96621441024,"internally_reserved":0,"allocated":1421312,"data_stored":956482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35099999999999998,"15min":0.35199999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.502,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.24199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.40899999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.40000000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.52000000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.379,"15min":0.378},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.59199999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.47599999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.376,"15min":0.37},"min":{"1min":0.30199999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58299999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.41799999999999998,"15min":0.42199999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61299999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.34599999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.47699999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79900000000000004,"5min":0.879,"15min":0.879},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.47699999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86699999999999999,"5min":0.878,"15min":0.878},"last":0.501}]},{"osd":5,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58399999999999996,"15min":0.56100000000000005},"min":{"1min":0.45100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.627,"15min":0.626},"min":{"1min":0.496,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.48999999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.497,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.48499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88200000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.629,"15min":0.623},"min":{"1min":0.51400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.83299999999999996,"5min":1.026,"15min":1.026},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.52800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.623}]}]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9712,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362128,"statfs":{"total":96636764160,"available":96626819072,"internally_reserved":0,"allocated":827392,"data_stored":366114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.32200000000000001},"min":{"1min":0.24299999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.435,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.373},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.53700000000000003,"5min":1.131,"15min":1.131},"last":0.32900000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.372,"15min":0.34200000000000003},"min":{"1min":0.27500000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.55900000000000005,"5min":1.151,"15min":1.151},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.373,"15min":0.34499999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60499999999999998,"5min":1.177,"15min":1.177},"last":0.376}]},{"osd":3,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35199999999999998,"15min":0.33800000000000002},"min":{"1min":0.28999999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58099999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.314},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.40500000000000003,"15min":0.377},"min":{"1min":0.28799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59399999999999997,"5min":1.109,"15min":1.109},"last":0.433}]},{"osd":4,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.51600000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88400000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.433,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90200000000000002,"5min":1.502,"15min":1.502},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.498,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78600000000000003,"5min":1.323,"15min":1.323},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82199999999999995,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.61599999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60199999999999998,"15min":0.57999999999999996},"min":{"1min":0.48499999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89300000000000002,"5min":1.464,"15min":1.464},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.48999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":1.355,"15min":1.355},"last":0.46800000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.46600000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85199999999999998,"5min":1.421,"15min":1.421},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.47399999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79600000000000004,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.57899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14064,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357776,"statfs":{"total":96636764160,"available":96622362624,"internally_reserved":0,"allocated":827392,"data_stored":366114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33800000000000002,"15min":0.34300000000000003},"min":{"1min":0.248,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.5,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.32200000000000001},"min":{"1min":0.27000000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44900000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.307}]},{"osd":1,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.25700000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47399999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.379},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39100000000000001,"15min":0.40500000000000003},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.45900000000000002,"5min":1.365,"15min":1.365},"last":0.33400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38,"15min":0.373},"min":{"1min":0.32400000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48799999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.40400000000000003},"min":{"1min":0.34000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56799999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.41299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.41399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77900000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.66100000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.75600000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.56799999999999995},"min":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.61399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79600000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.439,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.626}]},{"osd":7,"last update":"Tue Nov 28 11:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":9968,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361872,"statfs":{"total":96636764160,"available":96626556928,"internally_reserved":0,"allocated":827392,"data_stored":366114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77600000000000002,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60599999999999998,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71599999999999997,"5min":4.9740000000000002,"15min":4.9740000000000002},"last":0.59899999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.37,"15min":0.37},"max":{"1min":0.70499999999999996,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.84699999999999998,"15min":1.117},"min":{"1min":0.42999999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72999999999999998,"5min":12.263,"15min":12.263},"last":0.52100000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74199999999999999,"5min":1.524,"15min":1.524},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76600000000000001,"5min":1.236,"15min":1.236},"last":0.627}]},{"osd":3,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56999999999999995,"15min":0.54600000000000004},"min":{"1min":0.36499999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77700000000000002,"5min":1.54,"15min":1.54},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.66800000000000004,"15min":0.71299999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79100000000000004,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.65800000000000003,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.59699999999999998,"15min":0.55100000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71399999999999997,"5min":5.0220000000000002,"15min":5.0220000000000002},"last":0.34599999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71799999999999997,"5min":1.923,"15min":1.923},"last":0.38200000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68400000000000005,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.75,"5min":1.49,"15min":1.49},"last":0.49199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9776,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362064,"statfs":{"total":96636764160,"available":96626753536,"internally_reserved":0,"allocated":827392,"data_stored":366114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.628},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.53400000000000003},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69199999999999995,"5min":0.75,"15min":0.75},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Tue Nov 28 11:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.56299999999999994},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.49399999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78300000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.76000000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.47399999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83899999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48699999999999999,"15min":0.49399999999999999},"min":{"1min":0.33900000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.56299999999999994,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.55000000000000004,"15min":0.56100000000000005},"min":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.623,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.72399999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.38100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68000000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.60299999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.74399999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.375,"5min":0.374,"15min":0.374},"max":{"1min":0.72999999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88600000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:37.104 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-28T11:12:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.223+0000 7fd859d93700 1 Processor -- start 2023-11-28T11:12:37.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.223+0000 7fd859d93700 1 -- start start 2023-11-28T11:12:37.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.224+0000 7fd859d93700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd8540d0a00 0x7fd85415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.224+0000 7fd859d93700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85406b420 con 0x7fd8540ce040 2023-11-28T11:12:37.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.224+0000 7fd859d93700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85406b700 con 0x7fd8540b2930 2023-11-28T11:12:37.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.224+0000 7fd859d93700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85406ff60 con 0x7fd8540d0a00 2023-11-28T11:12:37.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.224+0000 7fd852ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd8540d0a00 0x7fd85415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.224+0000 7fd8537fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd8540ce040 0x7fd85406f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51768/0 (socket says 172.21.15.156:51768) 2023-11-28T11:12:37.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.224+0000 7fd8537fe700 1 -- 172.21.15.156:0/3452056252 learned_addr learned my addr 172.21.15.156:0/3452056252 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T11:12:37.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.224+0000 7fd852ffd700 1 -- 172.21.15.156:0/3452056252 >> v1:172.21.15.186:6789/0 conn(0x7fd8540b2930 legacy=0x7fd85406c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.224+0000 7fd852ffd700 1 -- 172.21.15.156:0/3452056252 >> v1:172.21.15.156:6789/0 conn(0x7fd8540ce040 legacy=0x7fd85406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.224+0000 7fd8527fc700 1 -- 172.21.15.156:0/3452056252 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629801637 0 0) 0x7fd85406b420 con 0x7fd8540ce040 2023-11-28T11:12:37.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.224+0000 7fd852ffd700 1 -- 172.21.15.156:0/3452056252 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85406b9e0 con 0x7fd8540d0a00 2023-11-28T11:12:37.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.224+0000 7fd852ffd700 1 --2- 172.21.15.156:0/3452056252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8540d0a00 0x7fd85415df10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd844009840 tx=0x7fd844009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a566583f2ec0760f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:37.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.225+0000 7fd8527fc700 1 -- 172.21.15.156:0/3452056252 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844011830 con 0x7fd8540d0a00 2023-11-28T11:12:37.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.225+0000 7fd8527fc700 1 -- 172.21.15.156:0/3452056252 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd844011990 con 0x7fd8540d0a00 2023-11-28T11:12:37.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.225+0000 7fd8527fc700 1 -- 172.21.15.156:0/3452056252 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844011d20 con 0x7fd8540d0a00 2023-11-28T11:12:37.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.225+0000 7fd859d93700 1 -- 172.21.15.156:0/3452056252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8540d0a00 msgr2=0x7fd85415df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.225+0000 7fd859d93700 1 --2- 172.21.15.156:0/3452056252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8540d0a00 0x7fd85415df10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd844009840 tx=0x7fd844009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:37.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.225+0000 7fd859d93700 1 -- 172.21.15.156:0/3452056252 shutdown_connections 2023-11-28T11:12:37.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.225+0000 7fd859d93700 1 --2- 172.21.15.156:0/3452056252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8540d0a00 0x7fd85415df10 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.225+0000 7fd859d93700 1 -- 172.21.15.156:0/3452056252 >> 172.21.15.156:0/3452056252 conn(0x7fd85406aed0 msgr2=0x7fd85406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:37.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.225+0000 7fd859d93700 1 -- 172.21.15.156:0/3452056252 shutdown_connections 2023-11-28T11:12:37.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.225+0000 7fd859d93700 1 -- 172.21.15.156:0/3452056252 wait complete. 2023-11-28T11:12:37.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.226+0000 7fd859d93700 1 Processor -- start 2023-11-28T11:12:37.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.227+0000 7fd859d93700 1 -- start start 2023-11-28T11:12:37.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.227+0000 7fd859d93700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8540b2930 0x7fd85414d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.227+0000 7fd859d93700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8540ce040 0x7fd8540c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.227+0000 7fd859d93700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8540d0a00 0x7fd8540ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.227+0000 7fd859d93700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540c9920 con 0x7fd8540b2930 2023-11-28T11:12:37.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.227+0000 7fd859d93700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540c8880 con 0x7fd8540d0a00 2023-11-28T11:12:37.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.227+0000 7fd859d93700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540c8b60 con 0x7fd8540ce040 2023-11-28T11:12:37.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.227+0000 7fd8537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8540b2930 0x7fd85414d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.227+0000 7fd8537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8540b2930 0x7fd85414d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36898/0 (socket says 172.21.15.156:36898) 2023-11-28T11:12:37.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.227+0000 7fd8537fe700 1 -- 172.21.15.156:0/35708738 learned_addr learned my addr 172.21.15.156:0/35708738 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:37.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd853fff700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8540d0a00 0x7fd8540ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd852ffd700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8540ce040 0x7fd8540c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd8537fe700 1 -- 172.21.15.156:0/35708738 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8540ce040 msgr2=0x7fd8540c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd8537fe700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8540ce040 0x7fd8540c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd8537fe700 1 -- 172.21.15.156:0/35708738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8540d0a00 msgr2=0x7fd8540ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd8537fe700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8540d0a00 0x7fd8540ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd8537fe700 1 -- 172.21.15.156:0/35708738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8440094a0 con 0x7fd8540b2930 2023-11-28T11:12:37.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd853fff700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8540d0a00 0x7fd8540ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:37.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd8537fe700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8540b2930 0x7fd85414d2a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd84800c730 tx=0x7fd84800caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:37.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd83bfff700 1 -- 172.21.15.156:0/35708738 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480074e0 con 0x7fd8540b2930 2023-11-28T11:12:37.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd83bfff700 1 -- 172.21.15.156:0/35708738 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd848007660 con 0x7fd8540b2930 2023-11-28T11:12:37.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd859d93700 1 -- 172.21.15.156:0/35708738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85406bfa0 con 0x7fd8540b2930 2023-11-28T11:12:37.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd83bfff700 1 -- 172.21.15.156:0/35708738 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848005a70 con 0x7fd8540b2930 2023-11-28T11:12:37.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.228+0000 7fd859d93700 1 -- 172.21.15.156:0/35708738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85406bcc0 con 0x7fd8540b2930 2023-11-28T11:12:37.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.229+0000 7fd859d93700 1 -- 172.21.15.156:0/35708738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8540b2db0 con 0x7fd8540b2930 2023-11-28T11:12:37.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.229+0000 7fd83bfff700 1 -- 172.21.15.156:0/35708738 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd848019030 con 0x7fd8540b2930 2023-11-28T11:12:37.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.230+0000 7fd83bfff700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd840068320 0x7fd84006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.230+0000 7fd83bfff700 1 -- 172.21.15.156:0/35708738 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd848084de0 con 0x7fd8540b2930 2023-11-28T11:12:37.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.230+0000 7fd852ffd700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd840068320 0x7fd84006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.230+0000 7fd852ffd700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd840068320 0x7fd84006a7d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd8440090b0 tx=0x7fd84401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:37.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.234+0000 7fd83bfff700 1 -- 172.21.15.156:0/35708738 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd8480518b0 con 0x7fd8540b2930 2023-11-28T11:12:37.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.417+0000 7fd859d93700 1 -- 172.21.15.156:0/35708738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fd85404e720 con 0x7fd8540b2930 2023-11-28T11:12:37.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.421+0000 7fd83bfff700 1 -- 172.21.15.156:0/35708738 <== mon.0 v2:172.21.15.156:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd8480518b0 con 0x7fd8540b2930 2023-11-28T11:12:37.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.421+0000 7fd83bfff700 1 -- 172.21.15.156:0/35708738 <== mon.0 v2:172.21.15.156:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd848017070 con 0x7fd8540b2930 2023-11-28T11:12:37.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.424+0000 7fd859d93700 1 -- 172.21.15.156:0/35708738 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd840068320 msgr2=0x7fd84006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.424+0000 7fd859d93700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd840068320 0x7fd84006a7d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd8440090b0 tx=0x7fd84401c040 comp rx=0 tx=0).stop 2023-11-28T11:12:37.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.424+0000 7fd859d93700 1 -- 172.21.15.156:0/35708738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8540b2930 msgr2=0x7fd85414d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.424+0000 7fd859d93700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8540b2930 0x7fd85414d2a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd84800c730 tx=0x7fd84800caf0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.424+0000 7fd859d93700 1 -- 172.21.15.156:0/35708738 shutdown_connections 2023-11-28T11:12:37.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.424+0000 7fd859d93700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8540d0a00 0x7fd8540ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.424+0000 7fd859d93700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8540b2930 0x7fd85414d2a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.424+0000 7fd859d93700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fd840068320 0x7fd84006a7d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.424+0000 7fd859d93700 1 --2- 172.21.15.156:0/35708738 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8540ce040 0x7fd8540c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.424+0000 7fd859d93700 1 -- 172.21.15.156:0/35708738 >> 172.21.15.156:0/35708738 conn(0x7fd85406aed0 msgr2=0x7fd8540cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:37.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.424+0000 7fd859d93700 1 -- 172.21.15.156:0/35708738 shutdown_connections 2023-11-28T11:12:37.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.424+0000 7fd859d93700 1 -- 172.21.15.156:0/35708738 wait complete. 2023-11-28T11:12:37.437 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:12:37.438 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:37.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.557+0000 7fef562ef700 1 Processor -- start 2023-11-28T11:12:37.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.557+0000 7fef562ef700 1 -- start start 2023-11-28T11:12:37.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.557+0000 7fef562ef700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fef5006ba90 0x7fef500af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.557+0000 7fef562ef700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef500cbdf0 0x7fef500ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.557+0000 7fef562ef700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef500cc5f0 con 0x7fef5015cef0 2023-11-28T11:12:37.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.557+0000 7fef562ef700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef5015d470 con 0x7fef5006ba90 2023-11-28T11:12:37.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.557+0000 7fef562ef700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5015d750 con 0x7fef500cbdf0 2023-11-28T11:12:37.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.558+0000 7fef5488c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef500cbdf0 0x7fef500ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.558+0000 7fef4f7fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fef5006ba90 0x7fef500af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.558+0000 7fef5488c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef500cbdf0 0x7fef500ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:40772/0 (socket says 172.21.15.156:40772) 2023-11-28T11:12:37.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.558+0000 7fef5488c700 1 -- 172.21.15.156:0/2560909589 learned_addr learned my addr 172.21.15.156:0/2560909589 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:37.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.558+0000 7fef4ffff700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fef5015cef0 0x7fef5006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:33910/0 (socket says 172.21.15.156:33910) 2023-11-28T11:12:37.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.558+0000 7fef5488c700 1 -- 172.21.15.156:0/2560909589 >> v2:172.21.15.186:3300/0 conn(0x7fef5006ba90 msgr2=0x7fef500af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:37.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.558+0000 7fef5488c700 1 --2- 172.21.15.156:0/2560909589 >> v2:172.21.15.186:3300/0 conn(0x7fef5006ba90 0x7fef500af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.558+0000 7fef5488c700 1 -- 172.21.15.156:0/2560909589 >> v1:172.21.15.186:6789/0 conn(0x7fef5015cef0 legacy=0x7fef5006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.558+0000 7fef5488c700 1 -- 172.21.15.156:0/2560909589 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5015da30 con 0x7fef500cbdf0 2023-11-28T11:12:37.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.558+0000 7fef4f7fe700 1 --2- 172.21.15.156:0/2560909589 >> v2:172.21.15.186:3300/0 conn(0x7fef5006ba90 0x7fef500af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:37.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.558+0000 7fef5488c700 1 --2- 172.21.15.156:0/2560909589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef500cbdf0 0x7fef500ca650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fef48009e50 tx=0x7fef4800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aaf6e13fc0b7166 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:37.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.558+0000 7fef4effd700 1 -- 172.21.15.156:0/2560909589 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4800e070 con 0x7fef500cbdf0 2023-11-28T11:12:37.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.559+0000 7fef4effd700 1 -- 172.21.15.156:0/2560909589 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fef4800b910 con 0x7fef500cbdf0 2023-11-28T11:12:37.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.559+0000 7fef4effd700 1 -- 172.21.15.156:0/2560909589 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4800bce0 con 0x7fef500cbdf0 2023-11-28T11:12:37.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.559+0000 7fef562ef700 1 -- 172.21.15.156:0/2560909589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef500cbdf0 msgr2=0x7fef500ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.559+0000 7fef562ef700 1 --2- 172.21.15.156:0/2560909589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef500cbdf0 0x7fef500ca650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fef48009e50 tx=0x7fef4800b330 comp rx=0 tx=0).stop 2023-11-28T11:12:37.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.559+0000 7fef562ef700 1 -- 172.21.15.156:0/2560909589 shutdown_connections 2023-11-28T11:12:37.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.559+0000 7fef562ef700 1 --2- 172.21.15.156:0/2560909589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef500cbdf0 0x7fef500ca650 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.559+0000 7fef562ef700 1 --2- 172.21.15.156:0/2560909589 >> v2:172.21.15.186:3300/0 conn(0x7fef5006ba90 0x7fef500af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.559+0000 7fef562ef700 1 -- 172.21.15.156:0/2560909589 >> 172.21.15.156:0/2560909589 conn(0x7fef500d0000 msgr2=0x7fef500d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:37.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.559+0000 7fef562ef700 1 -- 172.21.15.156:0/2560909589 shutdown_connections 2023-11-28T11:12:37.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.559+0000 7fef562ef700 1 -- 172.21.15.156:0/2560909589 wait complete. 2023-11-28T11:12:37.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.560+0000 7fef562ef700 1 Processor -- start 2023-11-28T11:12:37.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.560+0000 7fef562ef700 1 -- start start 2023-11-28T11:12:37.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.561+0000 7fef562ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef5006ba90 0x7fef500b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.561+0000 7fef562ef700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef500cbdf0 0x7fef500b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.561+0000 7fef562ef700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef5015cef0 0x7fef500b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.561+0000 7fef562ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef500b5d20 con 0x7fef5006ba90 2023-11-28T11:12:37.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.561+0000 7fef562ef700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef500c4090 con 0x7fef500cbdf0 2023-11-28T11:12:37.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.561+0000 7fef562ef700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef500c4370 con 0x7fef5015cef0 2023-11-28T11:12:37.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.561+0000 7fef4ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef5006ba90 0x7fef500b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.561+0000 7fef4ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef5006ba90 0x7fef500b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36910/0 (socket says 172.21.15.156:36910) 2023-11-28T11:12:37.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.561+0000 7fef4ffff700 1 -- 172.21.15.156:0/2236456601 learned_addr learned my addr 172.21.15.156:0/2236456601 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T11:12:37.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.561+0000 7fef4f7fe700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef500cbdf0 0x7fef500b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.562+0000 7fef5488c700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef5015cef0 0x7fef500b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.562+0000 7fef4ffff700 1 -- 172.21.15.156:0/2236456601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef5015cef0 msgr2=0x7fef500b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.562+0000 7fef4ffff700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef5015cef0 0x7fef500b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.562+0000 7fef4ffff700 1 -- 172.21.15.156:0/2236456601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef500cbdf0 msgr2=0x7fef500b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.562+0000 7fef4ffff700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef500cbdf0 0x7fef500b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.562+0000 7fef4ffff700 1 -- 172.21.15.156:0/2236456601 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef440094a0 con 0x7fef5006ba90 2023-11-28T11:12:37.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.562+0000 7fef4ffff700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef5006ba90 0x7fef500b3060 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fef4000c500 tx=0x7fef4000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:37.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.562+0000 7fef4cff9700 1 -- 172.21.15.156:0/2236456601 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40005cb0 con 0x7fef5006ba90 2023-11-28T11:12:37.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.562+0000 7fef4cff9700 1 -- 172.21.15.156:0/2236456601 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fef40005e10 con 0x7fef5006ba90 2023-11-28T11:12:37.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.562+0000 7fef562ef700 1 -- 172.21.15.156:0/2236456601 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef48009b30 con 0x7fef5006ba90 2023-11-28T11:12:37.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.562+0000 7fef4cff9700 1 -- 172.21.15.156:0/2236456601 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef400072f0 con 0x7fef5006ba90 2023-11-28T11:12:37.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.562+0000 7fef562ef700 1 -- 172.21.15.156:0/2236456601 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5015dff0 con 0x7fef5006ba90 2023-11-28T11:12:37.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.563+0000 7fef562ef700 1 -- 172.21.15.156:0/2236456601 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef5015dd10 con 0x7fef5006ba90 2023-11-28T11:12:37.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.563+0000 7fef4cff9700 1 -- 172.21.15.156:0/2236456601 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fef40019030 con 0x7fef5006ba90 2023-11-28T11:12:37.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.564+0000 7fef4cff9700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fef3c0682f0 0x7fef3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:37.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.564+0000 7fef4f7fe700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fef3c0682f0 0x7fef3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:37.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.564+0000 7fef4cff9700 1 -- 172.21.15.156:0/2236456601 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fef40084b90 con 0x7fef5006ba90 2023-11-28T11:12:37.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.564+0000 7fef4f7fe700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fef3c0682f0 0x7fef3c06a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fef44009d70 tx=0x7fef44009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:37.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.568+0000 7fef4cff9700 1 -- 172.21.15.156:0/2236456601 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fef400516f0 con 0x7fef5006ba90 2023-11-28T11:12:37.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.796+0000 7fef562ef700 1 -- 172.21.15.156:0/2236456601 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef500cab90 con 0x7fef5006ba90 2023-11-28T11:12:37.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.797+0000 7fef4cff9700 1 -- 172.21.15.156:0/2236456601 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fef40017020 con 0x7fef5006ba90 2023-11-28T11:12:37.798 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T11:12:37.799 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/206517987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6833","nonce":206517987},{"type":"v1","addr":"172.21.15.186:6835","nonce":206517987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6006,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1991065152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1991065152},{"type":"v1","addr":"172.21.15.156:6836","nonce":1991065152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/43543030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":43543030},{"type":"v1","addr":"172.21.15.156:6837","nonce":43543030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6834/2480189069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2480189069},{"type":"v1","addr":"172.21.15.186:6834","nonce":2480189069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-11-28T11:12:37.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.799+0000 7fef562ef700 1 -- 172.21.15.156:0/2236456601 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fef3c0682f0 msgr2=0x7fef3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.799+0000 7fef562ef700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fef3c0682f0 0x7fef3c06a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fef44009d70 tx=0x7fef44009210 comp rx=0 tx=0).stop 2023-11-28T11:12:37.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.799+0000 7fef562ef700 1 -- 172.21.15.156:0/2236456601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef5006ba90 msgr2=0x7fef500b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:37.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.799+0000 7fef562ef700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef5006ba90 0x7fef500b3060 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fef4000c500 tx=0x7fef4000c810 comp rx=0 tx=0).stop 2023-11-28T11:12:37.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.799+0000 7fef562ef700 1 -- 172.21.15.156:0/2236456601 shutdown_connections 2023-11-28T11:12:37.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.799+0000 7fef562ef700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.156:6832/103293,v1:172.21.15.156:6833/103293] conn(0x7fef3c0682f0 0x7fef3c06a7a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.799+0000 7fef562ef700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef5015cef0 0x7fef500b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.799+0000 7fef562ef700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef5006ba90 0x7fef500b3060 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.799+0000 7fef562ef700 1 --2- 172.21.15.156:0/2236456601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef500cbdf0 0x7fef500b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:37.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.799+0000 7fef562ef700 1 -- 172.21.15.156:0/2236456601 >> 172.21.15.156:0/2236456601 conn(0x7fef500d0000 msgr2=0x7fef5006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:37.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.799+0000 7fef562ef700 1 -- 172.21.15.156:0/2236456601 shutdown_connections 2023-11-28T11:12:37.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T11:12:37.799+0000 7fef562ef700 1 -- 172.21.15.156:0/2236456601 wait complete. 2023-11-28T11:12:37.802 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 34 2023-11-28T11:12:37.813 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-28T11:12:37.813 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T11:12:37.813 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:12:37.813 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:37.822 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:12:37.822 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:12:37.822 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:37.871 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:12:37.871 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:12:37.871 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:37.883 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:12:37.883 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:12:37.883 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:37.935 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:12:37.935 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T11:12:37.935 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T11:12:37.935 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:37.966 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T11:12:37.966 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T11:12:37.966 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:38.034 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T11:12:38.035 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T11:12:38.035 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:38.112 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T11:12:38.113 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T11:12:38.113 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:38.188 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T11:12:38.188 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T11:12:38.188 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:38.235 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T11:12:38.235 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T11:12:38.235 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:38.273 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T11:12:38.296 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T11:12:38.296 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:38.354 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T11:12:38.355 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T11:12:38.355 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:38.401 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T11:12:38.401 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T11:12:38.401 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T11:12:38.402 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:38.426 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T11:12:38.427 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T11:12:38.427 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:38.451 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T11:12:38.451 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T11:12:38.452 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T11:12:38.452 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:38.478 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T11:12:38.479 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T11:12:38.479 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:38.503 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T11:12:38.503 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T11:12:38.503 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:12:38.555 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T11:12:38.555 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T11:12:38.556 DEBUG:teuthology.orchestra.run.smithi156:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-28T11:12:38.588 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:12:38.588 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T11:12:38.883 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:12:38.883 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T11:12:38.979 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:12:38.980 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T11:12:39.080 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:12:39.080 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T11:12:39.157 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T11:12:39.158 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T11:12:39.382 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T11:12:39.382 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T11:12:39.456 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T11:12:39.457 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T11:12:39.521 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T11:12:39.521 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T11:12:39.588 INFO:tasks.ceph:Archiving mon data... 2023-11-28T11:12:39.589 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956/data/mon.a.tgz 2023-11-28T11:12:39.589 DEBUG:teuthology.orchestra.run.smithi156:> mktemp 2023-11-28T11:12:39.606 INFO:teuthology.orchestra.run.smithi156.stdout:/tmp/tmp.rVKa0dG2JW 2023-11-28T11:12:39.607 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.rVKa0dG2JW 2023-11-28T11:12:39.752 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0666 /tmp/tmp.rVKa0dG2JW 2023-11-28T11:12:39.838 DEBUG:teuthology.orchestra.remote:smithi156:/tmp/tmp.rVKa0dG2JW is 384KB 2023-11-28T11:12:39.935 DEBUG:teuthology.orchestra.run.smithi156:> rm -fr /tmp/tmp.rVKa0dG2JW 2023-11-28T11:12:39.951 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956/data/mon.b.tgz 2023-11-28T11:12:39.952 DEBUG:teuthology.orchestra.run.smithi186:> mktemp 2023-11-28T11:12:39.969 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.tbyf2ANoWc 2023-11-28T11:12:39.970 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.tbyf2ANoWc 2023-11-28T11:12:40.155 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.tbyf2ANoWc 2023-11-28T11:12:40.244 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.tbyf2ANoWc is 505KB 2023-11-28T11:12:40.340 DEBUG:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.tbyf2ANoWc 2023-11-28T11:12:40.357 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956/data/mon.c.tgz 2023-11-28T11:12:40.357 DEBUG:teuthology.orchestra.run.smithi186:> mktemp 2023-11-28T11:12:40.413 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.lr1Q60UL1K 2023-11-28T11:12:40.413 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.lr1Q60UL1K 2023-11-28T11:12:40.574 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.lr1Q60UL1K 2023-11-28T11:12:40.658 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.lr1Q60UL1K is 505KB 2023-11-28T11:12:40.757 DEBUG:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.lr1Q60UL1K 2023-11-28T11:12:40.773 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T11:12:40.773 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T11:12:40.776 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T11:12:40.840 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T11:12:40.840 DEBUG:teuthology.orchestra.run.smithi156:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T11:12:40.848 DEBUG:teuthology.orchestra.run.smithi186:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T11:12:40.893 INFO:teuthology.orchestra.run.smithi156.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T11:12:40.917 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T11:12:40.922 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T11:12:40.924 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956/remote/smithi156/crash 2023-11-28T11:12:40.924 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T11:12:40.955 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956/remote/smithi186/crash 2023-11-28T11:12:40.956 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T11:12:40.988 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T11:12:40.988 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:41.000 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:41.079 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:41.107 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:41.133 INFO:tasks.ceph:Compressing logs... 2023-11-28T11:12:41.134 DEBUG:teuthology.orchestra.run.smithi156:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:12:41.149 DEBUG:teuthology.orchestra.run.smithi186:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:12:42.651 INFO:tasks.ceph:Archiving logs... 2023-11-28T11:12:42.651 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956/remote/smithi156/log 2023-11-28T11:12:42.652 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T11:12:43.135 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956/remote/smithi186/log 2023-11-28T11:12:43.136 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T11:12:43.353 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T11:12:43.364 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T11:12:43.365 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T11:12:43.367 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T11:12:43.481 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T11:12:43.671 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T11:12:43.671 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:12:43.678 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:12:43.691 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-11-28T11:12:43.696 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:12:43.696 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-11-28T11:12:43.697 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 11 +840us[ +840us] +/- 80ms 2023-11-28T11:12:43.697 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -662us[ -659us] +/- 54ms 2023-11-28T11:12:43.697 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +179us[ +179us] +/- 73ms 2023-11-28T11:12:43.698 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:12:43.698 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-11-28T11:12:43.703 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:12:43.703 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-11-28T11:12:43.703 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 12 +838us[ +857us] +/- 80ms 2023-11-28T11:12:43.703 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -654us[ -636us] +/- 54ms 2023-11-28T11:12:43.703 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +210us[ +229us] +/- 72ms 2023-11-28T11:12:43.704 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:12:43.705 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T11:12:43.725 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T11:12:43.725 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T11:12:43.739 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T11:12:43.851 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T11:12:43.947 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:12:43.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:12:43.983 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:12:44.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:12:44.017 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T11:12:44.029 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T11:12:44.052 INFO:teuthology.task.internal:Duration was 1034.448818 seconds 2023-11-28T11:12:44.052 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T11:12:44.077 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T11:12:44.077 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T11:12:44.082 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T11:12:44.125 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:12:44.130 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:12:44.594 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T11:12:44.594 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T11:12:44.595 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T11:12:44.619 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T11:12:44.619 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T11:12:44.643 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T11:12:44.643 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:12:44.661 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:12:44.725 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T11:12:44.725 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T11:12:44.745 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T11:12:44.842 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T11:12:44.852 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T11:12:44.852 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T11:12:44.879 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T11:12:44.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T11:12:44.920 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:12:44.923 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:12:44.947 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2023-11-28T11:12:44.972 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2023-11-28T11:12:44.986 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:12:45.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:12:45.017 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:12:45.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:12:45.043 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T11:12:45.059 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T11:12:45.060 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956/remote/smithi156 2023-11-28T11:12:45.060 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T11:12:45.104 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956/remote/smithi186 2023-11-28T11:12:45.105 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T11:12:45.145 INFO:teuthology.task.internal:Removing archive directory... 2023-11-28T11:12:45.146 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T11:12:45.148 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T11:12:45.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T11:12:45.212 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T11:12:45.212 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T11:12:45.245 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T11:12:45.246 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T11:12:45.248 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T11:12:45.262 INFO:teuthology.orchestra.run.smithi156.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 11:12 /home/ubuntu/cephtest 2023-11-28T11:12:45.265 INFO:teuthology.orchestra.run.smithi186.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 11:12 /home/ubuntu/cephtest 2023-11-28T11:12:45.266 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T11:12:45.277 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T11:12:45.302 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T11:12:45.325 DEBUG:teuthology.nuke:shortname: smithi156 2023-11-28T11:12:45.325 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:12:45.347 DEBUG:teuthology.nuke:shortname: smithi186 2023-11-28T11:12:45.348 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:12:45.351 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:49:56.503330', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T11:12:45.371 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469956', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:49:56.501948', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T11:12:45.404 INFO:teuthology.orchestra.console.smithi156:Power off 2023-11-28T11:12:45.405 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T11:12:45.420 INFO:teuthology.orchestra.console.smithi186:Power off 2023-11-28T11:12:45.420 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T11:12:45.433 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2023-11-28T11:12:45.433 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:12:45.446 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2023-11-28T11:12:45.446 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:12:45.457 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-11-28T11:12:45.493 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-11-28T11:12:49.458 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:12:49.494 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:12:49.569 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-11-28T11:12:49.606 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-11-28T11:12:53.572 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:12:53.607 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:12:53.683 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2023-11-28T11:12:53.718 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2023-11-28T11:12:53.785 INFO:teuthology.orchestra.console.smithi156:Power off completed 2023-11-28T11:12:53.819 INFO:teuthology.orchestra.console.smithi186:Power off completed 2023-11-28T11:12:53.925 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 1034.448818206787 failure_reason: 'Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=66a04596572b4ca59ca9f6957153d158 status: fail success: false 2023-11-28T11:12:53.925 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T11:12:54.018 INFO:teuthology.run:FAIL